Image detection methods, apparatus, devices and storage media

Abstract

This disclosure provides an image detection method, apparatus, device, and storage medium, relating to the field of computer technology, and particularly to the fields of image recognition, forgery detection, image forgery, and artificial intelligence. The specific implementation scheme is as follows: A first sampling is performed on the input image to obtain a first sampling result set; a second sampling is performed based on the last sampling result in the first sampling result set to obtain a second sampling result set; based on the difference between the first sampling result set and the second sampling result set, the detection result of the input image is obtained. According to this disclosure, by comparing the differences in the sampling result sets under different sampling methods, a more accurate detection result for the input image can be obtained.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of image recognition, forgery recognition, image forgery, and artificial intelligence. Background Technology

[0002] With the development of artificial intelligence technology, generative models such as deepfakes and generative adversarial networks (GANs) can generate and forge facial images in a realistic way. While these technologies have brought tremendous innovation opportunities to media production, film and television industries, and other fields, they may also lead to the spread of misinformation. Summary of the Invention

[0003] This disclosure provides an image detection method, apparatus, device, and storage medium.

[0004] According to one aspect of this disclosure, an image detection method is provided, comprising:

[0005] The first sampling is performed on the input image to obtain the first sampling result set;

[0006] A second sampling is performed based on the last sampling result in the first sampling result set to obtain the second sampling result set.

[0007] Based on the difference between the first sampling result set and the second sampling result set, the detection result of the input image is obtained.

[0008] According to another aspect of this disclosure, a method for training an image forgery detection model is provided, comprising:

[0009] The image forgery detection model to be trained is trained using training samples to perform conditional generation training, and the conditional generation results are obtained.

[0010] Using the training samples, the image forgery detection model to be trained is unconditionally generated and trained to obtain unconditionally generated results.

[0011] Based on the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result, the image forgery detection model to be trained is adjusted to obtain the trained image forgery detection model.

[0012] According to another aspect of this disclosure, an image detection apparatus is provided, comprising:

[0013] The first sampling module is used to perform a first sampling based on the input image to obtain a first sampling result set;

[0014] The second sampling module is used to perform a second sampling based on the last sampling result in the first sampling result set to obtain a second sampling result set.

[0015] The detection module is used to obtain the detection result of the input image based on the difference between the first sampling result set and the second sampling result set.

[0016] According to another aspect of this disclosure, a training apparatus for an image forgery detection model is provided, comprising:

[0017] The conditional generation module is used to train the image forgery detection model to be trained using training samples, and to obtain the conditional generation results.

[0018] The unconditional generation module is used to perform unconditional generation training on the image forgery detection model to be trained using the training samples, and obtain the unconditional generation results.

[0019] The training module is used to adjust the image forgery detection model to be trained based on the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result, so as to obtain the trained image forgery detection model.

[0020] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0021] At least one processor; and

[0022] The memory is communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0024] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0025] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0026] According to this disclosure, by comparing the differences in the sampling result sets under different sampling methods, more accurate detection results of the input image can be obtained.

[0027] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0028] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0029] Figure 1 This is a schematic flowchart of an image detection method according to an embodiment of the present disclosure;

[0030] Figure 2 This is a schematic flowchart of an image detection method according to another embodiment of the present disclosure;

[0031] Figure 3 This is a flowchart illustrating a training method for an image forgery detection model according to an embodiment of the present disclosure;

[0032] Figure 4 This is an application scenario diagram of a face forgery detection method according to an embodiment of the present disclosure;

[0033] Figure 5 This is a schematic flowchart of a face forgery detection method according to an embodiment of the present disclosure;

[0034] Figure 6 This is an application scenario diagram of reverse sampling according to an embodiment of the present disclosure;

[0035] Figure 7 This is an application scenario diagram of forward sampling according to an embodiment of this disclosure;

[0036] Figures 8a to 8c This is a diagram illustrating the comparison of the difference indicators based on this disclosure;

[0037] Figure 9 This is a schematic diagram of the structure of an image detection apparatus according to an embodiment of the present disclosure;

[0038] Figure 10 This is a schematic diagram of the structure of an image detection apparatus according to another embodiment of the present disclosure;

[0039] Figure 11 This is a schematic diagram of the structure of a training device for an image forgery detection model according to an embodiment of the present disclosure;

[0040] Figure 12 This is a schematic diagram of the structure of a training apparatus for an image forgery detection model according to another embodiment of the present disclosure;

[0041] Figure 13 This is a block diagram of an electronic device used to implement embodiments of the present disclosure. Detailed Implementation

[0042] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0043] Generative models may lead to the spread of misinformation in areas such as security, privacy, and public opinion guidance, and the spread of misinformation may have negative impacts.

[0044] Figure 1 This is a schematic flowchart of an image detection method 100 according to an embodiment of the present disclosure. In one embodiment, the method includes:

[0045] S101. Perform a first sampling based on the input image to obtain a first sampling result set;

[0046] S102. Perform a second sampling based on the last sampling result in the first sampling result set to obtain a second sampling result set;

[0047] S103. Based on the difference between the first sampling result set and the second sampling result set, the detection result of the input image is obtained.

[0048] In this embodiment, the input image may be real or fake. The input image may include input image features. Features may also be referred to as distributions, feature distributions, probability distributions, etc. Sampling can be performed on the input image from different directions to obtain multiple sampling results. The input image is determined to be fake by comparing these multiple sampling results. The input image may include face images, such as face images uploaded by users, face images recorded by image acquisition devices, face image frames from video streams, etc.

[0049] In this embodiment of the disclosure, a first sampling result set can be obtained by performing a first sampling based on the input image in a first direction. The first sampling result set may include features of the input image, intermediate sampling results, and the last sampling result. For example, the first sampling result set may include the sampling results of the input image at each time step in the first sampling process. The sampling result at the last time step of the first sampling process is the last sampling result in the first sampling result set. A second sampling result set can be obtained by performing a second sampling based on the last sampling result in the first sampling result set (which can be referred to as the second sampling input feature) in a second direction. The second sampling result set may include the second sampling input feature, intermediate sampling results, and the last sampling result. For example, the second sampling result set may include the sampling results of the last sampling result in the first sampling result set at each time step in the second sampling process. The sampling result at the last time step of the second sampling process is the last sampling result in the second sampling result set, which can also be understood as the sampling result of the regenerated image.

[0050] In some examples, the first and second samples can be two associated sampling methods, and the first and second directions can be two different sampling directions. For example, the first direction is reverse, and the first sample is a reverse sampling. Image noise (the result of the last reverse sampling) can be obtained by progressively degrading the image through reverse sampling. Alternatively, the first direction is forward, and the second sample is forward sampling. A regenerated image can be obtained by progressively recovering the image noise through forward sampling (e.g., it can be obtained by transforming the result of the last forward sampling).

[0051] In this embodiment of the disclosure, peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and learned perceptual image patch similarity index (LPIPS) can be calculated based on the first sampling result set and the second sampling result set. Then, the detection result of the input image is obtained by comprehensively comparing and judging the above indicators.

[0052] According to this disclosure, by comparing the differences in sampling result sets under different sampling methods, more accurate detection results for the input image can be obtained. For example, the detection results for the input image can include whether the image is genuine or fake. In the case that the input image is a forged image, the specific forgery technique used can also be determined, such as deepfake, face swapping, face-to-face (Face2Face), and face-swapping (FaceShifter).

[0053] Figure 2 This is a schematic flowchart of an image detection method 200 according to another embodiment of the present disclosure. Method 200 can be used to implement step S101 in image detection method 100. In one embodiment, the first sampling includes backsampling, performing a first sampling based on the input image to obtain a first sampling result set. S101 further includes:

[0054] S201. Input the input image and condition label into the image forgery detection model to obtain the conditional flow field that matches the condition label;

[0055] S202. Based on the conditional flow field that matches the conditional label, the input image is backsampled according to the reverse time step to obtain the backsampling result corresponding to each reverse time step.

[0056] In this embodiment of the disclosure, the image forgery detection model can be a flow matching generative model. The flow matching generative model can be constructed based on a flow matching algorithm. Flow matching generative models are a type of generative model based on probabilistic flow theory. The flow matching generative model can generate samples of the target distribution by learning a probability flow field from the initial distribution to the target distribution. The flow matching generative model can process the input features based on conditional flow matching (CFM), by defining a conditional flow field in the data space such that the flow field can match the conditional probability path from the initial distribution to the target distribution.

[0057] In this embodiment, the conditional label may include a real image label and a forgery technology label, etc. For example, the forgery technology label may include a Deepfake label, a FaceSwap label, a Face2Face label, a FaceShifter label, or other forgery technology labels, etc., and this disclosure is not limited thereto. The conditional label may also be called conditional information. This conditional information can be processed by an encoder to generate a high-dimensional representation (e.g., a vector), which can be used as a signal to guide the generation process. Based on the input image and the conditional label, a first sampling can be performed in a first direction using a conditional flow field matching the conditional label to obtain a first sampling result set. Based on the last sampling result in the first sampling result set and the conditional label, a second sampling can be performed in a second direction to obtain a second sampling result set.

[0058] In this embodiment, the conditional flow field may also be referred to as a conditional guided vector field, a conditional probabilistic flow field, etc. The conditional flow field may include the sampling function corresponding to the time step. If the first sampling is a backsampling based on a flow matching generation model, the conditional flow field may include v(z)t ,t;c). And the reverse flow can be achieved at each time step t using the following formula:

[0059]

[0060] Among them, z t The input image can be represented by its latent representation or its distribution at each time step; t can represent the sampling time step. For example, t can range from T to 0, with T greater than 0; c can represent the conditional label. v() can be implemented using a neural network. During training, this neural network can learn the target image distribution x from the initial noise z0. In the backsampling process, the time step can be called the backsampling time step. As the backsampling time step t gradually decreases from T to 0, the latent representation z of the input image... t It can gradually degenerate into Gaussian noise. During this process, the condition label 'c' can guide the image to degenerate along the path corresponding to that condition label, simulating the change of the image from its current state to standard Gaussian noise.

[0061] In this embodiment of the disclosure, the backsampling results corresponding to each inverse time step output by the image forgery detection model can constitute a set of overall backsampling results. This set of backsampling results can also be referred to as a backsampling path, etc. Backsampling can include a process of progressively degrading image features in the input image until pure noise is obtained. For example, the inverse time step can start from t=T and progressively execute until t=0. Time step T corresponds to the features of the input image (or the features, initial distribution, etc. of the input image), time steps T-1 to 1 correspond to intermediate results of backsampling, and time step 0 corresponds to the last sampling result of backsampling. In this example, the time steps decrease by 1, but they can also decrease by other values, such as 0.1, 0.2, etc. The last time step can also be t=1, for example, the last sampling result is z1. The sampling results output by the image forgery detection model at each inverse time step can be used as the input of the model at the next inverse time step. For example, if the reverse time steps include 3, 2, 1, and 0, and the image forgery detection model outputs a distribution corresponding to time step 2 (e.g., Gaussian noise), then during backsampling at time step 1, the distribution corresponding to time step 2 can be used as input to the image forgery detection model to obtain the distribution corresponding to time step 1. Similarly, during backsampling at time step 0, the distribution corresponding to time step 1 can be used as input to the image forgery detection model.

[0062] In this embodiment of the disclosure, the input image can be backsampled according to the conditional flow field that matches the conditional label, thereby degrading the image features in the input image in the opposite direction of the image generation path corresponding to the selected forgery technique, and obtaining noise that can be processed by the forgery technique.

[0063] In one implementation, the second sampling includes forward sampling, such as... Figure 2 As shown, S102 performs a second sampling based on the sampling results in the first sampling result set to obtain a second sampling result set, including:

[0064] S203. Input the last backsampling result in the backsampling result set and the condition label into the image forgery detection model to obtain the conditional flow field that matches the condition label;

[0065] S204. Based on the conditional flow field that matches the conditional label, perform forward sampling on the back sampling result according to the forward time step to obtain the forward sampling result corresponding to each forward time step.

[0066] In this embodiment, the image forgery detection model, conditional labels, and conditional flow field can be found in the relevant descriptions in the above embodiments. If the second sampling is a forward sampling based on the flow matching generation model, forward flow can be performed at each time step t using the following formula:

[0067]

[0068] Among them, z t The values ​​of t and c can be found in the explanation of Formula 1 above. The sampling time step t can range from 0 to T. That is, the forward and reverse time steps can be opposite. For example, the reverse time step includes {3, 2, 1, 0}, and the forward time step includes {0, 1, 2, 3}. Similarly, the reverse time step includes {1, 0.8, 0.6, 0.4, 0.2, 0}, and the forward time step includes {0, 0.2, 0.4, 0.6, 0.8, 1}.

[0069] In this embodiment, the image forgery detection model can perform progressive sampling starting from the backsampling result according to the forward time step. The forward sampling result corresponding to each forward time step output by the image forgery detection model can constitute the overall forward sampling result set, which can also be called the forward sampling path, etc. Forward sampling can progressively sample pure noise, and the last sampling result of the forward sampling path can be converted to obtain the regenerated image. For example, the forward time step can start from t=0 and proceed progressively to t=T. The sampling result output by the image forgery detection model at each forward time step can be used as the input of the model at the next forward time step. For example, the forward time steps include 0, 1, 2, and 3. After the image forgery detection model outputs the distribution corresponding to time step 1, such as Gaussian noise, when performing forward sampling at time step 2, the distribution corresponding to time step 1 can be used as the input of the image forgery detection model to obtain the distribution corresponding to time step 2. When performing forward sampling at time step 3, the distribution corresponding to time step 2 can be used as the input of the image forgery detection model.

[0070] In this embodiment of the disclosure, the same conditional label can be used in both the forward and reverse sampling processes, thereby enabling the matching of relevant conditional flow fields. The conditional flow fields used for forward and reverse sampling can be the same, but the flow patterns can be opposite, as shown in Formulas 1 and 2 above. In some of the examples above, the forward time step of forward sampling increases, the forward time step of reverse sampling increases, and the forward and reverse time steps have a corresponding relationship. For example, forward and reverse time steps with the same value are corresponding. Based on the correspondence between the forward and reverse time steps, it is convenient to compare the characteristic differences between the forward and reverse sampling processes at the same time step.

[0071] In this embodiment of the disclosure, the input image can be forward sampled according to the conditional flow field that matches the conditional label, thereby adding image features to the noise according to the direction of the image generation path corresponding to the selected forgery technique, and generating an image obtained by the forgery technique based on the noise.

[0072] In one implementation, S204, based on the conditional flow field matching the conditional label, forward samples the backsampling result according to the forward time step, obtaining the forward sampling result corresponding to each forward time step, including:

[0073] Based on the conditional flow field that matches the conditional label, conditional forward sampling is performed on the last backward sampling result according to the forward time step to obtain the conditional forward sampling result corresponding to each forward time step.

[0074] Based on the stochastic conditional flow field, the last backward sampling result is unconditionally forward sampled according to the forward time step to obtain the unconditional forward sampling result corresponding to each forward time step;

[0075] The regenerated image is obtained from the last sample in the conditional positive sampling result set and the unconditional positive sampling result set.

[0076] In this embodiment of the disclosure, during the forward sampling process, conditional and unconditional forward sampling can be performed on the backsampling results. For example, the image forgery detection model performs conditional forward sampling on the backsampling results according to the conditional flow field matching the conditional label, obtaining conditional intermediate forward sampling results and conditional last forward sampling results. Alternatively, the conditional label can be cleared or ignored, and the image forgery detection model uses a randomly acquired conditional flow field to perform forward sampling on the backsampling results, obtaining unconditional intermediate forward sampling results and unconditional forward sampling results.

[0077] In this embodiment of the disclosure, the conditional and unconditional positive sampling results can be further processed to obtain a comprehensive positive sampling result, which can strengthen or weaken the influence of the conditional label on the generation process, thereby obtaining a more accurate regenerated image.

[0078] In one implementation, obtaining a regenerated image based on the last sampled result in the conditional positive sampling result set and the unconditional positive sampling result set includes:

[0079] Input the condition label into the image forgery detection model to obtain the guiding strength coefficient that matches the condition label;

[0080] The last sampled result in the conditional positive sampling result set and the unconditional positive sampling result set are combined according to the guiding intensity coefficient to obtain the regenerated image.

[0081] In this embodiment of the disclosure, the image forgery detection model can be a guided flow matching generation model. This model can also obtain a guided intensity coefficient that matches the conditional label based on the conditional label. The guided intensity coefficient is used to combine conditional and unconditional positive sampling results, and different conditional labels can correspond to different guided intensity coefficients.

[0082] In this embodiment of the disclosure, the image forgery detection model can combine conditional and unconditional positive sampling results based on a guiding strength coefficient that matches the conditional label. An example of one calculation method is as follows:

[0083] x t = (1+ω)·x cond ―ω·x uncond Formula 3

[0084] Where, x t ω can represent the regenerated image, and x can represent the guiding intensity coefficient. cond It can represent the image obtained by transforming the last conditionally positive sample result, x uncond It can represent the image obtained by unconditionally transforming the last sample result in the positive direction.

[0085] In one implementation, such as Figure 2 As shown, S103 obtains the detection result of the input image based on the difference between the first sampling result set and the second sampling result set, including:

[0086] S205. Based on the matching degree of the sampling results of the reverse sampling result set and the forward sampling result set under the target condition label at the associated time step, the detection result of the input image is obtained.

[0087] In this embodiment of the disclosure, the associated time step of the backsampling result set and the forward sampling result set may include time steps whose time step values ​​are the same during their sampling process. The matching degree can be calculated based on the backsampling results and forward sampling results at the same time step to obtain the detection result of the input image. For example, the matching degree can be calculated based on metrics such as PSNR, SSIM, and LPIPS.

[0088] For example, the formula for calculating PSNR can be built upon the mean squared error (MSE). For instance, MSE can represent the average of the sum of squares of the differences in pixel values ​​between two images. An example formula for PSNR is as follows:

[0089]

[0090] Here, MAX represents the maximum possible pixel value in the image. For example, for an 8-bit image, the pixel value range is 0 to 255, so MAX = 255. MSE is the mean squared error.

[0091] Generally speaking, the higher the PSNR value, the smaller the differences between the images. For example, the sampling results corresponding to the reverse time step 0 and the forward time step 0 can be converted into corresponding images, and then the PSNR can be calculated based on the converted images.

[0092] SSIM is a metric for measuring the structural similarity between two images. It emphasizes the structural information of the images and the perception of the human visual system, and can better reflect the subjective perception of image quality by the human eye. SSIM can compare the similarity of two images from several aspects, such as brightness, contrast, and structure. For example, it compares the sampling results of backsampling at multiple time steps 0, 1, 2, 3 and forward sampling at time steps 0, 1, 2, 3, converting them into corresponding images, and then calculating the SSIM based on the brightness, contrast, structure, etc. of the converted images.

[0093] LPIPS measures the similarity between two images based on high-dimensional features extracted by deep neural networks. For example, backsampling and forward sampling results at multiple time steps (0, 1, 2, 3) are converted into corresponding images, and then LPIPS is calculated based on the high-dimensional features extracted from the converted images by deep neural networks.

[0094] In this embodiment of the disclosure, by comparing the differences between the forward and reverse sampling result sets at the same time step, it is possible to determine whether an image is forged or to obtain techniques for image forgery. For an image, under the conditional flow field corresponding to the correct conditional label pair, the difference between the forward and reverse sampling results at the same time step is generally small, indicating a high degree of matching. Therefore, if the difference between the forward and reverse sampling results at the same time step is large, or the degree of matching is not high, then the image is not an image under that label.

[0095] In one implementation, the detection result of the input image is obtained based on the degree of matching between the backsampled result set and the forward sampled result set under the target condition label at the associated time step, including:

[0096] If one or more of the following comparison results are true, the input image is the image type corresponding to the target condition label:

[0097] The degree of matching between the intermediate sampling results of the reverse sampling result set and the forward sampling result set at the same time step under the target condition label is greater than the first threshold.

[0098] The feature matching degree between the input image of the backsampled result set and the last sampled result of the forward sampled result set under the target condition label is greater than the second threshold.

[0099] In this embodiment, the matching degree under the target condition label can be obtained by comparing the backsampling results and forward sampling results at the same time step under the conditional flow field corresponding to the target condition label. If the matching degree is higher than a certain threshold, the input image can be determined to be the image type corresponding to the target condition label. The start time step of forward sampling, for example, t=0, corresponds to the end time step of backsampling; the forward sampling result at time step t=0 is the backsampling result obtained at the end time step of backsampling. The start time step of backsampling, for example, t=T, corresponds to the end time step of forward sampling; the backsampling result at time step t=T is compared with the forward sampling result at forward sampling t=T, i.e., the features of the input image and the features of the regenerated image are compared. The first threshold and the second threshold can be the same or different. For example, the first threshold can be a large value between 0 and 1, and the second threshold can be 1. The values ​​of the above thresholds are merely examples and not limitations; other values ​​are also possible. If the comparison results satisfy the thresholds, the input image can be determined to conform to the target condition label, i.e., the input image is the image type corresponding to the target condition label.

[0100] In this embodiment of the disclosure, by comparing the forward and reverse sampling results of images under the same conditional flow field corresponding to the same condition label, it is possible to accurately determine whether the image is forged or whether a forgery technique is used.

[0101] In one implementation, the input image is an image type corresponding to the target condition label, including at least one of the following:

[0102] If the target condition label is a true label, then the input image is a true image;

[0103] If the target condition label is a target forgery technique label, the input image is an image generated using a target forgery technique.

[0104] In this embodiment of the disclosure, the target condition label may include a real label, a forgery technique label, etc. The image forgery detection model can detect the input image based on the real label, obtaining a set of forward sampling results (or forward sampling paths) and a set of backward sampling results (or backward sampling paths) under the real label. Then, the differences between the set of forward sampling results and the set of backward sampling results under the real label are compared. If the difference is small, for example, the matching degree is greater than a certain threshold, the input image can be determined to be a real image. Otherwise, the input image can be determined to be either not a real image or a forged image.

[0105] In this embodiment of the disclosure, the image forgery detection model can detect the input image based on one or more forgery technology tags, obtaining a forward sampling result set (or forward sampling path) and a backward sampling result set (or backward sampling path) under each forgery technology tag. Then, the difference between the forward sampling result set and the backward sampling result set under a certain target forgery technology tag is compared. If the difference is small, for example, the matching degree is greater than a certain threshold, the input image can be determined to be an image forged using the target forgery technology corresponding to the target forgery technology tag.

[0106] In this embodiment of the disclosure, if the comparison result meets the comparison condition under the conditional flow field that matches a certain condition label, that is, the feature matching degree of forward and reverse sampling is high, then it can be determined that the current image belongs to the image type under the current condition label.

[0107] Figure 3 This is a flowchart illustrating a training method 300 for an image forgery detection model according to an embodiment of the present disclosure. In one embodiment, the method 300 includes:

[0108] S301. Use training samples to perform conditional generation training on the image forgery detection model that needs to be trained, and obtain the conditional generation result;

[0109] S302. Use the training sample to perform unconditional generation training on the image forgery detection model that needs to be trained, and obtain the unconditional generation result.

[0110] S303. Based on the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result, adjust the image forgery detection model to be trained to obtain the trained image forgery detection model.

[0111] In this embodiment of the disclosure, the image forgery detection model can be a flow matching generation model. A flow matching generation model is a Continuous Normalizing Flow (CNF) model framework that can learn a continuous probability flow, transforming a simple prior distribution (such as a standard Gaussian distribution) into a complex data distribution.

[0112] In this embodiment of the disclosure, the training samples of the image forgery detection model may include images, corresponding conditional labels for the images, and corresponding noise. One image may correspond to multiple conditional labels and multiple noises. For example, a training sample may include an image, a true label, and true image noise. As another example, a training sample may include an image, a target forgery technique label, and image noise forged using the target forgery technique.

[0113] In this embodiment of the disclosure, during the model training process, a combination of conditional generation and unconditional generation can be used to balance the guidance of conditional information, such as conditional labels, with the learning of basic data structures.

[0114] Examples of conditional generation include image forgery detection models receiving specific conditional labels, such as whether the image is a real face (real label) or a forged face generated using a certain forgery technique (forgery technique label). The image forgery detection model uses this label information to guide the generation process. By learning the data distribution under different conditions, the model accurately generates images that match the conditional labels.

[0115] Examples of unconditional generation include situations where, in certain training steps, the model can ignore conditional labels and generate data solely based on the image data itself. This strategy allows the model to better learn the underlying distributional features of face images without relying on any specific conditions.

[0116] By switching between conditional and unconditional generation, the model can learn how to generate high-quality face images without a classifier or additional discriminant network, while ensuring that it can effectively distinguish between real and fake images.

[0117] In this embodiment, noise and conditional labels from the training samples are input into the image forgery detection model to be trained for forward sampling, and the model outputs a conditional generation result. The conditional generation loss is calculated based on the image from the training samples and the conditional generation result. Alternatively, noise from the training samples is input into the image forgery detection model to be trained for forward sampling, and an unconditional generation result is output. Or, noise and conditional labels from the training samples are input into the image forgery detection model to be trained; the model removes the conditional labels and performs forward sampling, outputting an unconditional generation result. The unconditional generation loss is calculated based on the image from the training samples and the unconditional generation result.

[0118] Within the flow matching generation framework, the training objective of the model can be to minimize the flow matching loss, thereby learning the flow field from the initial Gaussian noise distribution to the target face data distribution. The loss function in this embodiment can simultaneously consider the contributions of conditional and unconditional generation.

[0119] For example, the loss function can be a weighted sum of conditional generation loss and unconditional generation loss, as shown in the following formula:

[0120] L=λ cond ·L cond +λ uncond ·L uncond Formula 4

[0121] Among them, L cond L represents the conditional generation loss. uncond Represents the unconditional generation loss, λ cond and λ uncond Weights are used to balance conditional and unconditional generation. By adjusting these weights, the model can learn features of different forgery techniques while retaining its understanding of the overall structure of the face data.

[0122] In one implementation, the method further includes: performing interpolation calculations between the conditional generation result and the unconditional generation result to obtain a guided generation image.

[0123] In the disclosed embodiments, classifier-free guidance is used to improve the quality of generated samples, especially the ability to distinguish forgeries, by enhancing the strength of conditional generation. For example, interpolation is performed between conditional and unconditional generation to make the generated images more consistent with preset conditional labels.

[0124] For example, the guiding formula for generating samples can be:

[0125] x t = (1+ω)·x cond ―ω·x uncond

[0126] Where ω is the guiding intensity coefficient, x cond It is the result of conditional generation, x uncond This is an unconditionally generated result. By adjusting the value of ω, the consistency between the generated samples and the conditional information can be controlled. For example, choosing ω>1 can enhance conditional guidance, enabling the model to more accurately generate images that match forgery techniques, thereby improving the accuracy of forgery detection.

[0127] In this embodiment, by interpolating the conditional generation result and the unconditional generation result, a guided generation image can be obtained if the generated image meets the preset conditional label. An image forgery detection model can be used to process the guided generation image to obtain the corresponding conditional generation result and the unconditional generation result. Then, the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result are calculated respectively. For example, the conditional generation loss can be calculated based on the input noise of the conditional generation training and the image features output by the model's forward sampling, and the unconditional generation loss can be calculated based on the input noise of the unconditional generation training and the image features output by the model's forward sampling.

[0128] In this embodiment, referring to Formula 4, the loss value of the loss function is calculated based on the conditional generation loss and unconditional generation loss corresponding to the guided image. If the loss value does not meet the convergence condition, the model parameters are adjusted. After multiple iterations, the trained image forgery detection model can be obtained.

[0129] In this embodiment, an image forgery detection model can be trained to combine conditionally generated and unconditionally generated results according to the guidance intensity coefficients corresponding to the conditional labels, thereby obtaining guided generated images that conform to the characteristics of the conditional labels. This allows the image forgery detection model to match the corresponding guidance intensity coefficients based on the conditional labels, thus enabling forward sampling to obtain regenerated images that better conform to the label characteristics.

[0130] To address some issues in forged image detection, embodiments of this disclosure can provide corresponding solutions, as exemplified below:

[0131] 1. Insufficient accuracy in forged image detection: Face forgery detection methods often rely on fixed feature extraction or simple classifiers, which cannot cope with rapidly evolving image generation technologies. Generative models, such as Deepfake or GAN, can generate highly realistic forged images, and forgery techniques are constantly being updated and iterated, posing a significant challenge to traditional detection methods. The embodiments of this disclosure can utilize a flow matching generative model to model the potential distribution of face images, thereby improving the accuracy of forgery detection when dealing with complex forgery techniques.

[0132] 2. Inability to identify specific forgery techniques: Most current detection methods can only determine whether an image is forged, but cannot accurately identify the specific techniques used to forge the image. Different forgery methods generate images with different distribution characteristics. The embodiments of this disclosure can capture the feature distribution of different forgery methods through a conditional guidance mechanism and a flow matching generation model, thereby effectively identifying the specific forgery method.

[0133] 3. Lack of interpretability of detection results: Most forgery detection methods rely on black-box models, lacking the ability to interpret forged images and failing to clearly point out the details of the forgery. The embodiments of this disclosure, through comparison of forward and backward sampling paths, can demonstrate in detail the differences in images under different forgery conditions, making the detection results highly interpretable and helping users understand how images are forged and the methods used to forge them.

[0134] To address the aforementioned problems, the image detection method of this disclosure can detect the authenticity of images, such as facial images, and also identify their forgery techniques. The image detection method proposed in this disclosure may include a face forgery detection method based on a conditionally guided flow matching generation model. This method can provide an efficient and interpretable detection mechanism for complex forgery techniques (such as Deepfake, GAN forgery, image editing, etc.). By comparing the feature differences of images in the forward and backward sampling paths, it can not only distinguish between real and fake faces, but also further identify specific forgery techniques, providing technical support for combating misinformation and improving image security.

[0135] Figure 4 This diagram illustrates an application scenario of a face forgery detection method based on a conditionally guided flow matching generation model. By modeling real face images and face images generated using various forgery techniques, the conditionally guided flow matching generation model is trained to capture the latent distribution features of different face images. This method not only effectively distinguishes between real and forged faces but also provides strong support for identifying specific forgery techniques. It determines the authenticity of an image by comparing the differences between the forward and backward sampling paths, and further identifies the specific type of forgery technique based on these differences.

[0136] Figure 5 This is a flowchart illustrating a face forgery detection method 500 according to an embodiment of the present disclosure, which may include the following steps:

[0137] S501. Guided Stream Matching Generation Model Training: Train the stream matching generation model for images with different label types (such as real face images and fake face images generated using different forgery techniques). Each image type corresponds to a specific condition label, and the model learns the distribution of face data under these condition labels to obtain its latent features.

[0138] S502. Condition-guided backsampling process: Given a face image, different conditional labels (including labels for real images and various types of forged images) are set as guides. A flow matching model is used to progressively degrade the image into standard Gaussian noise that matches the corresponding conditional labels. This process can capture the feature differences of the image relative to each label type, regardless of whether the image originates from a real face or is generated using forgery techniques.

[0139] S503. Condition-guided forward sampling process: Guided by the same condition labels, a flow matching model is used to generate images of the corresponding type from standard Gaussian noise. This process simulates the path from Gaussian noise to generate real or fake faces, reflecting the specific data distribution represented by each condition label.

[0140] S504. Difference Comparison and Discrimination: By comparing the feature differences between the input image during the reverse degradation and forward generation processes under various conditions, it is determined whether the image conforms to the face data distribution corresponding to the set condition labels. If significant deviations are detected, these features can be used to determine whether the image is forged and further identify the type of forgery method.

[0141] The embodiments of this disclosure can fully utilize the reversibility and conditional guidance of the flow matching generation model, enabling accurate modeling of complex face image distributions. By comparing forward and backward sampling paths, this method can not only effectively detect the authenticity of face images, but also provide clear explanatory evidence for the identification of forgery techniques, such as deepfakes, generative adversarial network (GAN) forgeries, and image editing.

[0142] The main process of this disclosure embodiment is exemplified as follows:

[0143] (I) Training of Guided Stream Matching Generation Model

[0144] 1. Flow matching generation model

[0145] Suppose we have some data samples from a distribution of interest, whose density is unknown. If we want to use these samples to learn a probabilistic model to approximate that distribution, for example, to efficiently generate new samples that approximate that distribution, this task is called generative modeling.

[0146] Flow matching generative models are a new class of generative models based on probabilistic flow theory. These models can generate samples of a target distribution by learning a probability flow field from an initial distribution to a target distribution.

[0147] The main principle of flow matching generative models includes Conditional Flow Matching (CFM). This involves defining a conditional flow field in the data space that matches a conditional probability path from an initial distribution to a target distribution. The goal of flow matching is to optimize the flow field so that generated samples evolve along this conditional path, thereby reaching the target distribution. Compared to other generative models, flow matching models can directly utilize the analytical solution of the generation path without requiring complex adversarial training or variational inference processes, thus offering advantages in generation quality and stability. By introducing noise and backdiffusion processes, high-quality image generation has been successfully achieved, reaching state-of-the-art performance on multiple benchmark tasks.

[0148] 2. Guided Flow Matching Generation Model

[0149] Guided flow matching generative models are an extension of flow matching generative models, combining conditional generation and guidance techniques for more precise control of the generation process. Guided flow matching generative models can utilize external guidance information to instruct the generation model to generate data according to specific conditions, thereby improving the quality of generated samples or meeting specific generation requirements.

[0150] In guided flow matching generative models, the generation process is achieved by learning a conditional probabilistic flow field. This flow field depends not only on the current state (data distribution) but also on additional conditions (such as labels or specific task requirements). These conditions are embedded in the generative model, guiding the generation path to evolve from the initial distribution to the target distribution. The conditional information can be any form of external signal, such as category labels, text descriptions, or image features.

[0151] Furthermore, guided flow matching generative models can also be applied to tasks such as adversarial example detection, data augmentation, and density estimation. By introducing external guiding signals, the model can be made to model complex distributions more accurately and has stronger robustness and generalization ability.

[0152] 3. Model Training

[0153] In this disclosure, a conditionally guided flow matching generation model learns a probability flow field to progressively transform an initial Gaussian noise distribution into a target face data distribution. Combined with a classifier-free guidance mechanism, conditional information is introduced or removed during the generation process, thereby capturing the feature distributions of different types of real and fake faces. The model is trained using a weighted loss to ensure efficient differentiation and detection of different forgery techniques. For example, the model training process typically includes the following steps:

[0154] 1) Conditional generation and unconditional generation

[0155] During training, the model can employ a combination of conditional and unconditional generation methods to balance the guidance of conditional information with the learning of basic data structures.

[0156] a. Conditional Generation: The model receives specific conditional labels (e.g., whether the image is a real face or a fake face generated by a certain forgery technique) and uses this label information to guide the generation process. By learning the data distribution under different conditions, the model accurately generates images that match the conditional labels, ensuring that the model can capture the features of different forgery techniques.

[0157] b. Unconditional Generation: In some training steps, the model can intentionally ignore conditional labels and generate data solely based on the image data itself. This strategy allows the model to better learn the underlying distribution features of face images without relying on any specific conditions.

[0158] c. By switching between conditional and unconditional generation, the model can learn how to generate high-quality face images without a classifier or additional discriminative network, while ensuring that it can effectively distinguish between real and fake images.

[0159] 2) Training Objectives

[0160] Within the flow matching generation framework, the model's training objective is to minimize the flow matching loss, thereby learning the flow field from the initial Gaussian noise distribution to the target face data distribution. To ensure effective detection of various forgery techniques, the loss function is designed to consider the contributions of both conditional and unconditional generation.

[0161] For example, the loss function can be defined as a weighted sum of conditional generation and unconditional generation, as shown in the following example:

[0162] L=λ cond ·L cond +λuncond ·L uncond Formula 4

[0163] Among them, L cond L represents the conditional generation loss. uncond Represents the unconditional generation loss, λ cond and λ uncond The weights are used to balance conditional and unconditional generation. By adjusting the weights, the model can learn the features of different forgery techniques while retaining its understanding of the overall structure of the face data.

[0164] 3) Classifier-free generation

[0165] In the actual generation process, classifier-free guidance technology improves the quality of generated samples, especially the ability to distinguish forgeries, by enhancing the strength of conditional generation. Specifically, it involves interpolation between conditional and unconditional generation to make the generated images more consistent with preset conditional labels.

[0166] For example, the guiding formula for generating samples can be:

[0167] x t = (1+ω)·x cond ―ω·x uncond

[0168] Where ω is the guiding intensity coefficient, x cond It is the result of conditional generation, x uncond This is an unconditionally generated result. By adjusting the guidance intensity coefficient ω, the consistency between the generated sample and the conditional information can be controlled. Generally, choosing a guidance intensity coefficient greater than 1 can enhance conditional guidance, enabling the model to more accurately generate images that match forgery techniques, thereby improving the accuracy of forgery detection.

[0169] (II) Condition-guided reverse and forward sampling processes

[0170] 1. Backsampling

[0171] The goal of the backsampling process is to progressively degenerate the input face image into standard Gaussian noise using a flow matching model. By setting different condition labels (such as labels for real images and various types of fake images), the model can capture the potential feature differences between the image and each condition. By analyzing these differences, the model can determine the authenticity of the image and infer its possible forgery techniques. The specific steps are as follows:

[0172] 1) Setting the input image and condition labels

[0173] The backsampling process begins with a given input face image x. A conditional label is simultaneously set for the image; this label can be a "real image" or a specific label for various forgery techniques (such as Deepfake, FaceSwap, etc.). This conditional label guides the model to sample along the path associated with this label during the degradation process.

[0174] 2) Image projected into latent space

[0175] The input image x is mapped into the model's latent space, transforming it into the initial latent representation z. T , where T represents the starting time step of backsampling. This step represents the image as a latent variable within the model, providing the basis for subsequent flow matching operations.

[0176] 3) Condition-guided gradual degradation

[0177] The model performs a progressive degradation operation based on the flow matching equation, v(z) t c), reverse flow occurs at each time step t:

[0178]

[0179] As time t gradually decreases from T to 0, the latent representation z of the input image... t The image gradually degenerates into Gaussian noise. During this process, the condition label 'c' guides the image to degenerate along the path corresponding to that label, simulating the change of the image from its current state to standard Gaussian noise.

[0180] 4) Stepwise reduction of image features

[0181] The model is solved using an inverse ODE solver or discrete-time step. By continuously adjusting the flow field state, the input face image gradually degrades, and the high-level semantic features and detailed information of the image are gradually eliminated, replaced by noise components. Conditional guidance plays a crucial role in the backsampling process; different conditional labels 'c' correspond to different types of image generation paths. The image degradation behavior will differ under different conditional labels. For example, the degradation of a real image under a real label should be smooth, while the degradation of a fake image under a real label may exhibit abnormalities. Under a fake label, the degradation path of a fake image may better match the potential distribution of the fakery method.

[0182] 5) Degrades to standard Gaussian noise

[0183] When time t reaches 0, the image completely degenerates into standard Gaussian noise z0. This final state no longer contains specific image information, but rather reflects the latent distribution of the input image guided by the conditional label c. At this point, by comparing the degradation paths under different conditions, the model can extract information about the degree of matching between the image and various labels. Figure 6 This is an application scenario diagram of backsampling according to an embodiment of this disclosure. Through the backsampling process of the same input image under different conditions, it is possible to degrade an image to noise.

[0184] 2. Forward sampling

[0185] The forward sampling process starts from the initial noise distribution and gradually evolves into a face image that meets specific conditions through a flow matching model. The specific steps are as follows:

[0186] 1) Noise initialization

[0187] Sampling begins with a random noise vector z0 of a standard Gaussian distribution. This noise vector represents the initial state in the latent space, with dimensions consistent with those of the target image. At this point, no specific features of the image have been determined.

[0188] 2) Conditional Information Injection

[0189] At each time step t in the generation process, the model receives specific conditional information, which includes one or more of the following:

[0190] Image authenticity label: Indicates whether the image is "real" or "fake".

[0191] Forgery Technique Label: If the image is forged, specify the specific forgery technique category, such as "Deepfake forgery", "FaceSwap synthesis", "Face2Face tampering", etc.

[0192] This conditional information is processed by an encoder to generate a high-dimensional representation, which will be used as a signal to guide the generation process.

[0193] 3) Generate path guide

[0194] In the flow matching model, the generation process is guided by a pre-trained probabilistic flow field from the initial noise z0 to the target image distribution x. The flow field represents the evolution path of the image at t, and its changes are described by the following flow equation:

[0195]

[0196] At each time step, the model uses the conditional flow field v(z) t,t;c) Calculate how to update the noise vector z t The distribution of the final target face image is gradually approximated.

[0197] 4) Condition-guided adjustment

[0198] To ensure that the generated image remains consistent with the given conditional information, the model employs a classifier-free guidance mechanism, enhancing the effect of conditional guidance by adjusting the results of conditional and unconditional generation. Specifically, the model adjusts the conditional generation of x... cond and unconditionally generate x uncond Combine them:

[0199] x t = (1+ω)·x cond ―ω·x uncond

[0200] Here, ω is a coefficient that controls the intensity of the guidance. By adjusting ω, the model can strengthen or weaken the influence of conditional information on the generation process, making the output image match the given real / fake labels and the category of forgery techniques.

[0201] 5) Gradual evolution

[0202] By continuously adjusting the flow field state, the initial noise vector z t At each time step t, the model evolves towards the final target state. It solves the noise step-by-step into a latent representation of the face image using either an Ordinary Differential Equation (ODE) solver or discrete-time step solutions. Throughout this process, conditional information continuously influences the generation path, ensuring that the authenticity or forgery features of the output image are consistent with the input conditions.

[0203] 6) Generate image output

[0204] noise vector z t Ultimately, it evolves into a representation z1 in the latent space. This representation is then decoded into the final output image x. Due to the conditional guidance during the generation process, the output image is not only realistic, but its authenticity or forgery characteristics are highly consistent with the given conditions. Figure 7 This is an application scenario diagram of forward sampling according to an embodiment of this disclosure. By using Gaussian noise corresponding to different degradation conditions, a face image can be gradually generated through a forward sampling process guided by each condition.

[0205] (III) Comparison and Discrimination of Differences

[0206] 1. Sampling path difference analysis

[0207] By comparing the feature differences between the reverse degradation and forward generation processes of the input image under various conditions, the degree of matching between the image and the specific condition label can be revealed. This allows for the determination of whether the image conforms to the face data distribution corresponding to the set condition label. For example, the forward generation and reverse degradation paths of a real image under the "real image" condition are highly consistent; while a forged image may exhibit significant path deviations, especially under the real label, where the forward generation and reverse degradation paths deviate considerably. In this embodiment, commonly used image quality assessment metrics such as PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity Index), and LPIPS (Perceptual Similarity Index) can be used as indicators for analyzing the differences in generated image samples along the sampling path.

[0208] 2. Image authenticity determination

[0209] Path matching degree: In the comparative analysis, the sampling path of the input image is first evaluated to see if it matches the given condition labels. The path of the real image should show consistency in both forward and backward sampling, while the path of the fake image may show anomalies under certain conditions.

[0210] Dissimilarity measurement: This bias is quantified by calculating the distance or feature difference between the forward and reverse paths. If the path difference exceeds a certain threshold, it can be inferred that the image does not conform to the distribution under the corresponding condition label and may be a forged image.

[0211] 3. Identification of counterfeiting methods

[0212] Different forgery techniques are labeled, and the reverse and forward sampling paths are compared for each technique. By comparing the path performance of the input image under different forgery labels, it is possible to further determine which forgery technique generated the image. For example, if the path performance of the input image under a specific forgery label matches the distribution of that label more closely, the model can infer that the image was generated using that forgery technique.

[0213] 4. Comprehensive Judgment Mechanism

[0214] Finally, by combining the differences in forward and reverse paths of the input image under various condition labels, the authenticity and forgery methods are comprehensively evaluated. Figures 8a to 8c This is a schematic diagram comparing the difference indicators according to this disclosure. It compares the difference indicators of the input image during reverse degradation and forward generation processes under different conditions. Each condition label corresponds to an indicator curve. From these curves, it can be seen that the input image has better PSNR, SSIM, and LPIPS values ​​under the "real face" label, indicating that the input image is a real face.

[0215] Figure 9This is a schematic diagram of the structure of an image detection apparatus 900 according to an embodiment of the present disclosure. In one embodiment, the apparatus includes:

[0216] The first sampling module 901 is used to perform a first sampling based on the input image to obtain a first sampling result set;

[0217] The second sampling module 902 is used to perform a second sampling based on the last sampling result in the first sampling result set to obtain a second sampling result set.

[0218] The detection module 903 is used to obtain the detection result of the input image based on the difference between the first sampling result set and the second sampling result set.

[0219] Figure 10 This is a schematic diagram of the structure of an image detection apparatus 1000 according to another embodiment of the present disclosure. The apparatus 1000 includes a first sampling module 1001, a second sampling module 1002, and a detection module 1003. The functions of these modules are the same as those of the modules in the image detection apparatus 900 of the above embodiment. In one embodiment, the first sampling includes reverse sampling. The first sampling module 1001 includes:

[0220] The first input submodule 10011 is used to input the input image and condition label into the image forgery detection model to obtain the conditional flow field that matches the condition label;

[0221] The backsampling submodule 10012 is used to backsample the input image according to the back time step based on the conditional flow field that matches the conditional label, so as to obtain the backsampling result corresponding to each back time step.

[0222] In one implementation, such as Figure 10 As shown, the second sampling includes forward sampling. The second sampling module 1002 includes:

[0223] The second input submodule 10021 is used to input the last backsampling result in the backsampling result set and the condition label into the image forgery detection model to obtain the conditional flow field that matches the condition label.

[0224] The forward sampling submodule 10022 is used to perform forward sampling on the reverse sampling result according to the forward time step based on the conditional flow field that matches the condition label, so as to obtain the forward sampling result corresponding to each forward time step.

[0225] In one implementation, the forward sampling submodule 10022 is further configured to:

[0226] Based on the conditional flow field that matches the conditional label, conditional forward sampling is performed on the last backward sampling result according to the forward time step to obtain the conditional forward sampling result corresponding to each forward time step.

[0227] Based on the stochastic conditional flow field, the last backward sampling result is unconditionally forward sampled according to the forward time step to obtain the unconditional forward sampling result corresponding to each forward time step;

[0228] The regenerated image is obtained from the last sample in the conditional positive sampling result set and the unconditional positive sampling result set.

[0229] In one implementation, the forward sampling submodule is further configured to obtain a regenerated image based on the last sampling result in the conditional forward sampling result set and the unconditional forward sampling result set, including:

[0230] Input the condition label into the image forgery detection model to obtain the guiding strength coefficient that matches the condition label;

[0231] The last sampled result in the conditional positive sampling result set and the unconditional positive sampling result set are combined according to the guiding intensity coefficient to obtain the regenerated image.

[0232] In one implementation, such as Figure 10 As shown, the detection module 1003 is used to obtain the detection result of the input image based on the degree of matching between the back sampling result set and the forward sampling result set under the target condition label at the associated time step.

[0233] In one implementation, the input image is the image type corresponding to the target condition label if one or more of the following comparison results are true:

[0234] The degree of matching between the intermediate sampling results of the reverse sampling result set and the forward sampling result set at the same time step under the target condition label is greater than the first threshold.

[0235] The feature matching degree between the input image of the backsampled result set and the last sampled result of the forward sampled result set under the target condition label is greater than the second threshold.

[0236] In one implementation, the input image is an image type corresponding to the target condition label, including at least one of the following:

[0237] If the target condition label is a true label, then the input image is a true image;

[0238] If the target condition label is a target forgery technique label, the input image is an image generated using a target forgery technique.

[0239] Figure 11 This is a schematic diagram of the structure of a training apparatus 1100 for an image forgery detection model according to an embodiment of the present disclosure. In one embodiment, the apparatus includes:

[0240] The condition generation module 1101 is used to perform condition generation training on the image forgery detection model to be trained using training samples, and obtain the condition generation result.

[0241] The unconditional generation module 1102 is used to perform unconditional generation training on the image forgery detection model to be trained using the training sample, and obtain the unconditional generation result.

[0242] The training module 1103 is used to adjust the image forgery detection model to be trained based on the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result, so as to obtain the trained image forgery detection model.

[0243] Figure 12 This is a schematic diagram of a training apparatus 1200 for an image forgery detection model according to another embodiment of the present disclosure. The apparatus 1200 includes: a conditional generation module 1201, an unconditional generation module 1202, and a training module 1203. The functions of these modules are the same as those of the modules in the image forgery detection model training apparatus 1100 of the above embodiment. In one embodiment, the apparatus further includes:

[0244] Interpolation module 1204 is used to perform interpolation calculations between the conditional generation result and the unconditional generation result to obtain the guided generation image.

[0245] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0246] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0247] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0248] Figure 13A schematic block diagram of an example electronic device 1300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0249] like Figure 13 As shown, device 1300 includes a computing unit 1301, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1302 or a computer program loaded from storage unit 1308 into random access memory (RAM) 1303. The RAM 1303 may also store various programs and data required for the operation of device 1300. The computing unit 1301, ROM 1302, and RAM 1303 are interconnected via bus 1304. Input / output (I / O) interface 1305 is also connected to bus 1304.

[0250] Multiple components in device 1300 are connected to I / O interface 1305, including: input unit 1306, such as keyboard, mouse, etc.; output unit 1307, such as various types of monitors, speakers, etc.; storage unit 1308, such as disk, optical disk, etc.; and communication unit 1309, such as network card, modem, wireless transceiver, etc. Communication unit 1309 allows device 1300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0251] The computing unit 1301 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1301 performs the various methods and processes described above, such as image detection methods and / or image forgery detection model training methods. For example, in some embodiments, the image detection methods and / or image forgery detection model training methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1308. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1300 via ROM 1302 and / or communication unit 1309. When the computer program is loaded into RAM 1303 and executed by the computing unit 1301, one or more steps of the image detection methods and / or image forgery detection model training methods described above can be performed. Alternatively, in other embodiments, the computing unit 1301 may be configured by any other suitable means (e.g., by means of firmware) to perform an image detection method and / or a training method for an image forgery detection model.

[0252] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0253] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0254] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0255] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0256] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0257] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0258] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0259] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image detection method, comprising: The first sampling is performed on the input image to obtain the first sampling result set; A second sampling is performed based on the last sampling result in the first sampling result set to obtain a second sampling result set. Based on the difference between the first sampling result set and the second sampling result set, the detection result of the input image is obtained; The first sampling includes reverse sampling, which involves performing a first sampling based on the input image to obtain a first sampling result set, including: The input image and conditional labels are input into the image forgery detection model to obtain a conditional flow field that matches the conditional labels; wherein, the conditional labels include real image labels and forgery technique labels; Based on the conditional flow field matching the conditional label, the input image is backsampled according to the reverse time step to obtain the backsampling result corresponding to each reverse time step; wherein, the conditional flow field includes the sampling function corresponding to the time step; The second sampling includes forward sampling, which involves performing a second sampling based on the last sampling result in the first sampling result set to obtain a second sampling result set, including: The last backsampling result in the backsampling result set and the condition label are input into the image forgery detection model to obtain the conditional flow field that matches the condition label; Based on the conditional flow field matching the conditional label, the reverse sampling result is forward sampled according to the forward time step to obtain the forward sampling result corresponding to each forward time step; wherein, the forward time step and the reverse time step are opposite.

2. The method according to claim 1, wherein, Based on the conditional flow field matching the conditional label, the backsampling results are forward sampled according to the forward time step to obtain the forward sampling results corresponding to each forward time step, including: Based on the conditional flow field matching the conditional label, conditional forward sampling is performed on the last backward sampling result according to the forward time step to obtain the conditional forward sampling result corresponding to each forward time step. Based on the random conditional flow field, the last reverse sampling result is unconditionally forward sampled according to the forward time step to obtain the unconditional forward sampling result corresponding to each forward time step; The regenerated image is obtained based on the last sample result in the conditional positive sampling result set and the unconditional positive sampling result set.

3. The method according to claim 2, wherein, The regenerated image is obtained based on the last sample result in the conditional positive sampling result set and the unconditional positive sampling result set, including: The conditional label is input into the image forgery detection model to obtain the guiding strength coefficient that matches the conditional label; The last sampled result in the conditional positive sampling result set and the unconditional positive sampling result set are combined according to the guiding intensity coefficient to obtain the regenerated image.

4. The method according to claim 2 or 3, wherein, Based on the difference between the first sampling result set and the second sampling result set, the detection result of the input image is obtained, including: The detection result of the input image is obtained based on the degree of matching between the back sampling result set and the forward sampling result set under the target condition label at the associated time step.

5. The method according to claim 4, wherein, Based on the degree of matching between the backsampling result set and the forward sampling result set under the target condition label at the associated time step, the detection result of the input image is obtained, including: If one or more of the following comparison results are true, the input image is the image type corresponding to the target condition label: The degree of matching between the intermediate sampling results of the reverse sampling result set and the forward sampling result set under the target condition label at the same time step is greater than the first threshold. The matching degree between the features of the input image of the backsampling result set under the target condition label and the last sampling result of the forward sampling result set is greater than the second threshold.

6. The method according to claim 5, wherein, The input image is an image type corresponding to the target condition label, including at least one of the following: If the target condition label is a true label, then the input image is a true image; When the target condition label is a target forgery technique label, the input image is an image generated using the target forgery technique.

7. A training method for an image forgery detection model, applied to the image detection method according to any one of claims 1 to 6, comprising: The image forgery detection model to be trained is trained using training samples to perform conditional generation training, and the conditional generation results are obtained. The training samples are used to perform unconditional generation training on the image forgery detection model that needs to be trained, and the unconditional generation result is obtained. Based on the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result, the image forgery detection model to be trained is adjusted to obtain the trained image forgery detection model.

8. The method according to claim 7, wherein, The method further includes: Interpolation is performed between the conditional generation result and the unconditional generation result to obtain the guided generation image.

9. An image detection device, comprising: The first sampling module is used to perform a first sampling based on the input image to obtain a first sampling result set; The second sampling module is used to perform a second sampling based on the last sampling result in the first sampling result set to obtain a second sampling result set. The detection module is used to obtain the detection result of the input image based on the difference between the first sampling result set and the second sampling result set; The first sampling includes reverse sampling, and the first sampling module includes: The first input submodule is used to input the input image and conditional labels into the image forgery detection model to obtain a conditional flow field that matches the conditional labels; wherein, the conditional labels include image authenticity labels and forgery technology labels; The backsampling submodule is used to backsample the input image according to the back time step based on the conditional flow field that matches the conditional label, so as to obtain the backsampling result corresponding to each back time step; wherein, the conditional flow field includes the sampling function corresponding to the time step; The second sampling includes forward sampling, and the second sampling module includes: The second input submodule is used to input the last backsampling result in the backsampling result set and the condition label into the image forgery detection model to obtain a conditional flow field that matches the condition label; The forward sampling submodule is used to perform forward sampling on the reverse sampling result according to the forward time step based on the conditional flow field that matches the conditional label, so as to obtain the forward sampling result corresponding to each forward time step; wherein the forward time step and the reverse time step are opposite.

10. The apparatus according to claim 9, wherein, The forward sampling submodule is also used for: Based on the conditional flow field matching the conditional label, conditional forward sampling is performed on the last backward sampling result according to the forward time step to obtain the conditional forward sampling result corresponding to each forward time step. Based on the random conditional flow field, the last reverse sampling result is unconditionally forward sampled according to the forward time step to obtain the unconditional forward sampling result corresponding to each forward time step; The regenerated image is obtained based on the last sample result in the conditional positive sampling result set and the unconditional positive sampling result set.

11. The apparatus according to claim 10, wherein, The forward sampling submodule is further configured to obtain a regenerated image based on the sampling results in the conditional forward sampling result set and the unconditional forward sampling result set, including: The conditional label is input into the image forgery detection model to obtain the guiding strength coefficient that matches the conditional label; The last sampled result in the conditional positive sampling result set and the unconditional positive sampling result set are combined according to the guiding intensity coefficient to obtain the regenerated image.

12. The apparatus according to claim 10 or 11, wherein, The detection module is used to obtain the detection result of the input image based on the degree of matching between the back sampling result set and the forward sampling result set under the target condition label at the associated time step.

13. The apparatus according to claim 12, wherein, If one or more of the following comparison results are true, the input image is the image type corresponding to the target condition label: The degree of matching between the intermediate sampling results of the reverse sampling result set and the forward sampling result set under the target condition label at the same time step is greater than the first threshold. The matching degree between the features of the input image of the backsampling result set under the target condition label and the last sampling result of the forward sampling result set is greater than the second threshold.

14. The apparatus according to claim 13, wherein, The input image is an image type corresponding to the target condition label, including at least one of the following: If the target condition label is a true label, then the input image is a true image; When the target condition label is a target forgery technique label, the input image is an image generated using the target forgery technique.

15. A training apparatus for an image forgery detection model, applied to the image detection apparatus according to any one of claims 9 to 14, comprising: The conditional generation module is used to train the image forgery detection model to be trained using training samples, and to obtain the conditional generation results. An unconditional generation module is used to perform unconditional generation training on the image forgery detection model to be trained using the training samples, and obtain unconditional generation results. The training module is used to adjust the image forgery detection model to be trained based on the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result, so as to obtain the trained image forgery detection model.

16. The apparatus according to claim 15, wherein, The device further includes: An interpolation module is used to perform interpolation calculations between the conditional generation result and the unconditional generation result to obtain a guided generation image.

17. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-6 or 7-8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6 or 7-8.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6 or 7-8.

Images