Image detection method and device, equipment and storage medium

Abstract

The invention provides an image detection method and device, equipment and a storage medium, and relates to the technical field of computers, in particular to the technical fields of image recognition, forgery recognition, image forgery, artificial intelligence and the like. According to the specific implementation scheme, first sampling is performed based on an input image to obtain a first sampling result set; performing second sampling based on the last sampling result in the first sampling result set to obtain a second sampling result set; and obtaining a detection result of the input image based on the difference between the first sampling result set and the second sampling result set. According to the method and the device, a more accurate detection result of the input image can be obtained by comparing the differences of the sampling result sets in different sampling modes.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to technical fields such as image recognition, forgery recognition, image forgery, and artificial intelligence. Background Art

[0002] With the development of artificial intelligence technology, generative models such as deepfakes and generative adversarial networks (GANs) can generate and forge facial images in a realistic way. Although these technologies have brought huge innovation opportunities to the fields of media production, film and television industry, they may cause the spread of false information. Summary of the invention

[0003] The present disclosure provides an image detection method, apparatus, device and storage medium.

[0004] According to one aspect of the present disclosure, there is provided an image detection method, comprising:

[0005] Performing first sampling based on the input image to obtain a first sampling result set;

[0006] Perform a second sampling based on the last sampling result in the first sampling result set to obtain a second sampling result set;

[0007] Based on the difference between the first sampling result set and the second sampling result set, a detection result of the input image is obtained.

[0008] According to another aspect of the present disclosure, a method for training an image forgery detection model is provided, comprising:

[0009] Use the training samples to perform conditional generation training on the image forgery detection model to be trained to obtain the conditional generation result;

[0010] Using the training sample to perform unconditional generation training on the image forgery detection model to be trained, to obtain an unconditional generation result;

[0011] Based on the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result, the image forgery detection model to be trained is adjusted to obtain a trained image forgery detection model.

[0012] According to another aspect of the present disclosure, there is provided an image detection device, comprising:

[0013] A first sampling module, configured to perform first sampling based on an input image to obtain a first sampling result set;

[0014] A second sampling module, configured to perform a second sampling based on the last sampling result in the first sampling result set to obtain a second sampling result set;

[0015] The detection module is used to obtain a detection result of the input image based on the difference between the first sampling result set and the second sampling result set.

[0016] According to another aspect of the present disclosure, a training device for an image forgery detection model is provided, comprising:

[0017] A conditional generation module, used to perform conditional generation training on the image forgery detection model to be trained using the training samples, and obtain a conditional generation result;

[0018] An unconditional generation module is used to use the training sample to perform unconditional generation training on the image forgery detection model to be trained to obtain an unconditional generation result;

[0019] The training module is used to adjust the image forgery detection model to be trained based on the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result to obtain a trained image forgery detection model.

[0020] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0021] at least one processor; and

[0022] a memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0024] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0025] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0026] According to the present disclosure, by comparing the differences in sampling result sets under different sampling modes, a more accurate detection result of the input image can be obtained.

[0027] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0029] Figure 1 is a flowchart of an image detection method according to an embodiment of the present disclosure;

[0030] Figure 2 is a flowchart of an image detection method according to another embodiment of the present disclosure;

[0031] Figure 3 is a flowchart of a method for training an image forgery detection model according to an embodiment of the present disclosure;

[0032] Figure 4 is an application scenario diagram of a face forgery detection method according to an embodiment of the present disclosure;

[0033] Figure 5 is a flowchart of a method for detecting forged faces according to an embodiment of the present disclosure;

[0034] Figure 6 This is an application scenario diagram of reverse sampling according to an embodiment of the present disclosure;

[0035] Figure 7 This is an application scenario diagram of forward sampling according to an embodiment of the present disclosure;

[0036] Figures 8a to 8c is a schematic diagram of the comparison of difference indicators according to the present disclosure;

[0037] Fig. 9 is a structural schematic diagram of an image detection device according to an embodiment of the present disclosure;

[0038] Fig.10 is a structural schematic diagram of an image detection device according to another embodiment of the present disclosure;

[0039] Fig.11 is a structural schematic diagram of a training device for an image forgery detection model according to an embodiment of the present disclosure;

[0040] Fig.12 is a structural schematic diagram of a training device for an image forgery detection model according to another embodiment of the present disclosure;

[0041] Fig.13 is a block diagram of an electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0043] Generative models may cause the spread of false information in areas such as security, privacy, and public opinion guidance, and the spread of false information may have negative effects.

[0044] Figure 1 FIG. 1 is a flow chart of an image detection method 100 according to an embodiment of the present disclosure. In one implementation, the method includes:

[0045] S101, performing first sampling based on an input image to obtain a first sampling result set;

[0046] S102, performing a second sampling based on the last sampling result in the first sampling result set to obtain a second sampling result set;

[0047] S103: Obtain a detection result of the input image based on a difference between the first sampling result set and the second sampling result set.

[0048] In the disclosed embodiment, the input image may be real or forged. The input image may include input image features. Features may also be referred to as distributions, feature distributions, probability distributions, etc. Sampling may be performed in different directions based on the input image to obtain a variety of sampling results. By comparing the multiple sampling results, it is determined whether the input image is a forged image. The input image may include a face image, such as a face image uploaded by a user, a face image recorded by an image acquisition device, a face image frame in a video stream, etc.

[0049] In the embodiment of the present disclosure, a first sampling result set can be obtained by performing a first sampling in a first direction based on an input image. The first sampling result set may include features of the input image, intermediate sampling results, and a last sampling result. For example, the first sampling result set may include sampling results of the input image at each time step in the first sampling process. Among them, the sampling result of the last time step of the first sampling process is the last sampling result in the first sampling result set. A second sampling result set can be obtained by performing a second sampling in a second direction based on the last sampling result in the first sampling result set (which may be referred to as the second sampling input feature). The second sampling result set may include second sampling input features, intermediate sampling results, and the last sampling result. For example, the second sampling result set may include sampling results of the last sampling result in the first sampling result set at each time step in the second sampling process. Among them, the sampling result of the last time step of the second sampling process is the last sampling result in the second sampling result set, which can also be understood as the sampling result of the regenerated image.

[0050] In some examples, the first sampling and the second sampling may be two associated sampling modes, and the first direction and the second direction may be two different sampling directions. For example, the first direction is reverse, and the first sampling is reverse reverse sampling. Through reverse sampling, image noise (the last reverse sampling result) can be obtained based on the step-by-step degradation of the image. The first direction is forward, and the second sampling is forward sampling. Through forward sampling, a regenerated image can be obtained based on the step-by-step recovery of the image noise (for example, it can be converted based on the last forward sampling result).

[0051] In the disclosed embodiment, the peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) and perceptual similarity index (LPIPS) can be calculated based on the first sampling result set and the second sampling result set, and then the detection result of the input image can be obtained by comprehensive comparison and judgment based on the above indicators.

[0052] According to the present disclosure, by comparing the differences in the sampling result sets under different sampling methods, a more accurate detection result of the input image can be obtained. For example, the detection result of the input image can include the authenticity of the image. In the case where the input image is a forged image, the specific forgery technology of the input image can also be determined, such as deepfake, face swap, face to face, and face shifter.

[0053] Figure 2 2 is a flow chart of an image detection method 200 according to another embodiment of the present disclosure. The method 200 can be used to implement step S101 in the image detection method 100. In one embodiment, the first sampling includes reverse sampling, and the first sampling is performed based on the input image to obtain a first sampling result set. S101 further includes:

[0054] S201, inputting the input image and the conditional label into an image forgery detection model to obtain a conditional flow field matching the conditional label;

[0055] S202 . Based on the conditional flow field matching the conditional label, reverse sampling is performed on the input image according to the reverse time step to obtain a reverse sampling result corresponding to each reverse time step.

[0056] In the disclosed embodiment, the image forgery detection model may be a flow matching generative model (Flow Matching Generative Models). The flow matching generative model may be constructed based on a flow matching algorithm. The flow matching generative model is a type of generative model based on probability flow theory. The flow matching generative model may generate target distribution samples by learning a probability flow field from an initial distribution to a target distribution. The flow matching generative model may process input features based on conditional flow matching (CFM), by defining a conditional flow field in the data space so that the flow field can match the conditional probability path from the initial distribution to the target distribution.

[0057] In an embodiment of the present disclosure, the conditional label may include an image authenticity label and a counterfeit technology label, etc. For example, the counterfeit technology label may include a Deepfake label, a FaceSwap label, a Face2Face label, a FaceShifter label or other counterfeit technology labels, etc., which is not limited in the present disclosure. The conditional label may also be referred to as conditional information, which can generate a high-dimensional representation (such as a vector) through encoder processing, and the representation can be used as a signal to guide the generation during the generation process. Based on the input image and the conditional label, a conditional flow field matching the conditional label can be used to perform a first sampling in a first direction to obtain a first sampling result set. Based on the last sampling result in the first sampling result set and the conditional label, a second sampling is performed in a second direction to obtain a second sampling result set.

[0058] In the embodiment of the present disclosure, the conditional flow field may also be referred to as a conditional guidance vector field, a conditional probability flow field, etc. The conditional flow field may include a sampling function corresponding to a time step. If the first sampling is a reverse sampling based on a flow matching generation model, the conditional flow field may include v(zt ,t; c). And the reverse flow can be performed at each time step t by the following formula:

[0059]

[0060] Among them, z t It can represent the potential representation of the input image, or the distribution of the input image at each time step; t can represent the sampling time step. For example, the value of t can be from T to 0, T is greater than 0; c can represent the conditional label. v() can be implemented by a neural network. During the training process, the neural network can learn the target image distribution x from the initial noise z0. In the reverse sampling process, the time step can be called the reverse time step. As the reverse time step t gradually decreases from T to 0, the potential representation z of the input image t It can be gradually degraded into Gaussian noise. In this process, the conditional label c can guide the image to degenerate along the path corresponding to the conditional label, simulating the change of the image from the current state to the standard Gaussian noise.

[0061] In the disclosed embodiment, the reverse sampling results corresponding to each reverse time step output by the image forgery detection model may constitute a set of reverse sampling results as a whole. The reverse sampling result set may also be referred to as a reverse sampling path, etc. Reverse sampling may include a process of gradually degrading the image features in the input image until pure noise is obtained. For example, the reverse time step may be started from t=T and executed step by step to t=0. Time step T corresponds to the features of the input image (or referred to as the features of the input image, the initial distribution, etc.), time steps T-1 to 1 correspond to the intermediate results of reverse sampling, and time step 0 corresponds to the last sampling result of reverse sampling. In this example, the time step is decremented by 1, and may also be decremented by other values, such as 0.1, 0.2, etc. The last time step may also be t=1, for example, the last sampling result is z1. The sampling results output by the image forgery detection model at each reverse time step may be used as the input of the model at the next reverse time step. For example, the reverse time steps include 3, 2, 1, and 0. After the image forgery detection model outputs the distribution corresponding to time step 2, such as Gaussian noise, when reverse sampling is performed at time step 1, the distribution corresponding to time step 2 can be used as the input of the image forgery detection model to obtain the distribution corresponding to time step 1. When reverse sampling is performed at time step 0, the distribution corresponding to time step 1 can be used as the input of the image forgery detection model.

[0062] In the disclosed embodiment, the input image can be reversely sampled according to the conditional flow field that matches the conditional label, thereby achieving the opposite direction of the image generation path corresponding to the selected forgery technology, degrading the image features in the input image, and obtaining noise that can be processed by the forgery technology.

[0063] In one embodiment, the second sampling includes forward sampling, such as Figure 2 As shown, S102 performs a second sampling based on the sampling results in the first sampling result set to obtain a second sampling result set, including:

[0064] S203, inputting the last reverse sampling result in the reverse sampling result set and the conditional label into the image forgery detection model to obtain a conditional flow field matching the conditional label;

[0065] S204 . Based on the conditional flow field matching the conditional tag, forward sampling is performed on the reverse sampling result according to the forward time step to obtain a forward sampling result corresponding to each forward time step.

[0066] In the embodiment of the present disclosure, the image forgery detection model, conditional labels, and conditional flow fields can refer to the relevant descriptions in the above embodiments. If the second sampling is a forward sampling based on the flow matching generation model, the forward flow can be performed at each time step t by the following formula:

[0067]

[0068] Among them, z t , t and c can refer to the relevant explanation of the above formula 1. Among them, the sampling time step t can be from 0 to T. That is to say, the forward time step and the reverse time step can be opposite. For example, the reverse time step includes {3, 2, 1, 0}, and the forward time step includes {0, 1, 2, 3}. For another example, the reverse time step includes {1, 0.8, 0.6, 0.4, 0.2, 0}, and the forward time step includes {0, 0.2, 0.4, 0.6, 0.8, 1}.

[0069] In the disclosed embodiment, the image forgery detection model can be sampled step by step starting from the reverse sampling result according to the forward time step. The forward sampling result corresponding to each forward time step output by the image forgery detection model can constitute a whole forward sampling result set, which can also be called a forward sampling path, etc. Forward sampling can sample pure noise step by step, and the last sampling result of the forward sampling path can be converted to obtain a regenerated image. For example, the forward time step can be started from t=0 and step by step to t=T. The sampling result output by the image forgery detection model at each forward time step can be used as the input of the model at the next forward time step. For example, the forward time step includes 0, 1, 2, and 3. After the image forgery detection model outputs the distribution corresponding to time step 1, such as Gaussian noise, when forward sampling is performed at time step 2, the distribution corresponding to time step 1 can be used as the input of the image forgery detection model to obtain the distribution corresponding to time step 2. When forward sampling is performed at time step 3, the distribution corresponding to time step 2 can be used as the input of the image forgery detection model.

[0070] In the disclosed embodiments, the same conditional label can be used in the forward sampling process and the reverse sampling process, and thus the relevant conditional flow field can be matched. The conditional flow fields used for the forward sampling and the reverse sampling can be the same, but the flow modes can be opposite, see the above formulas 1 and 2. In some of the above examples, the forward time step of the forward sampling increases, and the forward time step of the reverse sampling increases, and the forward time step and the reverse time step have a corresponding relationship. For example, the forward time step and the reverse time step with the same value are corresponding. Based on the corresponding relationship between the forward time step and the reverse time step, it is convenient to compare the feature differences of the forward sampling process and the reverse sampling process at the same time step.

[0071] In the disclosed embodiment, the input image can be forward sampled according to the conditional flow field that matches the conditional label, so as to achieve the direction of the image generation path corresponding to the selected forgery technology, add image features to the noise, and generate an image obtained by the forgery technology based on the noise.

[0072] In one implementation, S204 performs forward sampling on the reverse sampling result according to the forward time step based on the conditional flow field matching the conditional tag to obtain the forward sampling result corresponding to each forward time step, including:

[0073] Based on the conditional flow field matching the conditional label, conditionally forward sampling is performed on the last reverse sampling result according to the forward time step to obtain the conditional forward sampling result corresponding to each forward time step;

[0074] Based on the random conditional flow field, unconditional forward sampling is performed on the last reverse sampling result according to the forward time step to obtain the unconditional forward sampling result corresponding to each forward time step;

[0075] A regenerated image is obtained according to the last sampling result in the conditional forward sampling result set and the unconditional forward sampling result set.

[0076] In the disclosed embodiment, during the forward sampling process, conditional forward sampling and unconditional forward sampling can be performed on the reverse sampling results. For example, the image forgery detection model performs conditional forward sampling on the reverse sampling results according to the conditional flow field that matches the conditional label, and obtains a conditional intermediate forward sampling result and a conditional last forward sampling result. For another example, the conditional label can be cleared or ignored, and the image forgery detection model uses a randomly acquired conditional flow field to perform forward sampling on the reverse sampling results to obtain an unconditional intermediate forward sampling result and an unconditional forward sampling result.

[0077] In the disclosed embodiment, the conditional forward sampling results and the unconditional forward sampling results can be further processed to obtain a comprehensive forward sampling result, which can strengthen or weaken the influence of the conditional label on the generation process, thereby obtaining a more accurate regenerated image.

[0078] In one embodiment, obtaining a regenerated image according to the conditional forward sampling result set and the last sampling result in the unconditional forward sampling result set includes:

[0079] The conditional label is input into the image forgery detection model to obtain a guidance strength coefficient matching the conditional label;

[0080] The conditional forward sampling result set and the last sampling result in the unconditional forward sampling result set are combined according to the guided strength coefficient to obtain a regenerated image.

[0081] In the disclosed embodiment, the image forgery detection model may be a guided flow matching generation model, which may also obtain a guided strength coefficient that matches the conditional label based on the conditional label. The guided strength coefficient is used to combine the conditional forward sampling result and the unconditional forward sampling result, and different conditional labels may correspond to different guided strength coefficients.

[0082] In the disclosed embodiment, the image forgery detection model can combine and calculate the conditional forward sampling results and the unconditional forward sampling results based on the guidance strength coefficient that matches the conditional label. An example of a calculation method is as follows:

[0083] x t =(1+ω)·x cond ―ω·x uncond Formula 3

[0084] Among them, x t can represent the regenerated image, ω can represent the guidance strength coefficient, x cond It can represent the image converted from the last conditional forward sampling result, x uncond It can represent the image obtained by unconditionally converting the last sampling result forward.

[0085] In one embodiment, Figure 2 As shown, S103 obtains the detection result of the input image based on the difference between the first sampling result set and the second sampling result set, including:

[0086] S205 . Obtain a detection result of the input image based on the matching degree of the sampling results of the reverse sampling result set and the forward sampling result set at the associated time step under the target condition label.

[0087] In the disclosed embodiment, the associated time step of the reverse sampling result set and the forward sampling result set may include the time step with the same value of the time step in the sampling process. The matching degree can be calculated based on the reverse sampling result and the forward sampling result at the same time step to obtain the detection result of the input image. For example, the matching degree can be calculated based on indicators such as PSNR, SSIM and LPIPS.

[0088] For example, the calculation formula of PSNR can be constructed based on mean square error (MSE). For example, MSE can represent the average of the sum of squares of the differences in pixel values ​​between two images. For example, an example formula of PSNR is as follows:

[0089]

[0090] Where MAX represents the maximum possible pixel value in the image. For example, for an 8-bit image, the pixel value range is 0 to 255, and MAX = 255. MSE is the mean square error.

[0091] Generally speaking, the larger the PSNR value, the smaller the difference between the images. For example, the sampling results corresponding to the reverse time step 0 and the forward time step 0 can be converted into corresponding images, and then the PSNR is calculated based on the converted images.

[0092] SSIM is an indicator to measure the structural similarity between two images. It focuses on the structural information of the image and the perception of the human visual system, and can better reflect the subjective perception of the human eye on the image quality. SSIM can compare the similarity of two images from several aspects such as brightness, contrast, and structure. For example, the reverse sampling results at multiple time steps 0, 1, 2, and 3 and the forward sampling results at time steps 0, 1, 2, and 3 are respectively compared and converted into corresponding images, and then the SSIM is calculated based on the brightness, contrast, structure, etc. of the converted images.

[0093] LPIPS can measure the similarity of two images based on the high-dimensional features extracted by deep neural networks. For example, the reverse sampling results and forward sampling results at multiple time steps 0, 1, 2, and 3 are converted into corresponding images, and then the high-dimensional features of the converted images are extracted based on the deep neural network to calculate LPIPS.

[0094] In the disclosed embodiment, by comparing the difference between the forward and reverse sampling result sets at the same time step, it is possible to determine whether an image is forged or to obtain a technique for image forgery. For an image, under the conditional flow field corresponding to the correct conditional label pair, the difference between the sampling results of the forward and reverse sampling at the same time step is generally small, and the degree of matching is high. Therefore, if the difference between the sampling results of the forward and reverse sampling at the same time step is large, or the degree of matching is not high, then the image is not an image under the label.

[0095] In one embodiment, based on the matching degree of the sampling results of the reverse sampling result set and the forward sampling result set at the associated time step under the target condition label, the detection result of the input image is obtained, including:

[0096] If one or more of the following comparison results are true, the input image is the image type corresponding to the target conditional label:

[0097] The matching degree of the intermediate sampling results of the reverse sampling result set and the forward sampling result set at the same time step under the target condition label is greater than a first threshold;

[0098] The matching degree between the feature of the input image of the reverse sampling result set under the target condition label and the last sampling result of the forward sampling result set is greater than a second threshold.

[0099] In the disclosed embodiment, by comparing the reverse sampling result and the forward sampling result of the same time step under the conditional flow field corresponding to the target condition label, the matching degree under the target condition label can be obtained. When the matching degree is higher than a certain threshold, the input image can be determined to be the image type corresponding to the target condition label. The starting time step of the forward sampling, for example, t=0, corresponds to the ending time step of the reverse sampling, and the forward sampling result corresponding to the time step t=0 is the reverse sampling result obtained at the ending time step of the reverse sampling. The starting time step of the reverse sampling, for example, t=T, corresponds to the ending time step of the forward sampling, and the reverse sampling result of the time step t=T is compared with the forward sampling result of the forward sampling t=T, that is, the feature of the input image is compared with the feature of the regenerated image. The first threshold and the second threshold can be the same or different. For example, the first threshold can be a larger value between 0-1, and the second threshold can be 1. The values ​​of the above thresholds are examples only and are not limited, and can also be other values. When the above respective comparison results meet the thresholds, it can be determined that the input image meets the target condition label, that is, the input image is the image type corresponding to the target condition label.

[0100] In the disclosed embodiment, by comparing the forward and reverse sampling results of the image under the conditional flow field corresponding to the same conditional label, it is possible to accurately determine whether the image is forged or the forgery technique used.

[0101] In one implementation, the input image is an image type corresponding to the target condition label, including at least one of the following:

[0102] When the target conditional label is a true label, the input image is a true image;

[0103] When the target condition label is a target forging technique label, the input image is an image generated using the target forging technique.

[0104] In the disclosed embodiment, the target condition label may include a real label, a forgery technology label, etc. The image forgery detection model can detect the input image based on the real label to obtain a forward sampling result set (or forward sampling path) and a reverse sampling result set (or reverse sampling path) under the real label. Then, the difference between the forward sampling result set and the reverse sampling result set under the real label is compared. If the difference between the two is small, for example, the degree of match is greater than a certain threshold, the input image can be determined to be a real image. Otherwise, it can be determined that the input image is not a real image, or that the input image is a forged image.

[0105] In the disclosed embodiment, the image forgery detection model can detect the input image based on one or more forgery technology labels, and obtain a forward sampling result set (or forward sampling path) and a reverse sampling result set (or reverse sampling path) under each forgery technology label. Then, the difference between the forward sampling result set and the reverse sampling result set under a certain target forgery technology label is compared. If the difference between the two is small, for example, the matching degree is greater than a certain threshold, it can be determined that the input image is an image forged using the target forgery technology corresponding to the target forgery technology label.

[0106] In the disclosed embodiment, if the comparison result meets the comparison condition under the conditional flow field matching a certain conditional label, that is, the feature matching degree of the forward and reverse sampling is high, then it can be determined that the current image belongs to the image type under the current conditional label.

[0107] Figure 3 FIG. 3 is a flow chart of a method 300 for training an image forgery detection model according to an embodiment of the present disclosure. In one implementation, the method 300 includes:

[0108] S301, using training samples to perform conditional generation training on an image forgery detection model to be trained, to obtain a conditional generation result;

[0109] S302, using the training sample to perform unconditional generation training on the image forgery detection model to be trained, to obtain an unconditional generation result;

[0110] S303: Based on the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result, the image forgery detection model to be trained is adjusted to obtain a trained image forgery detection model.

[0111] In the disclosed embodiment, the image forgery detection model can be a flow matching generation model. The flow matching generation model is a continuous normalizing flow (CNF) model framework that can learn continuous probability flows and transform simple prior distributions (such as standard Gaussian distributions) into complex data distributions.

[0112] In the embodiment of the present disclosure, the training sample of the image forgery detection model may include an image, a conditional label corresponding to the image, and noise corresponding to the image. An image may correspond to multiple conditional labels and multiple noises. For example, a training sample may include an image, a true label, and true image noise. For another example, a training sample may include an image, a target forgery technology label, and image noise forged using the target forgery technology.

[0113] In the disclosed embodiments, during the model training process, a combination of conditional generation and unconditional generation may be used to balance the guidance of conditional information such as conditional labels and the learning of basic data structures.

[0114] Examples of conditional generation include: the image forgery detection model receives specific conditional labels, for example, whether the image is a real face (real label) or a forged face generated by a certain forgery technique (forgery technology label). The image forgery detection model uses this label information to guide the generation process. The model accurately generates images that match the conditional labels by learning the data distribution under different conditions.

[0115] Examples of unconditional generation include: in some training steps, the model can ignore the conditional labels and generate only based on the image data itself. This strategy allows the model to better learn the underlying distribution characteristics of face images without relying on any specific conditions.

[0116] By switching between conditional and unconditional generation, the model is able to learn how to generate high-quality face images without a classifier or additional discriminant network, while ensuring that it can effectively distinguish between real and fake images.

[0117] In the disclosed embodiment, the noise and conditional labels in the training sample are input into the image forgery detection model to be trained for forward sampling, and the conditional generation result is output. According to the image in the training sample and the conditional generation result, the conditional generation loss is calculated. The noise in the training sample is input into the image forgery detection model to be trained for forward sampling, and the unconditional generation result is output. Alternatively, the noise and conditional labels in the training sample are input into the image forgery detection model to be trained, and the model removes the conditional labels and performs forward sampling, and outputs the unconditional generation result. According to the image in the training sample and the unconditional generation result, the unconditional generation loss is calculated.

[0118] In the flow matching generation framework, the training goal of the model can be to minimize the flow matching loss, thereby learning the flow field from the initial Gaussian noise distribution to the target face data distribution. The loss function of the embodiment of the present disclosure can simultaneously consider the contribution of conditional generation and unconditional generation.

[0119] For example, the loss function can be a weighted sum of the conditional generation loss and the unconditional generation loss, and the formula example is as follows:

[0120] L=λ cond ·L cond +λ uncond ·L uncond Formula 4

[0121] Among them, L cond represents the conditional generation loss, L uncond represents the unconditional generation loss, λ cond and λ uncond The weights for controlling the balance between conditional generation and unconditional generation. By adjusting these weights, the model can learn the characteristics of different forgery techniques while retaining its understanding of the overall structure of the face data.

[0122] In one embodiment, the method further includes: performing interpolation calculation between the conditional generation result and the unconditional generation result to obtain a guided generation image.

[0123] In the disclosed embodiments, the Classifier-Free Guidance technology can be used to improve the quality of generated samples, especially the discrimination of forgery techniques, by enhancing the strength of conditional generation. For example, interpolation is performed between conditional generation and unconditional generation to make the generated image more consistent with the preset conditional label.

[0124] For example, the bootstrap formula for generating samples can be:

[0125] x t =(1+ω)·x cond ―ω·x uncond

[0126] Where ω is the guiding strength coefficient, x cond is the result of conditional generation, x uncond is the unconditional generation result. By adjusting the size of ω, the consistency between the generated samples and the conditional information can be controlled. For example, selecting ω>1 can enhance the conditional guidance, allowing the model to more accurately generate images that match the forgery technique, thereby improving the accuracy of forgery detection.

[0127] In the disclosed embodiment, by performing interpolation calculation on the conditional generation result and the unconditional generation result, a guided generation image can be obtained when the calculated generated image meets the preset conditional label. The guided generation image can be processed using an image forgery detection model to obtain the corresponding conditional generation result and the unconditional generation result. Then, the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result are calculated respectively. For example, the conditional generation loss can be calculated based on the input noise of the conditional generation training and the image features of the model forward sampling output, and the unconditional generation loss can be calculated based on the input noise of the unconditional generation training and the image features of the model forward sampling output.

[0128] In the disclosed embodiment, the loss value of the loss function can be calculated based on the conditional generation loss and unconditional generation loss corresponding to the guided generated image with reference to Formula 4. When the loss value does not meet the convergence condition, the parameters of the model are adjusted, and after multiple iterations, a trained image forgery detection model can be obtained.

[0129] In the disclosed embodiment, an image forgery detection model can be obtained through training, which can combine conditional generation results and unconditional generation results according to the guidance strength coefficient corresponding to the conditional label to obtain a guided generated image that meets the conditional label characteristics. The image forgery detection model can match the corresponding guidance strength coefficient based on the conditional label, so that forward sampling can obtain a regenerated image that better meets the label characteristics.

[0130] For some problems of forged image detection, the embodiments of the present disclosure may provide corresponding solutions, as shown below:

[0131] 1. Insufficient accuracy in forged image detection: Most face forgery detection methods rely on fixed feature extraction or simple classifiers, which cannot cope with the rapidly developing image generation technology. Generative models, such as Deepfake or GAN, can generate highly realistic forged images, and the forgery techniques are constantly updated and iterated, which poses a huge challenge to traditional detection methods. The disclosed embodiments can use the flow matching generative model to model the potential distribution of face images, which can improve the accuracy of forgery detection when dealing with complex forgery techniques.

[0132] 2. Unable to identify specific counterfeiting techniques: Most current detection methods can only determine whether an image is forged, but cannot accurately identify the specific technical means used to forge the image. The distribution features of images generated by different forgery techniques are different. The disclosed embodiments can capture the feature distribution of different forgery techniques through the conditional guidance mechanism and flow matching generation model, thereby effectively identifying the specific forgery method.

[0133] 3. Lack of interpretability of detection results: Most forgery detection methods rely on black box models, lack the ability to interpret forged images, and cannot clearly point out the details of forgery. The disclosed embodiment can show the differences of images under different forgery conditions in detail by comparing the forward and reverse sampling paths, making the detection results highly interpretable and helping users understand how images are forged and how they are forged.

[0134] In response to the above problems, the image detection method of the embodiment of the present disclosure can detect the authenticity of images such as face images, and can also identify their forgery techniques. The image detection method proposed in the present disclosure may include a face forgery detection method based on a condition-guided stream matching generation model. The method can provide an efficient and interpretable detection mechanism for complex forgery techniques (such as Deepfake, GAN forgery, image editing, etc.). By comparing the feature differences of the image in the forward and reverse sampling paths, it is not only possible to distinguish between real faces and forged faces, but also to further identify specific forgery techniques, providing technical support for combating false information and improving image security.

[0135] Figure 4 This is an application scenario diagram of the face forgery detection method based on the condition-guided stream matching generation model. By modeling real face images and face images generated by various forgery techniques, the condition-guided stream matching generation model is trained to capture the potential distribution characteristics of various face images. This method can not only effectively distinguish real faces from forged faces, but also provide strong support for identifying specific forgery techniques. By comparing the differences between the images in the forward and reverse sampling paths, the authenticity is judged, and based on this difference, the specific type of forgery technology can be further identified.

[0136] Figure 5 FIG. 5 is a flow chart of a method 500 for detecting forged faces according to an embodiment of the present disclosure, which may include the following steps:

[0137] S501, guided flow matching generation model training: train the flow matching generation model for images of different label types (such as real face images and forged face images generated using different forging techniques). Each image type corresponds to a specific conditional label, and the model obtains its potential features by learning the distribution of face data under these conditional labels.

[0138] S502, condition-guided reverse sampling process: Given a face image, set different condition labels (including labels of real images and various forged images) to guide, and use the flow matching model to gradually degenerate the image into standard Gaussian noise matching the corresponding condition labels. This process can capture the feature differences of the image relative to each label type, regardless of whether the image comes from a real face or is generated by forgery.

[0139] S503, condition-guided forward sampling process: under the guidance of the same conditional label, the flow matching model is used to generate the corresponding type of image from the standard Gaussian noise. This process simulates the path of generating real or fake faces from Gaussian noise, reflecting the specific data distribution represented by each conditional label.

[0140] S504, difference comparison and discrimination: By comparing the feature differences of the input image in the reverse degradation and forward generation process under the guidance of various conditions, it is judged whether the image meets the face data distribution corresponding to the set condition label. If significant deviations are detected, it can be determined whether the image is forged based on these features, and the type of forgery technique can be further identified.

[0141] The disclosed embodiments can make full use of the reversibility and conditional guidance of the flow matching generation model, so that complex face image distributions can be accurately modeled. By comparing the forward and reverse sampling paths, this method can not only effectively detect the authenticity of face images, but also provide a clear explanatory basis for the identification of counterfeiting techniques, such as deepfakes, generative adversarial network (GAN) counterfeiting, image editing, etc.

[0142] An example of the main process of the embodiment of the present disclosure is as follows:

[0143] (I) Guided flow matching generation model training

[0144] 1. Stream Matching Generation Model

[0145] Suppose there are some data samples from a distribution of interest, whose density is unknown. If we want to use these samples to learn a probabilistic model to approximate the distribution. For example, we want to effectively generate new samples that are approximately from the distribution. This task is called generative modeling.

[0146] Flow Matching Generative Models is a new type of generative model based on probability flow theory. The model can generate target distribution samples by learning the probability flow field from the initial distribution to the target distribution.

[0147] The main principles of the flow matching generative model include conditional flow matching (CFM). By defining a conditional flow field in the data space, the flow field can match the conditional probability path from the initial distribution to the target distribution. The goal of flow matching is to optimize the flow field so that the generated samples evolve along this conditional path to reach the target distribution. Compared with other generative models, the flow matching model can directly use the analytical solution of the generation path without going through complex adversarial training or variational inference processes, so it has advantages in generation quality and stability. By introducing noise and back diffusion processes, high-quality image generation is successfully achieved, and advanced performance is achieved on multiple benchmark tasks.

[0148] 2. Guided Flow Matching Generation Model

[0149] Guided Flow Matching Generative Models is an extension of the flow matching generative model, which combines conditional generation and guidance techniques to more accurately control the generation process. Guided Flow Matching Generative Models can use external guidance information (Guidance) to guide the generative model to generate data according to specific conditions, thereby improving the quality of generated samples or meeting specific generation requirements.

[0150] In the guided flow matching generation model, the generation process is achieved by learning a conditional probability flow field. This flow field not only depends on the current state (data distribution), but is also affected by additional conditions (such as labels or specific task requirements). These conditions are embedded in the generation model to guide the generation path from the initial distribution to the target distribution. The conditional information can be any form of external signal, such as category labels, text descriptions, or image features.

[0151] In addition, the guided flow matching generation model can also be applied to tasks such as adversarial sample detection, data enhancement, and density estimation. By introducing external guidance signals, the model can model complex distributions more accurately and have stronger robustness and generalization capabilities.

[0152] 3. Model training

[0153] In the present disclosure, the condition-guided flow matching generation model gradually transforms the initial Gaussian noise distribution into the target face data distribution by learning a probability flow field, and introduces or removes conditional information in the generation process in combination with the classifier-free guidance mechanism, thereby capturing the characteristic distribution of different types of real faces and forged faces. The model is trained by weighted loss to ensure efficient distinction and detection of different forgery techniques. For example, the training process of the model generally includes the following steps:

[0154] 1). Conditional generation and unconditional generation

[0155] During the training process, the model can adopt a combination of conditional generation and unconditional generation to balance the guidance of conditional information and the learning of the basic data structure.

[0156] a. Conditional Generation: The model receives specific conditional labels (for example, whether the image is a real face or a fake face generated by a certain forgery technique) and uses this label information to guide the generation process. The model accurately generates images that match the conditional labels by learning the data distribution under different conditions, ensuring that the model can capture the characteristics of different forgery techniques.

[0157] b. Unconditional Generation: In some training steps, the model can deliberately ignore the conditional labels and generate only based on the image data itself. This strategy allows the model to better learn the underlying distribution characteristics of face images without relying on any specific conditions.

[0158] c. By switching between conditional and unconditional generation, the model is able to learn how to generate high-quality face images without a classifier or additional discriminant network, while ensuring that it can effectively distinguish between real and forged images.

[0159] 2) Training objectives

[0160] In the flow matching generation framework, the training goal of the model is to minimize the flow matching loss, thereby learning the flow field from the initial Gaussian noise distribution to the target face data distribution. To ensure effective detection of various forgery techniques, the loss function is designed to take into account the contributions of both conditional generation and unconditional generation.

[0161] For example, the loss function can be defined as the weighted sum of conditional generation and unconditional generation, as shown below:

[0162] L=λ cond ·L cond +λuncond ·L uncond Formula 4

[0163] Among them, L cond represents the conditional generation loss, L uncond represents the unconditional generation loss, λ cond and λ uncond The weights are used to control the balance between conditional generation and unconditional generation. By adjusting the weights, the model can learn the characteristics of different forgery techniques while retaining its understanding of the overall structure of the face data.

[0164] 3) Generation without classifier guidance

[0165] In the actual generation process, the Classifier-Free Guidance technology improves the quality of generated samples, especially the discrimination of forgery techniques, by enhancing the strength of conditional generation. The specific approach is to interpolate between conditional generation and unconditional generation to make the generated image more consistent with the preset conditional label.

[0166] For example, the bootstrap formula for generating samples can be:

[0167] x t =(1+ω)·x cond ―ω·x uncond

[0168] Where ω is the guiding strength coefficient, x cond is the result of conditional generation, x uncond is the unconditional generation result. By adjusting the size of the guidance strength coefficient ω, the consistency between the generated sample and the conditional information can be controlled. Generally, selecting a guidance strength coefficient greater than 1 can enhance the conditional guidance, allowing the model to more accurately generate images that match the forgery technique, thereby improving the accuracy of forgery detection.

[0169] (II) Condition-guided reverse and forward sampling process

[0170] 1. Reverse sampling

[0171] The goal of the reverse sampling process is to gradually degenerate the input face image into standard Gaussian noise through the flow matching model. By setting different conditional labels (such as real image labels and various forged image labels), the model can capture the potential feature differences between the image and each condition. By analyzing these differences, the model can determine the authenticity of the image and infer its possible forgery techniques. The specific steps are as follows:

[0172] 1). Input image and condition tag settings

[0173] The reverse sampling process starts with a given input face image x. At the same time, a conditional label corresponding to the image is set. The conditional label can be a "real image" or a specific forgery label of various forgery techniques (such as Deepfake, FaceSwap, etc.). The conditional label guides the model to sample along the path related to this label during the degradation process.

[0174] 2) Image projection into latent space

[0175] The input image x is mapped into the model’s latent space and converted into an initial latent representation z T , where T represents the starting time step of reverse sampling. This step represents the image as a latent variable inside the model, providing a basis for the subsequent flow matching operation.

[0176] 3) Gradual degradation of conditional guidance

[0177] The model performs a step-by-step degradation operation based on the flow matching equation, v(z t ,t; c), reverse flow is performed at each time step t:

[0178]

[0179] As time t decreases from T to 0, the potential representation z of the input image t Gradually degenerate into Gaussian noise. In this process, the conditional label c guides the image to degenerate along the path corresponding to the label, simulating the change of the image from the current state to standard Gaussian noise.

[0180] 4) Image features are gradually eliminated

[0181] The model is solved by an inverse ODE solver or discrete time steps. By continuously adjusting the flow field state, the input face image gradually degrades, and the high-level semantic features and detail information of the image are gradually eliminated, replaced by noise components. The conditional guidance in the reverse sampling process plays a key role. Different conditional labels c correspond to different types of image generation paths. Under different conditional labels, the performance of image degradation will be different. For example, the degradation of a real image under a real label should be smooth, while the degradation of a forged image under a real label may be abnormal. Under a forged label, the degradation path of the forged image may be more consistent with the potential distribution of the forgery technique.

[0182] 5). Degraded to standard Gaussian noise

[0183] When time t reaches 0, the image completely degenerates into standard Gaussian noise z0. This final state no longer contains specific image information, but reflects the potential distribution of the input image under the guidance of the conditional label c. At this point, by comparing the degradation paths under different conditions, the model can extract the matching degree information between the image and various labels. At this point, by comparing the degradation paths under different conditions, the model can extract the matching degree information between the image and various labels. Figure 6 This is an application scenario diagram of reverse sampling of an embodiment of the present disclosure. Through the reverse sampling process under different conditions of the same input image, the image can be degraded to noise.

[0184] 2. Forward sampling

[0185] The forward sampling process starts from the initial noise distribution and gradually evolves into a face image that meets specific conditions through the flow matching model. The specific steps are as follows:

[0186] 1). Noise initialization

[0187] Sampling starts with a random noise vector z0 from a standard Gaussian distribution. This noise vector represents the initial state in the latent space, and its dimension is consistent with the dimension of the target image. At this point, any specific features of the image have not yet been determined.

[0188] 2). Conditional information injection

[0189] At each time step t of the generation process, the model receives specific conditioning information, which includes one or more of the following:

[0190] Image authenticity label: Identifies whether the image is “real” or “fake”.

[0191] Forgery technology label: If the image is forged, the specific forgery technology category will be indicated, such as "Deepfake forgery", "FaceSwap synthesis", "Face2Face tampering", etc.

[0192] This conditional information is processed through an encoder to produce a high-dimensional representation, which will be used as a signal to guide the generation during the generation process.

[0193] 3). Generate path guidance

[0194] In the flow matching model, the generation process is guided by a trained probability flow field from the initial noise z0 to the target image distribution x. The flow field represents the evolution path of the image at t, and its changes are described by the following flow equation:

[0195]

[0196] At each time step, the model passes through the conditional flow field v(z t,t; c) Calculate how to update the noise vector z t , gradually approaching the final target face image distribution.

[0197] 4). Condition-guided adjustment

[0198] In order to ensure that the generated image is consistent with the given conditional information, the model adopts a classifier-free guidance mechanism to enhance the effect of conditional guidance by adjusting the results of conditional generation and unconditional generation. cond and unconditionally generate x uncond Combine between:

[0199] x t =(1+ω)·x cond ―ω·x uncond

[0200] Among them, ω is a coefficient that controls the guidance strength. By adjusting ω, the model can strengthen or weaken the influence of conditional information on the generation process, so that the output image is consistent with the given real / fake label and the category of forgery technology.

[0201] 5) Gradual evolution

[0202] By continuously adjusting the flow field state, the initial noise vector z t At each time step t, the model evolves towards the final target state. The model gradually transforms the noise into a potential representation of the face image through a forward ordinary differential equation (ODE) solver or discrete time step solution. In this process, the conditional information continuously affects the generation path to ensure that the authenticity or forgery characteristics of the output image are consistent with the input conditions.

[0203] 6). Generate image output

[0204] Noise vector z t Finally, it evolves into a representation z1 in the latent space. This representation is transformed into the final output image x through decoding. Due to the conditional guidance in the generation process, the output image is not only realistic, but also its authenticity or forgery characteristics are highly consistent with the given conditions. Figure 7 This is an application scenario diagram of forward sampling of an embodiment of the present disclosure. Through the Gaussian noise corresponding to different degradation conditions, the forward sampling process guided by the respective conditions can gradually remove noise and generate a face image.

[0205] (III) Difference comparison and discrimination

[0206] 1. Sampling path difference analysis

[0207] By comparing the feature differences of the input image in the reverse degradation and forward generation processes under various conditions, the degree of match between the image and the specific conditional label can be revealed. Then determine whether the image meets the face data distribution corresponding to the set conditional label. For example, the forward generation and reverse degradation paths of real images under the "real image" condition are highly consistent; while forged images may show significant path deviations, especially under real labels, the forward generation and reverse degradation path deviations are large. In the disclosed embodiment, commonly used indicators in image quality evaluation such as PSNR (peak signal-to-noise ratio), SSIM (structural similarity index) and LPIPS (perceptual similarity index) can be used as indicators for analyzing the differences in generated image samples in the sampling path.

[0208] 2. Image authenticity determination

[0209] Path matching degree: In the contrastive analysis, we first evaluate whether the sampling path of the input image matches the given conditional label. The paths of real images in forward and backward sampling should show consistency, while the paths of forged images may show anomalies under certain conditions.

[0210] Difference metric: quantify this deviation by calculating the distance or feature difference between the forward and reverse paths. If the path difference exceeds a certain threshold, it can be inferred that the image does not conform to the distribution under the corresponding conditional label and may be a forged image.

[0211] 3. Identification of counterfeiting techniques

[0212] Different forgery techniques are labeled, and the reverse and forward sampling paths are compared for each forgery technique. By comparing the path performance of the input image under different forgery labels, it is possible to further determine which forgery technique the image was generated by. For example, if the path performance of the input image under a specific forgery label is more consistent with the distribution of the label, the model can infer that the image was generated by the forgery technique.

[0213] 4. Comprehensive judgment mechanism

[0214] Finally, the authenticity and forgery techniques are comprehensively evaluated by comparing the difference indicators of the forward and reverse paths of the input image under various conditional labels. Figures 8a to 8c : It is a schematic diagram of the difference index comparison according to the present disclosure. The difference index comparison of the reverse degradation and forward generation process of the input image under different conditions. Each condition label can correspond to an index curve. From these curves, it can be seen that the PSNR, SSIM, and LPIPS values ​​of the input image under the real face label are better, and the input image can be judged as a real face.

[0215] Fig. 9FIG. 9 is a schematic diagram of the structure of an image detection device 900 according to an embodiment of the present disclosure. In one implementation, the device includes:

[0216] A first sampling module 901, configured to perform a first sampling based on an input image to obtain a first sampling result set;

[0217] A second sampling module 902, configured to perform a second sampling based on the last sampling result in the first sampling result set to obtain a second sampling result set;

[0218] The detection module 903 is used to obtain a detection result of the input image based on the difference between the first sampling result set and the second sampling result set.

[0219] Fig.10 1 is a schematic diagram of the structure of an image detection device 1000 according to another embodiment of the present disclosure. The device 1000 includes: a first sampling module 1001, a second sampling module 1002 and a detection module 1003. The functions of the above modules can refer to the functions of the modules of the image detection device 900 of the above embodiment. In one embodiment, the first sampling includes reverse sampling, and the first sampling module 1001 includes:

[0220] The first input submodule 10011 is used to input the input image and the conditional label into the image forgery detection model to obtain a conditional flow field matching the conditional label;

[0221] The reverse sampling submodule 10012 is used to reversely sample the input image according to the reverse time step based on the conditional flow field matching the conditional label, and obtain the reverse sampling result corresponding to each reverse time step.

[0222] In one embodiment, Fig.10 As shown, the second sampling includes forward sampling, and the second sampling module 1002 includes:

[0223] The second input submodule 10021 is used to input the last reverse sampling result in the reverse sampling result set and the conditional label into the image forgery detection model to obtain a conditional flow field matching the conditional label;

[0224] The forward sampling submodule 10022 is used to perform forward sampling on the reverse sampling result according to the forward time step based on the conditional flow field matching the conditional label, so as to obtain the forward sampling result corresponding to each forward time step.

[0225] In one implementation, the forward sampling submodule 10022 is further configured to:

[0226] Based on the conditional flow field matching the conditional label, conditionally forward sampling is performed on the last reverse sampling result according to the forward time step to obtain the conditional forward sampling result corresponding to each forward time step;

[0227] Based on the random conditional flow field, unconditional forward sampling is performed on the last reverse sampling result according to the forward time step to obtain the unconditional forward sampling result corresponding to each forward time step;

[0228] A regenerated image is obtained according to the last sampling result in the conditional forward sampling result set and the unconditional forward sampling result set.

[0229] In one embodiment, the forward sampling submodule is further used to obtain a regenerated image according to the conditional forward sampling result set and the last sampling result in the unconditional forward sampling result set, including:

[0230] The conditional label is input into the image forgery detection model to obtain a guidance strength coefficient matching the conditional label;

[0231] The conditional forward sampling result set and the last sampling result in the unconditional forward sampling result set are combined according to the guided strength coefficient to obtain a regenerated image.

[0232] In one embodiment, Fig.10 As shown, the detection module 1003 is used to obtain the detection result of the input image based on the matching degree of the sampling results of the reverse sampling result set and the forward sampling result set at the associated time step under the target condition label.

[0233] In one implementation, when one or more of the following comparison results are true, the input image is of the image type corresponding to the target condition label:

[0234] The matching degree of the intermediate sampling results of the reverse sampling result set and the forward sampling result set at the same time step under the target condition label is greater than a first threshold;

[0235] The matching degree between the feature of the input image of the reverse sampling result set under the target condition label and the last sampling result of the forward sampling result set is greater than a second threshold.

[0236] In one implementation, the input image is an image type corresponding to the target condition label, including at least one of the following:

[0237] When the target conditional label is a true label, the input image is a true image;

[0238] When the target condition label is a target forging technique label, the input image is an image generated using the target forging technique.

[0239] Fig.11 1 is a schematic diagram of a structure of a training device 1100 for an image forgery detection model according to an embodiment of the present disclosure. In one embodiment, the device includes:

[0240] The condition generation module 1101 is used to perform condition generation training on the image forgery detection model to be trained using the training samples to obtain a condition generation result;

[0241] An unconditional generation module 1102 is used to use the training sample to perform unconditional generation training on the image forgery detection model to be trained to obtain an unconditional generation result;

[0242] The training module 1103 is used to adjust the image forgery detection model to be trained based on the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result to obtain a trained image forgery detection model.

[0243] Fig.12 1 is a schematic diagram of the structure of a training device 1200 for an image forgery detection model according to another embodiment of the present disclosure. The device 1200 includes: a conditional generation module 1201, an unconditional generation module 1202 and a training module 1203. The functions of the above modules can refer to the functions of the modules of the training device 1100 for the image forgery detection model in the above embodiment. In one embodiment, the device also includes:

[0244] The interpolation module 1204 is used to perform interpolation calculation between the conditional generation result and the unconditional generation result to obtain a guided generation image.

[0245] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0246] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0247] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0248] Fig.13A schematic block diagram of an example electronic device 1300 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0249] like Fig.13 As shown, the device 1300 includes a computing unit 1301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1302 or a computer program loaded from a storage unit 1308 into a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the device 1300 can also be stored. The computing unit 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0250] A number of components in the device 1300 are connected to the I / O interface 1305, including: an input unit 1306, such as a keyboard, a mouse, etc.; an output unit 1307, such as various types of displays, speakers, etc.; a storage unit 1308, such as a disk, an optical disk, etc.; and a communication unit 1309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1309 allows the device 1300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0251] The computing unit 1301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1301 performs the various methods and processes described above, such as image detection methods and / or training methods for image forgery detection models. For example, in some embodiments, the image detection method and / or the training method for the image forgery detection model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1308. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1300 via the ROM 1302 and / or the communication unit 1309. When the computer program is loaded into the RAM 1303 and executed by the computing unit 1301, one or more steps of the image detection method and / or the training method for the image forgery detection model described above may be performed. Alternatively, in other embodiments, the computing unit 1301 may be configured to execute the image detection method and / or the image forgery detection model training method in any other appropriate manner (for example, by means of firmware).

[0252] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0253] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0254] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0255] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0256] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0257] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0258] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0259] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image detection method, comprising: Performing first sampling based on the input image to obtain a first sampling result set; Performing a second sampling based on the last sampling result in the first sampling result set to obtain a second sampling result set; Based on the difference between the first sampling result set and the second sampling result set, a detection result of the input image is obtained.

2. The method according to claim 1, wherein: The first sampling includes reverse sampling, and the first sampling is performed based on the input image to obtain a first sampling result set, including: Inputting the input image and the conditional label into an image forgery detection model to obtain a conditional flow field matching the conditional label; Based on the conditional flow field matching the conditional label, the input image is reversely sampled according to the reverse time step to obtain a reverse sampling result corresponding to each reverse time step.

3. The method according to claim 2, wherein: The second sampling includes forward sampling, and the second sampling is performed based on the sampling results in the first sampling result set to obtain a second sampling result set, including: Inputting the last reverse sampling result in the reverse sampling result set and the conditional label into the image forgery detection model to obtain a conditional flow field matching the conditional label; Based on the conditional flow field matching the conditional label, the reverse sampling result is forward sampled according to the forward time step to obtain the forward sampling result corresponding to each forward time step.

4. The method according to claim 3, wherein: Based on the conditional flow field matching the conditional label, forward sampling is performed on the reverse sampling result according to the forward time step to obtain the forward sampling result corresponding to each forward time step, including: Based on the conditional flow field matching the conditional label, conditionally forward sampling is performed on the last reverse sampling result according to the forward time step to obtain the conditional forward sampling result corresponding to each forward time step; Based on the random conditional flow field, unconditionally forward sampling is performed on the last reverse sampling result according to the forward time step to obtain an unconditional forward sampling result corresponding to each forward time step; A regenerated image is obtained according to the conditional forward sampling result set and the last sampling result in the unconditional forward sampling result set.

5. The method according to claim 4, wherein: Obtaining a regenerated image according to the conditional forward sampling result set and the last sampling result in the unconditional forward sampling result set, comprising: Inputting the conditional label into the image forgery detection model to obtain a guidance strength coefficient matching the conditional label; The conditional forward sampling result set and the last sampling result in the unconditional forward sampling result set are combined according to the guided strength coefficient to obtain a regenerated image.

6. The method according to any one of claims 3 to 5, wherein: Obtaining a detection result of the input image based on a difference between the first sampling result set and the second sampling result set, including: Based on the matching degree of the sampling results of the reverse sampling result set and the forward sampling result set at the associated time step under the target condition label, the detection result of the input image is obtained.

7. The method according to claim 6, wherein: Based on the matching degree of the sampling results of the reverse sampling result set and the forward sampling result set at the associated time step under the target condition label, the detection result of the input image is obtained, including: When one or more of the following comparison results are true, the input image is of the image type corresponding to the target condition label: The matching degree of the intermediate sampling results of the reverse sampling result set and the forward sampling result set at the same time step under the target condition label is greater than a first threshold; The matching degree between the feature of the input image of the reverse sampling result set under the target condition label and the last sampling result of the forward sampling result set is greater than a second threshold.

8. The method according to claim 7, wherein: The input image is an image type corresponding to the target condition label, including at least one of the following: When the target conditional label is a real label, the input image is a real image; In the case where the target condition label is a target forgery technique label, the input image is an image generated using a target forgery technique.

9. A method for training an image forgery detection model, comprising: Use the training samples to perform conditional generation training on the image forgery detection model to be trained to obtain the conditional generation result; Using the training samples to perform unconditional generation training on the image forgery detection model to be trained, to obtain an unconditional generation result; Based on the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result, the image forgery detection model to be trained is adjusted to obtain a trained image forgery detection model.

10. The method according to claim 9, wherein: The method further comprises: An interpolation calculation is performed between the conditional generation result and the unconditional generation result to obtain a guided generation image.

11. An image detection device, comprising: A first sampling module, configured to perform first sampling based on an input image to obtain a first sampling result set; A second sampling module, configured to perform a second sampling based on the last sampling result in the first sampling result set to obtain a second sampling result set; A detection module is used to obtain a detection result of the input image based on the difference between the first sampling result set and the second sampling result set.

12. The device according to claim 11, wherein: The first sampling includes reverse sampling, and the first sampling module includes: A first input submodule, used for inputting the input image and the conditional label into the image forgery detection model to obtain a conditional flow field matching the conditional label; The reverse sampling submodule is used to reversely sample the input image according to the reverse time step based on the conditional flow field matching the conditional label to obtain the reverse sampling result corresponding to each reverse time step.

13. The device according to claim 12, wherein: The second sampling includes forward sampling, and the second sampling module includes: A second input submodule, configured to input the last reverse sampling result in the reverse sampling result set and the conditional label into the image forgery detection model to obtain a conditional flow field matching the conditional label; The forward sampling submodule is used to perform forward sampling on the reverse sampling result according to the forward time step based on the conditional flow field matching the conditional label to obtain the forward sampling result corresponding to each forward time step.

14. The device according to claim 13, wherein: The forward sampling submodule is further used for: Based on the conditional flow field matching the conditional label, conditionally forward sampling is performed on the last reverse sampling result according to the forward time step to obtain the conditional forward sampling result corresponding to each forward time step; Based on the random conditional flow field, unconditionally forward sampling is performed on the last reverse sampling result according to the forward time step to obtain an unconditional forward sampling result corresponding to each forward time step; A regenerated image is obtained according to the conditional forward sampling result set and the last sampling result in the unconditional forward sampling result set.

15. The device according to claim 14, wherein: The forward sampling submodule is further used to obtain a regenerated image according to the sampling results in the conditional forward sampling result set and the unconditional forward sampling result set, including: Inputting the conditional label into the image forgery detection model to obtain a guidance strength coefficient matching the conditional label; The conditional forward sampling result set and the last sampling result in the unconditional forward sampling result set are combined according to the guided strength coefficient to obtain a regenerated image.

16. The device according to any one of claims 13 to 15, wherein: The detection module is used to obtain the detection result of the input image based on the matching degree of the sampling results of the reverse sampling result set and the forward sampling result set at the associated time step under the target condition label.

17. The device according to claim 16, wherein: When one or more of the following comparison results are true, the input image is of the image type corresponding to the target condition label: The matching degree of the intermediate sampling results of the reverse sampling result set and the forward sampling result set at the same time step under the target condition label is greater than a first threshold; The matching degree between the feature of the input image of the reverse sampling result set under the target condition label and the last sampling result of the forward sampling result set is greater than a second threshold.

18. The device according to claim 17, wherein: The input image is an image type corresponding to the target condition label, including at least one of the following: When the target conditional label is a real label, the input image is a real image; In the case where the target condition label is a target forgery technique label, the input image is an image generated using a target forgery technique.

19. A training device for an image forgery detection model, comprising: A conditional generation module, used to perform conditional generation training on the image forgery detection model to be trained using the training samples, and obtain a conditional generation result; An unconditional generation module, used to perform unconditional generation training on the image forgery detection model to be trained using the training samples to obtain an unconditional generation result; A training module is used to adjust the image forgery detection model to be trained based on the conditional generation loss corresponding to the conditional generation result and the unconditional generation loss corresponding to the unconditional generation result to obtain a trained image forgery detection model.

20. The device according to claim 19, wherein The device also includes: The interpolation module is used to perform interpolation calculation between the conditional generation result and the unconditional generation result to obtain a guided generation image.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8 or 9-10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8 or 9-10.

23. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1-8 or 9-10.

Images