Diffusion model generation image detection method based on convolutional neural network image reconstruction

Abstract

The invention relates to a diffusion model generated image detection method based on convolutional neural network image reconstruction, and the method comprises the steps: obtaining a target data set which comprises a real image, a first image and a second image; the first image is determined by diffusion of the real image; the second image is an enhanced image of the real image and the first image; reconstructing the target data set to obtain a training data set; based on the training data set, training a preset original image detection model to obtain a target image detection model; and inputting the to-be-detected image into the target image detection model for detection, and determining whether the to-be-detected image is a real image or not. Through the target image detection method and device, the problems of low robustness and low generalization in the prior art are solved, the number of effective training data sets can be further expanded, the robustness and generalization of the target image detection model are improved, and the detection effect is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a diffusion model generated image detection method based on convolutional neural network image reconstruction. BACKGROUND

[0002] The diffusion model generated image detection algorithm is a kind of discriminant technology based on the analysis of the inherent characteristics of the generated image. The core is to identify the specific statistical feature difference left by the generation process in the image. This method mainly analyzes the potential traces of the generated image in noise distribution, pixel correlation pattern and reconstruction error, and combines a deep learning model to build a binary classifier, so as to realize the differentiation of diffusion generated images and real shooting images.

[0003] The current mainstream implementation scheme is to use the noise pattern residual (NPR) method. The theoretical basis of this method is that the multi-layer convolutional neural network used by the diffusion model will form a specific spatial correlation characteristic in the image generation process, resulting in a systematic numerical correlation rule between adjacent pixels of the generated image. The technical implementation is to divide the input image into 2*2 small blocks. For each small block [w1, w2, w3, w4], it is changed to [0, w2-w1, w3-w1, w4-w1], and then all the small blocks are recombined into a new image in the original order. The new image is used as training data to train the image detection model. Since NPR focuses on the numerical correlation rule between adjacent pixels caused by convolutional neural network, the pre-processing such as compression and blur will disturb the image and destroy the numerical correlation rule between adjacent pixels, resulting in low robustness and low generalization of the scheme.

[0004] At present, there is no effective solution to the problem of low robustness and low generalization in the related art. SUMMARY

[0005] A diffusion model generated image detection method based on convolutional neural network image reconstruction is provided in the embodiment to solve the problem of low robustness and low generalization in the related art.

[0006] In a first aspect, a diffusion model generated image detection method based on convolutional neural network image reconstruction is provided in the embodiment, comprising:

[0007] Obtain a target data set, the target data set comprising a real image, a first image and a second image; the first image is determined by diffusion of the real image; the second image is an enhanced image of the real image and the first image;

[0008] Reconstruct the target data set to obtain a training data set;

[0009] training, a target image detection model is obtained;

[0010] The image to be detected is input into the target image detection model for detection to determine whether the image to be detected is a real image.

[0011] In some embodiments, inputting the image to be detected into the target image detection model for detection to determine whether the image to be detected is a real image includes:

[0012] Based on the pre-trained convolutional neural network, the image to be detected is reconstructed to obtain a reconstructed image;

[0013] determining the Euclidean distance between the features of the image to be detected and the reconstructed image;

[0014] If the Euclidean distance is greater than a preset threshold, the image to be detected is determined to be a real image;

[0015] If the Euclidean distance is less than or equal to the preset threshold, the image to be detected is determined to be a diffusion image.

[0016] In some embodiments, the threshold is positively correlated with a hyperparameter in a second loss function used to train the original image detection model.

[0017] In some embodiments, the original image detection model includes a deep learning sub-model and a fully connected layer.

[0018] training, a target image detection model is obtained, including:

[0019] According to the training data set and a preset total loss function, the deep learning sub-model and the fully connected layer in the original image detection model are iteratively trained to obtain a target image detection model; the total loss function includes a first loss function and a second loss function; the first loss function corresponds to the deep learning sub-model; and the second loss function corresponds to the fully connected layer.

[0020] In some embodiments, according to the training data set and a preset total loss function, the deep learning sub-model and the fully connected layer in the original image detection model are iteratively trained to obtain a target image detection model, including:

[0021] The fully connected layer is frozen, and the deep learning sub-model is trained according to the training data set and the first loss function;

[0022] freezing the deep learning sub-model, training the deep learning sub-model according to the training data set and the second loss function;

[0023] until a stop condition of iterative training is met, obtaining a target image detection model.

[0024] In some embodiments, the target data set is obtained, including:

[0025] a real image is obtained, a text description of the real image is input into an input diffusion model to generate the first image, and labels of the real image and the first image are added to obtain an original data set;

[0026] the real image and the first image in the original data set are enhanced to obtain a second image;

[0027] the second image is added to the original data set to obtain a target data set.

[0028] In some embodiments, the target data set is reconstructed to obtain a training data set, including:

[0029] the target data set is reconstructed based on a pre-trained convolutional neural network to obtain a training data set.

[0030] In a second aspect, a diffusion model generation image detection device based on convolutional neural network image reconstruction is provided in the embodiments, including an acquisition module, a reconstruction module, a training module and a detection module.

[0031] The acquisition module is configured to acquire a target data set, the target data set including a real image, a first image and a second image; the first image being determined by diffusion of the real image; and the second image being an enhanced image of the real image and the first image.

[0032] The reconstruction module is configured to reconstruct the target data set to obtain a training data set.

[0033] The training module is configured to train a preset original image detection model based on the training data set to obtain a target image detection model.

[0034] The detection module is configured to input a to-be-detected image into the target image detection model for detection to determine whether the to-be-detected image is a real image.

[0035] In a third aspect, a computer device is provided in the present embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the image detection method based on the diffusion model generated by the convolutional neural network image reconstruction according to the first aspect when executing the computer program.

[0036] In a fourth aspect, a storage medium is provided in the present embodiment, and the storage medium stores a computer program, and the computer program is executable on a processor to implement the image detection method based on the diffusion model generated by the convolutional neural network image reconstruction according to the first aspect.

[0037] Compared with the related art, the image detection method based on the diffusion model generated by the convolutional neural network image reconstruction provided in the present embodiment can obtain a target data set, the target data set comprising a real image, a first image, and a second image; the first image is determined by diffusion of the real image; the second image is an enhanced image of the real image and the first image, which can reduce the influence of preprocessing on model training; the target data set is reconstructed to obtain a training data set, which can further expand the number of effective training data sets to improve the training effect; a preset original image detection model is trained based on the training data set to obtain a target image detection model; and a to-be-detected image is input into the target image detection model for detection to determine whether the to-be-detected image is a real image, thereby solving the problems of low robustness and low generalization in the related art, using the second image to reduce the influence of preprocessing on model training, and further expanding the number of effective training data sets by combining the training data set obtained by reconstructing the target data set, thereby improving the robustness and generalization of the target image detection model to improve the detection effect.

[0038] The details of one or more embodiments of the present application are presented in the following drawings and description to make other features, objects, and advantages of the present application more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings illustrated herein are used to provide further understanding of the present application, and constitute a part of the present application. The illustrative embodiments of the present application and their description serve to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 is a hardware structure block diagram of a terminal device for the image detection method based on the diffusion model generated by the convolutional neural network image reconstruction provided in an embodiment of the present application;

[0041] Figure 2 is a flowchart of the image detection method based on the diffusion model generated by the convolutional neural network image reconstruction provided in an embodiment of the present application;

[0042] Figure 3 is a flowchart of step S210;

[0043] Figure 4 is a flowchart of step S240;

[0044] Figure 5 is a structural block diagram of a diffusion model generated image detection device based on convolutional neural network image reconstruction provided by an embodiment of the present application.

[0045] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 210, acquisition module; 220, reconstruction module; 230, training module; 240, detection module. DETAILED DESCRIPTION

[0046] In order to more clearly understand the purpose, technical scheme and advantages of the present application, the present application is described and explained below in combination with the drawings and embodiments.

[0047] Unless otherwise defined, technical terms or scientific terms related to the present application shall have the general meaning understood by a person with ordinary skill in the art to which the present application belongs. In the present application, "one", "a", "an", "the", "these" and similar words do not represent a quantitative limitation, and they can be singular or plural. In the present application, the terms "include", "contain", "have" and any variants thereof have the purpose of covering non-exclusive inclusion; for example, a process, method and system, product or device containing a series of steps or modules (units) are not limited to the listed steps or modules (units), but can include steps or modules (units) not listed, or can include other steps or modules (units) inherent to the process, method, product or device. In the present application, the terms "connected", "connected", "coupled" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. In the present application, "multiple" means two or more. The association between the associated objects is described by the term "and / or", which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. In general, the character " / " represents an "or" relationship between the objects before and after. In the present application, the terms "first", "second", "third" and the like are only used to distinguish similar objects, and do not represent a specific order of the objects.

[0048] The method embodiments provided in the present embodiment can be executed in a terminal, a computer or a similar computing device. For example, the method embodiments are executed on a terminal, Figure 1 is a hardware structural block diagram of a terminal of the diffusion model generated image detection method based on convolutional neural network image reconstruction of the present embodiment. As shown inFigure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0049] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the diffusion model-based image detection method based on convolutional neural network image reconstruction in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0050] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0051] This embodiment provides an image detection method based on a diffusion model generated by convolutional neural network image reconstruction. Figure 2 This is a flowchart of the diffusion model-based image detection method for image reconstruction using convolutional neural networks, as described in this embodiment. Figure 2 As shown, the process includes the following steps:

[0052] Step S210, obtaining a target data set, the target data set including a real image, a first image and a second image; the first image being determined by diffusion of the real image; the second image being an enhanced image of the real image and the first image;

[0053] Step S220, reconstructing the target data set to obtain a training data set;

[0054] Step S230, training a preset original image detection model based on the training data set to obtain a target image detection model;

[0055] Step S240, inputting a to-be-detected image into the target image detection model for detection to determine whether the to-be-detected image is a real image.

[0056] In actual application, the manner of obtaining the target data set in the embodiments of the present application includes but is not limited to being obtained from pre-stored in a database, obtaining the target data set meeting the above requirements; can also be downloading the target data set meeting the requirements from a network platform; can also be generating a corresponding target data set according to requirements, and the like. The manner of obtaining the target medical image data is not limited in the embodiments of the present application.

[0057] The target data set includes a real image, a first image and a second image; the real image is usually an image of a real scene taken by a camera or other image acquisition device as a real image. The first image is determined by diffusion of the real image, which can be obtained by diffusing the real image or a textual description of the real image through a diffusion model, an image processing algorithm or the like. The second image is an enhanced image of the real image and the first image. Specifically, all or a part of the real image and the first image can be selected, and a plurality of data enhancement manners are used for data enhancement to obtain the second image; the data enhancement manners include but are not limited to compression, blur processing, random erasing, hybrid enhancement and the like, and are not limited thereto.

[0058] Reconstructing the target data set refers to reconstructing each type of image in the target data set, thereby reconstructing a high-quality training data set on the basis of limited data volume, to improve training efficiency while ensuring training effect. The reconstruction method can use an interpolation-based image reconstruction algorithm, a deep learning neural network, etc., and is not limited in this regard. The original image detection model is an untrained model based on a neural network architecture, and the training method corresponding to the relevant neural network model can be selected. Then, after obtaining the target training set, each image in the target training set is used as the basis to train the original image detection model using the corresponding training method, thereby obtaining a target image detection model with high accuracy, robustness, and robustness. The accuracy refers to the correctness of the target image detection model in determining whether the input image is an image generated by the diffusion model. The robustness refers to the target image detection model being trained only on images generated by one diffusion model, but being able to detect images generated by multiple diffusion models to address the problem of new diffusion models emerging endlessly. The robustness refers to the input image still having high accuracy after being preprocessed by common image compression, blurring, etc. in real-world scenarios.

[0059] Since the target image detection model has high accuracy, robustness, and robustness, inputting each type of image to be detected into the target image detection model for detection can output accurate results to determine whether the image to be detected is a real image.

[0060] In the related art, the current mainstream implementation scheme is to use a noise pattern residual (NPR) method. The theoretical basis of this method is that the multi-layer convolutional neural network used by the diffusion model forms a specific spatial correlation characteristic in the image generation process, resulting in a systematic numerical correlation rule between adjacent pixels of the generated image. The technical implementation is to divide the input image into 2*2 small blocks. For each small block [w1, w2, w3, w4], it is changed to [0, w2-w1, w3-w1, w4-w1], and then all the small blocks are recombined into a new image in the original order. The new image is used as training data to train the image detection model. Since NPR focuses on the numerical correlation rule between adjacent pixels caused by the convolutional neural network, the compression, blurring and other preprocessing will disturb the image and destroy the numerical correlation rule between adjacent pixels, resulting in low robustness and low generalization of the scheme. In the present application, the target data set is obtained, the target data set includes a real image, a first image and a second image; the first image is determined by the diffusion of the real image; the second image is an enhanced image of the real image and the first image, which can reduce the influence of preprocessing on model training; the target data set is reconstructed to obtain a training data set, which can further expand the number of effective training data sets to improve the training effect; the preset original image detection model is trained based on the training data set to obtain a target image detection model; the image to be detected is input into the target image detection model for detection to determine whether the image to be detected is a real image, solving the problems of low robustness and low generalization in the related art. The second image is used to reduce the influence of preprocessing on model training, and the training data set obtained by reconstructing the target data set can further expand the number of effective training data sets, thereby improving the robustness and generalization of the target image detection model to improve the detection effect.

[0061] The above steps are described in detail as follows:

[0062] In some embodiments, as shown in Figure 3 the target data set in step S210 includes the following steps:

[0063] In step S211, a real image is obtained, the textual description of the real image is input into an input diffusion model to generate a first image; and labels of the real image and the first image are added to obtain an original data set;

[0064] In step S212, the real image and the first image in the original data set are enhanced to obtain a second image;

[0065] In step S213, the second image is added to the original data set to obtain the target data set.

[0066] In this embodiment, the real image can be an image of a real scene taken by an image acquisition device; the first image can be a diffusion image generated by inputting a textual description of the real image into Stablediffusion 1.4 as a unique diffusion model. In this embodiment, it is necessary to ensure that the real image and the generated first image have no obvious deviation in content as much as possible, and the content of the real image and the generated image should not be almost disjoint, which will cause the target image detection model to learn the content difference between the real image and the generated image in the data set during subsequent training. For example, if all the real images are bedroom images, and the content of the generated images is extensive, the target image detection model may judge all the bedroom images as real images, regardless of whether it is generated by the diffusion model.

[0067] After obtaining the real image and the first image, a label is added to distinguish the real image and the first image; for example: for the original data set X, and the image x in the original data set X i (including the real image and the first image), then x i ∈X, if x i is a real image, then its label y i = 0; if x i is a first image, then its label y i = 1.

[0068] Among them, the real image and the first image are enhanced by data enhancement to obtain the second image. Preferably, in order to avoid introducing data set bias, the real image and the first image use the same data enhancement method. For example: the real image and the first image are enhanced by using the data enhancement method of Gaussian blur and JPEG compression; and if the real image uses Gaussian blur but not the first image, the model trained subsequently will learn that "with blur is a real image" incorrectly. Of course, the same image can also be enhanced by using multiple data enhancement methods. For example, a Gaussian blur of an image is not used as a data enhancement result, but a Gaussian blur, a compression with a quality of 95, and a compression with a quality of 70 of the same image can be used as three second images to join the original data set to obtain the second image.

[0069] Through this embodiment, the target data set can be optimized, thereby providing a high-quality target data set, and further improving the efficiency and accuracy of model training.

[0070] Preferably, the selection of the data enhancement method can be determined by the actual scene; for example: in the scene of the Internet, compression and blur are very common image processing methods, so the data enhancement method adopts the data enhancement method of compression and blur, thereby avoiding the interference of the model by these common processing methods, and thereby improving the accuracy.

[0071] In some embodiments, reconstructing the target dataset in step S220 to obtain the training dataset comprises the following steps:

[0072] In step S221, reconstructing the target dataset based on the pre-trained convolutional neural network to obtain the training dataset.

[0073] Specifically, in order to reduce the occupation of computing resources, a pre-trained convolutional neural network can be used. For the pre-trained convolutional neural network, a preset number of downsampling is performed first, and then a preset number of upsampling is performed; convolution operation is required after each downsampling and upsampling. That is, given any image, the output image can be very close to the input image through the pre-trained parameters in the convolutional neural network. At the same time, the output image also retains the features of the image processed by the convolutional neural network, so as to further optimize the images in the target dataset.

[0074] The process of reconstructing the target dataset is as follows: for each image x in the target dataset i , first, a vector z with the same dimension as the image is randomly generated, and the parameters θ of the convolutional neural network R θ are initialized. For example: for an image with a resolution of HxW, each pixel contains C channels (common RGB image is 3 channels), which can be represented by a matrix with a dimension of HxWxC. The vector z is also a matrix with a dimension of HxWxC, and each element in the matrix is independently sampled from a uniform distribution. For example, for an RGB image, the value of each channel is an integer between 0 and 255, so each element of the vector z is randomly generated with equal probability from all integers between 0 and 255.

[0075] Then, the parameters of the convolutional neural network are trained using the Adam optimizer to minimize the mean square error between R θ (z) and the image x i , until the stopping condition is met. The stopping condition can be that the number of training steps reaches a threshold value, or the mean square error between R θ (z) and x i is less than a certain threshold value, and the specific threshold value is not limited.

[0076] Then, for the image x' in the target dataset i , the reconstruction result can be represented as the image x' ; i , and the image x' is the R i that minimizes the mean square error between the image x i during the training process. θ(z) In some embodiments, the training data set can be expressed in the form of a triple to facilitate management and use; for example, a record in the training data set can be expressed in the form of a triple (x i ,x′ i ,y i ).

[0077] In some embodiments, the training of the preset original image detection model based on the training data set in step S230 to obtain the target image detection model includes the following steps:

[0078] According to the training data set and the preset total loss function, the deep learning sub-model and the full connection layer in the original image detection model are iteratively trained to obtain the target image detection model; the total loss function includes a first loss function and a second loss function.

[0079] Specifically, the original image detection model includes a deep learning sub-model and a full connection layer; the deep learning sub-model can be a vision transformer architecture model; the full connection layer is connected to the deep learning sub-model, and is used to map the output dimension of the vision transformer to a target dimension; the specific result of the full connection layer is not limited,

[0080] In this embodiment, the total loss function includes a first loss function and a second loss function; the first loss function is the loss function of the deep learning sub-model; the second loss function is the loss function of the full connection layer. The weighted average of the first loss function and the second loss function is the total loss function.

[0081] Based on this, the training data set is used as training data, the weighted average of the first loss function and the second loss function is used as the total loss function, and the deep learning sub-model and the full connection layer are iteratively trained using the Adam optimizer to obtain the target image detection model. The iterative training method includes but is not limited to using the total loss function as a loss function and training the deep learning sub-model and the full connection layer as a whole to obtain the target image detection model, thereby improving the training efficiency; or, under the first loss function, the deep learning sub-model is trained first; after the deep learning sub-model is trained, the full connection layer is trained again under the second loss function until the full connection layer is trained to obtain the target image detection model; this method can shorten the training process of the deep learning sub-model; or, the process of training the deep learning sub-model under the first loss function and training the full connection layer under the second loss function is alternately performed to obtain the target image detection model; this method can reduce the occupation of training resources and reduce the demand for video memory.

[0082] Through this embodiment, the training method can be flexibly selected to match the training scene to enhance adaptability.

[0083] In some embodiments, the deep learning sub-model and the fully connected layer in the original image detection model are iteratively trained according to a training data set and a preset total loss function to obtain a target image detection model, including the following steps:

[0084] Step S231, freeze the fully connected layer, and train the deep learning sub-model according to the training data set and the first loss function;

[0085] Step S232, freeze the deep learning sub-model, and train the deep learning sub-model according to the training data set and the second loss function;

[0086] Step S233, until the iteration training meets the stop condition, the target image detection model is obtained.

[0087] In the present embodiment, the above process can be considered as a process of alternating training using Adam optimizer and contrastive learning strategy. Wherein the deep learning sub-model is a vision transformer; then the training process is specifically:

[0088] Freeze the fully connected layer, and train the vision transformer using Adam optimizer. The training target is that if the image x i in the training data set is a real image, then the Euclidean distance between f θ (x i ) and f θ (x i ′ ) is as large as possible, and vice versa; then the first loss function can be expressed as:

[0089]

[0090] Wherein, D represents the Euclidean distance; m is a hyperparameter; f θ represents the vision transformer.

[0091] Freeze the vision transformer, and train the fully connected layer using Adam optimizer. The training target is that for two images x i and x j in the training data set, if their labels are the same, then the Euclidean distance between g φ (f θ (x i )) and g φ (f θ (x i ′ )) is as small as possible; and vice versa. φ (f θ (xj )) and g φ (f θ (x j ′ the gap between the two Euclidean distances; otherwise, increase the gap between the two Euclidean distances.

[0092] For example, there are N pairs of binary tuples in the training data set Let Y k represent whether is the same, 0 for the same and 1 for different. If Y k = 1, let be a real image, then the second loss function can be represented as:

[0093]

[0094] where g φ represents a fully connected layer; the hyperparameters in the first loss function and the second loss function can be the same or different, and this is not limited.

[0095] The training process of the vision transformer and the fully connected layer is alternately performed until the stop condition of the training is met, which can be that the number of iterations reaches an iteration threshold, or the training target is completed.

[0096] Through the above embodiment, by using the iterative training method, combining the dependence of the vision transformer output on the fully connected layer (determined by two loss functions), the demand for video memory is reduced while the training effect is guaranteed.

[0097] In some embodiments, as shown in Figure 4 the step S240 of inputting the to-be-detected image into the target image detection model for detection to determine whether the to-be-detected image is a real image, comprising:

[0098] Step S241, reconstructing the to-be-detected image based on the pre-trained convolutional neural network to obtain a reconstructed image;

[0099] Step S242, determining the Euclidean distance between the features of the to-be-detected image and the reconstructed image;

[0100] Step S243, if the Euclidean distance is greater than a preset threshold, determining that the to-be-detected image is a real image;

[0101] Step S244, if the Euclidean distance is less than or equal to the preset threshold, determining that the to-be-detected image is a diffusion image.

[0102] Specifically, when the target image detection model predicts whether the to-be-detected image is a diffusion model generated image (a real image or a diffusion image), first, the to-be-detected image is reconstructed based on the pre-trained convolutional neural network to obtain a reconstructed image. Specifically, the implementation process can adopt the implementation of step S220, which will not be repeated here. Then, the Euclidean distance between the feature g φ (f θ (x)) of the to-be-detected image and the feature g φ (f θ (x ′ )) of the reconstructed image is calculated. If the Euclidean distance is greater than a preset threshold, the target image detection model predicts that the to-be-detected image is a real image; if the Euclidean distance is less than or equal to the preset threshold, the target image detection model predicts that the to-be-detected image is a diffusion image.

[0103] The threshold value is positively correlated with the hyperparameter in the second loss function for training the original image detection model. For example, the hyperparameter is 4, and the threshold value is 1.06. The larger the hyperparameter, the larger the threshold value.

[0104] As shown in Table 1, the accuracy of each type of diffusion image detected by DRCT (diffusion reconstruction contrast training-based general diffusion generated image detection method) and the embodiment of the method is compared.

[0105]

[0106]

[0107] LDM in Table 1 refers to the earliest model using the latent space diffusion model architecture, not the latent space diffusion model architecture.

[0108] As shown in Table 2, the accuracy of each type of real image detected by NPR, DRCT and the embodiment of the method is compared.

[0109] MSCOCO dataset LSUN dataset NPR 60 100 DRCT 92.7 46 The present method 100 76

[0110] Since NPR has a high probability of directly predicting the compressed image as a real image, the accuracy of NPR on real images is very high, but this data has no reference significance.

[0111] Through this embodiment, the target image detection model with high accuracy, high generalization and high robustness is used to predict the to-be-detected image, thereby improving the accuracy of prediction.

[0112] It is noted that the steps shown in the above flow or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from here.

[0113] In the embodiment, an image detection device based on a diffusion model generated by a convolutional neural network image reconstruction is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. The terms "module", "unit", "sub-unit" and the like used below can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware is also possible and is contemplated.

[0114] Figure 5 is a structural block diagram of the image detection device based on a diffusion model generated by a convolutional neural network image reconstruction of the embodiment, as Figure 5 shown, the device includes an acquisition module 210, a reconstruction module 220, a training module 230, and a detection module 240.

[0115] The acquisition module 210 is configured to acquire a target data set, the target data set including a real image, a first image, and a second image; the first image is determined by diffusion of the real image; and the second image is an enhanced image of the real image and the first image.

[0116] The reconstruction module 220 is configured to reconstruct the target data set to obtain a training data set.

[0117] The training module 230 is configured to train a preset original image detection model based on the training data set to obtain a target image detection model.

[0118] The detection module 240 is configured to input a to-be-detected image into the target image detection model for detection to determine whether the to-be-detected image is a real image.

[0119] Through the above device, the problems of low robustness and low generalization in the related art are solved, the second image is used to reduce the influence of preprocessing on model training, and the training data set obtained by reconstructing the target data set can further expand the number of effective training data sets, thereby improving the robustness and generalization of the target image detection model, to improve the detection effect.

[0120] In some embodiments, the detection module 240 is further configured to reconstruct the to-be-detected image based on a pre-trained convolutional neural network to obtain a reconstructed image.

[0121] determining a Euclidean distance between the to-be-detected image and the reconstructed image;

[0122] if the Euclidean distance is greater than a preset threshold, determining that the to-be-detected image is a real image;

[0123] if the Euclidean distance is less than or equal to the preset threshold, determining that the to-be-detected image is a diffusion image.

[0124] In some embodiments, the threshold is positively correlated with a hyperparameter in the second loss function for training the original image detection model.

[0125] In some embodiments, the original image detection model includes a deep learning sub-model and a fully connected layer.

[0126] The training module 230 is further configured to train the deep learning sub-model and the fully connected layer in the original image detection model according to the training data set and a preset total loss function to obtain a target image detection model; the total loss function includes a first loss function and a second loss function; the first loss function corresponds to the deep learning sub-model; and the second loss function corresponds to the fully connected layer.

[0127] In some embodiments, the training module 230 is further configured to freeze the fully connected layer, and train the deep learning sub-model according to the training data set and the first loss function.

[0128] freeze the deep learning sub-model, and train the deep learning sub-model according to the training data set and the second loss function.

[0129] obtain the target image detection model until a stop condition of the iterative training is met.

[0130] In some embodiments, the obtaining module 210 is further configured to obtain a real image, input a textual description of the real image into an input diffusion model to generate a first image, and add labels of the real image and the first image to obtain an original data set.

[0131] augment the real image and the first image in the original data set to obtain a second image.

[0132] add the second image to the original data set to obtain a target data set.

[0133] In some embodiments, the reconstruction module 220 is further configured to reconstruct the target data set based on a pre-trained convolutional neural network to obtain the training data set.

[0134] It should be noted that the above various modules can be functional modules or program modules, which can be implemented by software or hardware. For the modules implemented by hardware, the above various modules can be located in the same processor; or the above various modules can also be located in different processors in any combination.

[0135] In the embodiment, a computer device is also provided, including a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to perform the steps in any of the above method embodiments.

[0136] Optionally, the computer device can further include a transmission device and an input and output device, wherein the transmission device is connected with the processor, and the input and output device is connected with the processor.

[0137] Optionally, in the embodiment, the processor can be configured to execute the following steps through the computer program:

[0138] S1, obtaining a target data set, the target data set including a real image, a first image and a second image; the first image being determined by diffusion of the real image; the second image being an enhanced image of the real image and the first image;

[0139] S2, reconstructing the target data set to obtain a training data set;

[0140] S3, training a preset original image detection model based on the training data set to obtain a target image detection model;

[0141] S4, inputting a to-be-detected image into the target image detection model for detection to determine whether the to-be-detected image is a real image.

[0142] It should be noted that the specific examples in the embodiment can refer to the examples described in the above embodiments and optional implementation manners, which will not be described herein again.

[0143] In addition, in combination with the image detection method based on the diffusion model generated by the convolutional neural network image reconstruction provided in the above embodiments, a storage medium can also be provided to implement the method in the embodiment. The storage medium stores a computer program; the computer program is executed by a processor to implement any of the image detection methods based on the diffusion model generated by the convolutional neural network image reconstruction in the above embodiments.

[0144] It should be noted that the information and data involved in the present application are all information and data authorized by the user or authorized by all parties, which will be used legally.

[0145] It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not to be taken in a limiting sense. All other embodiments falling within the scope of the application are contemplated and are within the scope of the application.

[0146] It is apparent that the drawings depicted are only a few examples of the application and that many other embodiments of the application can be made without departing from the scope of the application disclosed herein. Furthermore, it should be understood that the drawings and detailed description thereto are not indicative of every possible embodiment of the application. In fact, many modifications and variations to the application disclosed herein will occur to those skilled in the art, once advised of the application disclosed herein. All such modifications and variations are believed to be within the scope of the application and are intended to be within the scope of the application.

[0147] The word "example" is used herein to mean serving as an example, instance, or illustration. Any aspect or embodiment described herein as "example" is not necessarily to be construed as preferred or advantageous over other aspects or embodiments. The disclosure herein using "example" terminology is to be understood that at least one aspect or embodiment described herein is in a way an example, and is used to elucidate a specific implementation of the application. A single feature of a example implementation cannot, therefore, be necessarily interpreted as an example of all, but rather an example of at least one of the aspects or embodiments described herein.

[0148] The above-described embodiments are merely illustrative of several embodiments of the application and do not limit the scope of the application. It is apparent that many modifications and improvements can be made to the embodiments without departing from the scope of the application. Therefore, the scope of the application should be determined by the appended claims.

Claims

1. An image detection method based on a diffusion model for image reconstruction using convolutional neural networks, characterized in that, include: Obtain the target dataset, which includes real images, a first image, and a second image; The first image is determined by the diffusion of the real image; The second image is an enhanced version of the real image and the first image; The target dataset is reconstructed to obtain the training dataset; Based on the training dataset, the pre-set original image detection model is trained to obtain the target image detection model; The image to be detected is input into the target image detection model for detection to determine whether the image to be detected is a real image.

2. The image detection method based on diffusion model generated by convolutional neural network image reconstruction according to claim 1, characterized in that, The image to be detected is input into the target image detection model for detection to determine whether the image to be detected is a real image, including: Based on a pre-trained convolutional neural network, the image to be detected is reconstructed to obtain a reconstructed image; Determine the Euclidean distance between features of the image to be detected and the reconstructed image; If the Euclidean distance is greater than a preset threshold, then the image to be detected is determined to be a real image; If the Euclidean distance is less than or equal to a preset threshold, then the image to be detected is determined to be a diffusion image.

3. The diffusion model-based image detection method based on convolutional neural network image reconstruction according to claim 2, characterized in that, The threshold is positively correlated with the hyperparameters in the second loss function used to train the original image detection model.

4. The image detection method based on diffusion model generated by convolutional neural network image reconstruction according to claim 1, characterized in that, The original image detection model includes a deep learning sub-model and a fully connected layer; Based on the training dataset, a pre-defined original image detection model is trained to obtain a target image detection model, including: Based on the training dataset and the preset total loss function, the deep learning sub-model and the fully connected layer in the original image detection model are iteratively trained to obtain the target image detection model; the total loss function includes a first loss function and a second loss function; the first loss function corresponds to the deep learning sub-model; the second loss function corresponds to the fully connected layer.

5. The diffusion model-based image detection method based on convolutional neural network image reconstruction according to claim 4, characterized in that, Based on the training dataset and a preset total loss function, the deep learning sub-model and the fully connected layer in the original image detection model are iteratively trained to obtain the target image detection model, including: Freeze the fully connected layer, and train the deep learning sub-model based on the training dataset and the first loss function; Freeze the deep learning sub-model, and train the deep learning sub-model based on the training dataset and the second loss function; The target image detection model is obtained by continuing to train until the cutoff condition for iterative training is met.

6. The diffusion model-based image detection method based on convolutional neural network image reconstruction according to claim 1, characterized in that, Obtain the target dataset, including: Obtain real images, input the text descriptions of the real images into the input diffusion model to generate the first image; and add labels to the real images and the first image to obtain the original dataset; The real image and the first image in the original dataset are enhanced to obtain the second image; The second image is added to the original dataset to obtain the target dataset.

7. The image detection method based on diffusion model generated by convolutional neural network image reconstruction according to claim 1, characterized in that, The target dataset is reconstructed to obtain the training dataset, which includes: The target dataset is reconstructed based on a pre-trained convolutional neural network to obtain a training dataset.

8. An image detection device based on a diffusion model for image reconstruction using a convolutional neural network, characterized in that, include: The module includes an acquisition module, a reconstruction module, a training module, and a detection module. The acquisition module is used to acquire a target dataset, which includes a real image, a first image, and a second image; The first image is determined by the diffusion of the real image; The second image is an enhanced version of the real image and the first image; The reconstruction module is used to reconstruct the target dataset to obtain the training dataset; The training module is used to train a preset original image detection model based on a training dataset to obtain a target image detection model. The detection module is used to input the image to be detected into the target image detection model for detection, and to determine whether the image to be detected is a real image.

9. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the diffusion model-based image detection method for image reconstruction according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the diffusion model-based image detection method for image reconstruction based on convolutional neural networks as described in any one of claims 1 to 7.

Images