Image detection method and device, equipment and storage medium

Abstract

The embodiment of the invention provides an image detection method and device, equipment and a storage medium. The method comprises the following steps: inputting a current image into a pre-trained diffusion model, and obtaining a reconstructed image under a target category under the guidance of a pre-trained category determination model, the target category being any one of an intelligent synthesis category and a real shooting category; and determining the category of the current image according to the difference characteristics between the current image and the reconstructed image. According to the embodiment of the invention, accurate detection of the category of any image can be realized to accurately distinguish the intelligent composite image from the real shot image, illegal use of the intelligent composite image on the network is avoided, and the information security when the image is used is ensured.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and specifically to an image detection method, apparatus, device and storage medium. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, a series of advanced image synthesis tools have emerged to assist users in creating a variety of realistic images using AI technology to enhance image diversity. However, since such image synthesis tools have greatly lowered the threshold for image creation, most users can easily generate realistic images and use them on the Internet to impersonate, forge information, infringe copyrights, and other harmful behaviors, causing great damage to information security.

[0003] Therefore, it is urgent to design an image detection solution to determine whether any image is a realistic image created using AI technology to ensure the authenticity of the image. Summary of the invention

[0004] The embodiments of the present application provide an image detection method, apparatus, device and storage medium to achieve accurate detection of the category to which any image belongs, to accurately distinguish between intelligently synthesized images and real captured images, and to ensure information security when the image is used.

[0005] In a first aspect, an embodiment of the present application provides an image detection method, the method comprising:

[0006] Inputting the current image into a pre-trained diffusion model, and obtaining a reconstructed image of a target category under the guidance of a pre-trained category determination model, wherein the target category is any one of an intelligent synthesis category and a real shooting category;

[0007] The category to which the current image belongs is determined according to the difference features between the current image and the reconstructed image.

[0008] In a second aspect, an embodiment of the present application provides an image detection device, the device comprising:

[0009] An image reconstruction module, used to input the current image into a pre-trained diffusion model, and obtain a reconstructed image under a target category under the guidance of a pre-trained category determination model, wherein the target category is any one of an intelligent synthesis category and a real shooting category;

[0010] The image category detection module is used to determine the category to which the current image belongs based on the difference features between the current image and the reconstructed image.

[0011] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0012] A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the image detection method provided in the first aspect of the present application.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute the image detection method provided in the first aspect of the present application.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the image detection method provided in the first aspect of the present application.

[0015] The technical solution provided by the embodiment of the present application can input the current image into a pre-trained diffusion model for any image, and after being guided by the pre-trained category determination model, the current image is diffused toward any target category of the intelligent synthesis category and the real shooting category configured by the diffusion model to obtain a reconstructed image under the target category. Then, by analyzing the difference features between the current image and the reconstructed image, the category to which the current image belongs can be determined, thereby achieving accurate detection of the category to which any image belongs, accurately distinguishing between intelligent synthesis images and real shooting images, avoiding illegal use of intelligent synthesis images on the Internet, and ensuring information security when using images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flowchart of an image detection method provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of the principle of the image detection process provided in the embodiment of the present application;

[0019] Figure 3 A flowchart of another image detection method shown in an embodiment of the present application;

[0020] Figure 4 A schematic diagram of the network structure of the category determination model provided in the embodiment of the present application;

[0021] Figure 5 A principle block diagram of an image detection device shown in an embodiment of the present application;

[0022] Figure 6 A schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] Considering that the threshold for creating smart composite images is low, most users can easily use smart composite images to perform various operations on the Internet, causing great damage to information security. To this end, this application designs an image detection scheme to accurately detect the category of any image to determine whether any image is a smart composite image or a real shot image, thereby accurately distinguishing smart composite images from real shots, avoiding illegal use of smart composite images on the Internet, and ensuring information security when using images.

[0026] Figure 1A flowchart of an image detection method provided in an embodiment of the present application. The method can be performed by an image detection device provided in the present application. Wherein, the image detection device can be implemented by any software and / or hardware. Exemplarily, the image detection device can be applied to any electronic device, and the electronic device may include but is not limited to tablet computers, mobile phones (such as folding screen mobile phones, large screen mobile phones, etc.), wearable devices, vehicle-mounted devices, laptop computers, ultra-mobile personal computers (ultra-mobile personal computers, UMPC), netbooks, personal digital assistants (personal digital assistants, PDA), smart TVs, smart screens, high-definition televisions, 4K TVs, smart speakers, smart projectors and other types of computing devices, and the present disclosure does not impose any restrictions on the specific types of electronic devices.

[0027] Specifically, Figure 1 As shown, the method may include the following steps:

[0028] S110, inputting the current image into a pre-trained diffusion model, and obtaining a reconstructed image of a target category under the guidance of the pre-trained category determination model, where the target category is any one of an intelligent synthesis category and a real shooting category.

[0029] The current image may be any image on the Internet. Before using any image on the Internet to perform various operations, it is possible to ensure the information security of the image when it is used by determining whether the image is an intelligently synthesized image or a real shot image.

[0030] The diffusion model is an advanced image generation model based on deep neural networks, also known as a denoising diffusion probability model, which creates new image samples by imitating the natural diffusion process of image data.

[0031] Specifically, the entire process of generating a new image using the diffusion model can be divided into two parts: the forward denoising process and the reverse denoising process. In the forward denoising process, by continuously adding different amounts of noise to the original image, the data distribution of the original image can be gradually diffused to the white noise state, and the corresponding white noise image can be obtained. Then, in the reverse denoising process, a neural network can be used to continuously denoise the white noise image to gradually reduce the noise component in the white noise image until a clear image with no noise at all is obtained as the generated new image.

[0032] Moreover, the neural network used by the diffusion model in the reverse denoising process to continuously denoise the white noise image can be a network under an encoder-decoder structure. The encoder-decoder structure is a deep learning model architecture for processing image data, which is generally suitable for extracting feature data from an input image and generating corresponding output tasks using the extracted feature data, such as image segmentation, image generation, image super-resolution, etc.

[0033] The encoder can be composed of a series of stacked convolutional layers to extract feature data of the input image. Each convolutional layer in the encoder generates a set of feature maps for the input image. The size of the feature maps will gradually decrease as the network goes deeper, and the input image will be decomposed into more abstract feature representations to capture deeper features of the input image. The decoder can be composed of a series of stacked upsampling or transposed convolutional layers to gradually upsample and refine the high-level features extracted by the encoder. The size of the feature maps will gradually increase as the network goes deeper to reconstruct the image or generate the required output task.

[0034] Then, for the current image, the diffusion model can be used to process the current image to generate a new image. However, since the diffusion model is trained with the goal of reconstructing a clear image, the new image generated by the diffusion model is a clear image, whether it is an intelligently synthesized image or a real shot image, and cannot directly reflect the category of the new image, and it is also impossible to use the category of the new image to analyze the category of the current image.

[0035] As for analyzing the category of the current image with the help of the category of the new image, in order to ensure accurate detection of the category to which the image belongs, it is first necessary to construct a discriminative detection feature and use the discriminative detection feature to train a binary classification network to determine whether the current image is an intelligently synthesized image or a real shot image.

[0036] It is understandable that the intelligently synthesized image and the real shot image are respectively in different data distribution domains.

[0037] Assume that the distribution domain of the intelligent synthetic image is p g (x), the distribution domain of the real captured image is p r (x). Then, if the current image is a real captured image, it can be regarded as the distribution domain p r (x). The current image is forward-noised by the diffusion model to obtain a white noise image, and then the white noise image is reversely denoised to reconstruct a new image. If the new image generated by the diffusion image is an intelligent synthetic image, then the distribution domain p of the new image relative to the current image is r(x), will produce a larger deviation and be in the distribution domain p g (x), so that the reconstructed new image is significantly different from the current image.

[0038] If the current image is an intelligent synthetic image, it can be regarded as the distribution domain p g A sampling point in (x). The current image is forward-noised by the diffusion model to obtain a white noise image, and then the white noise image is reversely denoised to reconstruct a new image. If the new image is still an intelligently synthesized image, then the new image is still in the distribution domain p g (x), so that the offset between the reconstructed new image and the current image is not large.

[0039] From the above content, it can be seen that the present application can use the feature difference between the current image and the reconstructed new image as the above-mentioned discriminative detection feature to analyze the category to which the current image belongs with the help of the category to which the new image belongs.

[0040] However, for the diffusion model, in the forward denoising process, by continuously adding different amounts of noise to the current image, the corresponding white noise image can be obtained. In the reverse denoising process, when the white noise image is continuously denoised to reconstruct the corresponding new image, the neural network composed of the encoder-decoder structure used is trained on some public data sets with the goal of "reconstructing a clear image". Then, the neural network used by the diffusion model in the reverse denoising process only uses image samples during training, and does not use the category label of the image sample as supervisory information. That is, the training goal of the neural network is to add noise from the original image sample to the white noise state, and then gradually denoise the white noise image to restore it to a clear image. In this process, no "image sample is an intelligently synthesized image or a real shot image" label is used for supervised training. Therefore, the new image reconstructed by the diffusion model, whether it is an intelligently synthesized image or a real shot image, will be a clear image, so that the diffusion model trained with the goal of "reconstructing a clear image" cannot directly reflect whether the new image is an intelligently synthesized image or a real shot image when reconstructing a new image in the reverse denoising process. Therefore, based on the technical solution provided in the present application, the present application first needs to be able to determine the category to which the new image reconstructed by the diffusion model belongs, so as to subsequently use the feature difference between the current image and the reconstructed new image as the above-mentioned distinctive detection feature to analyze the category to which the current image belongs with the help of the category to which the new image belongs.

[0041] Therefore, in order to ensure the certainty of the category to which the diffusion model belongs when reconstructing a new image, the present application can additionally collect a series of image sample sets with "intelligent synthetic image / real shot image" label information, and use the image sample set to train a category determination model. Then, using the category determination model, during the reverse denoising process of the diffusion model, it can be determined whether the intermediate image obtained after each denoising of the white noise image is an intelligent synthetic image or a real shot image, thereby guiding the diffusion model to iteratively denoise and reconstruct the forward-noised white noise image in the direction of a specific category of images among the intelligent synthetic image and the real shot image during the reverse denoising process, to ensure that the new image reconstructed by the diffusion model must be an image of a specific category among the intelligent synthetic image and the real shot image.

[0042] Specifically, in order to accurately detect the category to which any image belongs, the present application can specify a target category in the diffusion model so that the diffusion model can generate a new image in the direction of the target category to determine the category to which the new image belongs. The target category can be any one of the smart synthesis category and the real shooting category, so that the current image can be analyzed as a smart synthesis image or a real shooting image based on whether the new image is a smart synthesis image or a real shooting image.

[0043] In order to ensure that the diffusion model can generate new images in the direction of the target category, the present application can pre-train a category determination model to determine whether the intermediate image obtained after each denoising of the white noise image is an intelligent synthetic image or a real shot image during the reverse denoising process of the diffusion model, and update the denoising parameters in the diffusion model according to the difference between the category of the image after this denoising and the target category, so that the intermediate image obtained after the next denoising is closer to the target category. This cycle is repeated to guide the diffusion model to generate new images in the direction of the target category.

[0044] It can be seen that for the current image, the present application can first input the current image into the pre-trained diffusion model, and in the forward denoising process of the diffusion model, continuously add noise to the current image to obtain the corresponding white noise image. Then, in the reverse denoising process of the diffusion model, the white noise image can be continuously denoised to gradually reduce the noise components in the white noise image, and the category to which the intermediate image obtained after each denoising belongs is determined by the pre-trained category determination model, so that the diffusion model updates the corresponding denoising parameters according to the difference between the category to which the image after this denoising belongs and the target category, so that the intermediate image obtained after the next denoising is closer to the target category, thereby guiding the diffusion model to continuously denoise the white noise image in the direction of the target category in the reverse denoising process, until a clear image that is completely noise-free and belongs to the target category is obtained, as the reconstructed image under the target category in the present application.

[0045] It should be noted that the present application can use a pre-trained category determination model as a guiding condition for the output of the category to which the intermediate image obtained by the diffusion model after each denoising process in the reverse denoising process belongs, to guide the diffusion model to continuously denoise the intermediate image in the direction of the target category in the reverse denoising process, and output a reconstructed image under the target category.

[0046] The above method does not require additional training of the diffusion model. Based on the original trained diffusion model, the diffusion model is guided by the external category determination model to generate the reconstructed image under the desired target category. The only part of the diffusion model that needs to be changed is the sampling mean of the white noise image during the denoising sampling process. That is, during the denoising sampling process, the closer the sampling center of the expected white noise image is to the condition guided by the category determination model (that is, the target category), the better, thereby configuring the guiding condition (that is, the target category) into the diffusion model.

[0047] Among them, the sampling algorithm used in the diffusion model can be to directly predict the sampling mean and variance of each intermediate image obtained by continuous denoising of the white noise image in the reverse denoising process of the diffusion model, or directly predict the noise error of each intermediate image, so as to change the mean center of the intermediate image under the Gaussian distribution, to guide the denoising direction of the white noise image to the direction of the target category, thereby obtaining a reconstructed image under the target category.

[0048] S120, determining the category to which the current image belongs according to the difference features between the current image and the reconstructed image.

[0049] Since the reconstructed image under the target category is obtained by adding forward noise N times and removing backward noise N times in the diffusion model, the reconstructed image and the current image may have different categories, but their contents are very similar. Therefore, the feature difference between the current image and the reconstructed image is caused by their categories.

[0050] Therefore, in order to accurately detect the category to which the current image belongs, the present application can first perform corresponding feature extraction on the current image and the reconstructed image to determine the first feature of the current image and the second feature of the reconstructed image. Then, by comparing the first feature of the current image and the second feature of the reconstructed image, the difference feature between the current image and the reconstructed image can be obtained.

[0051] Furthermore, by analyzing the difference features between the current image and the reconstructed image, the feature offset between the current image and the reconstructed image can be determined. If the feature offset is less than or equal to the preset offset threshold, it means that the current image and the reconstructed image belong to the same category, and the category to which the current image belongs can be determined to be the target category. If the feature offset is greater than the preset offset threshold, it means that the current image and the reconstructed image belong to different categories, and the category to which the current image belongs can be determined to be another category other than the target category. According to the above method, it is possible to accurately determine whether the current image is an intelligently synthesized image or a real shot image, thereby accurately distinguishing between intelligently synthesized images and real shot images on the network.

[0052] In some implementations, the present application can obtain a large number of original images and determine the category to which each original image belongs. Then, by inputting each original image into the above diffusion model, a reconstructed image under the target category corresponding to each original image can be obtained.

[0053] Therefore, the present application can construct each original image and the reconstructed image corresponding to the original image as a training sample pair, and use the category to which the original image belongs as the sample label. Then, a classifier is trained through the difference features between the two images in each training sample pair and the corresponding sample labels to output the predicted category of the original image in each training sample pair. Then, after the classifier is trained, the present application can use the current image and the reconstructed image as an image pair and input them into the trained classifier, which will analyze the difference features between the current image and the reconstructed image and accurately output the category to which the current image belongs.

[0054] The technical solution provided by the embodiment of the present application can input the current image into a pre-trained diffusion model for any image, and after being guided by the pre-trained category determination model, the current image is diffused toward any target category of the intelligent synthesis category and the real shooting category configured by the diffusion model to obtain a reconstructed image under the target category. Then, by analyzing the difference features between the current image and the reconstructed image, the category to which the current image belongs can be determined, thereby achieving accurate detection of the category to which any image belongs, accurately distinguishing between intelligent synthesis images and real shooting images, avoiding illegal use of intelligent synthesis images on the Internet, and ensuring information security when using images.

[0055] As an optional implementation scheme in the present application, in order to support accurate diffusion reconstruction of the current image in the diffusion model, for the forward denoising process and the reverse denoising process in the diffusion model, as follows: Figure 2 As shown, the diffusion model may include a forward diffusion network and a backward denoising network.

[0056] Specifically, the forward diffusion network may correspond to the forward denoising process, which is used to continuously perform the corresponding forward denoising operation on the current image, so that the data distribution of the current image gradually diffuses to the white noise state, and obtains the corresponding white noise image. The backward denoising network may correspond to the reverse denoising process, which is used to continuously denoise the white noise image output by the forward diffusion network, so as to gradually reduce the white noise component in the white noise image, until a clear image without any noise is obtained, which is used as the newly generated reconstructed image after the current image passes through the diffusion model.

[0057] Moreover, considering that the diffusion model mainly generates a reconstructed image corresponding to the current image in the reverse denoising process, in order to ensure that the current image generates a reconstructed image in the direction of the target category, the present application can configure at least a target category in the backward denoising network to support continuous denoising of the white noise image in the direction of the target category to generate a corresponding reconstructed image.

[0058] Next, the present application may provide a detailed description of the specific process of the diffusion model, under the guidance of the pre-trained category determination model, processing the current image to obtain a reconstructed image under the target category.

[0059] Figure 3 FIG. 1 is a flow chart of another image detection method shown in an embodiment of the present application. Figure 3 As shown, the method may specifically include the following steps:

[0060] S310, input the current image into the forward diffusion network to obtain a corresponding white noise image.

[0061] For the current image, Figure 2 As shown, the present application can first input the current image into the forward diffusion network of the diffusion model, and continuously add noise to the current image through the forward diffusion network, so that the data distribution of the current image gradually diffuses to the white noise state to obtain the corresponding white noise image.

[0062] S320, inputting the white noise image into the backward denoising network, using the pre-trained category determination model to guide the white noise image to be continuously denoised in the direction of the target category, and obtaining a reconstructed image under the target category.

[0063] After the white noise image corresponding to the current image is output through the forward diffusion network, the present application can input the white noise image into the backward denoising network, and the white noise image is continuously denoised through the backward denoising network to gradually reduce the noise components in the white noise image. Moreover, after each denoising of the white noise image by the backward denoising network, the category to which the intermediate image obtained after each denoising can be determined by the pre-trained category determination model, so that the backward denoising network updates the corresponding denoising parameters according to the difference between the category to which the image after this denoising belongs and the target category, so that the category to which the intermediate image obtained after the next denoising belongs can be closer to the target category. In this way, in the backward denoising network, the category determination model can be used to continuously guide the white noise image to be denoised in the direction of the target category until a clear image completely free of noise is obtained, and the category to which the clear image belongs can be the target category. Then, the present application can use the clear image finally output by the backward denoising network as the reconstructed image under the target category in the present application.

[0064] In some implementable embodiments, in order to ensure the accuracy of the reconstructed image, the present application can determine the reconstructed image under the target category through the following steps: inputting the white noise image into the backward denoising network as the initial image to be denoised; inputting the image to be denoised into the pre-trained category determination model to obtain the category determination features of the image to be denoised to determine the classification gradient corresponding to the image to be denoised; using the classification gradient to guide the backward denoising network to perform a denoising on the image to be denoised in the direction of the target category to obtain a new image to be denoised, and continue to perform a new denoising in the direction of the target category until the new image to be denoised does not have noise, and then the latest image to be denoised is used as the reconstructed image under the target category.

[0065] Specifically, after obtaining the white noise image corresponding to the output current image through the forward diffusion network, the present application can input the white noise image into the backward denoising network to perform the subsequent reverse denoising process. Considering that the white noise image needs to be denoised N times in a loop in the backward denoising network to gradually reduce the noise component in the white noise image. Therefore, in the backward denoising network, the present application can use the white noise image as the initial image to be denoised.

[0066] Then, in order to ensure that the image to be denoised can be denoised in the direction of the target category, the present application can input the image to be denoised into a pre-trained category determination model, and perform corresponding feature analysis on the image to be denoised through the category determination model to extract the category determination features of the image to be denoised, so as to determine the category to which the image to be denoised belongs. Moreover, by analyzing the change rate or cross entropy loss of the category determination features of the image to be denoised relative to the target category, the classification gradient corresponding to the image to be denoised is determined, and the classification gradient is input into the backward denoising network.

[0067] Then, when the denoising image is denoised by the backward denoising network, the classification gradient corresponding to the denoised image and the pre-set gradient scaling coefficient can be used to update the denoising related parameters in the backward denoising network to guide the backward denoising network to denoise the denoised image in the direction of the target category to obtain a new denoised image. Among them, by scaling the corresponding classification gradient by the gradient scaling coefficient, the diversity and accuracy of the newly generated denoised image can be controlled. Then, the new denoised image is continuously input into the pre-trained category determination model to obtain the category determination feature of the new denoised image to determine the classification gradient corresponding to the new denoised image, and the new classification gradient is used to guide the backward denoising network to continue to denoise the new denoised image in the direction of the target category. This cycle is repeated to gradually reduce the noise component in the white noise image until the new denoised image does not have noise and the cycle is stopped. Then, the latest denoised image is used as the reconstructed image under the target category.

[0068] S330, determining the category to which the current image belongs according to the difference features between the current image and the reconstructed image.

[0069] The technical solution provided by the embodiment of the present application can input the current image into a pre-trained diffusion model for any image, and after being guided by the pre-trained category determination model, the current image is diffused toward any target category of the intelligent synthesis category and the real shooting category configured by the diffusion model to obtain a reconstructed image under the target category. Then, by analyzing the difference features between the current image and the reconstructed image, the category to which the current image belongs can be determined, thereby achieving accurate detection of the category to which any image belongs, accurately distinguishing between intelligent synthesis images and real shooting images, avoiding illegal use of intelligent synthesis images on the Internet, and ensuring information security when using images.

[0070] According to one or more embodiments of the present application, for any current image, in order to ensure accurate detection of the category to which any image belongs, it is necessary to ensure that when the current image is input into the pre-trained diffusion model, the diffusion model can accurately generate a reconstructed image under the target category under the guidance of the category determination model. That is, the ability of the category determination model to guide the diffusion model to continuously denoise the white noise image corresponding to the current image in the direction of the target category will directly affect whether the category to which the reconstructed image belongs is the target category, and thus affect the accurate detection of the category to which the current image belongs.

[0071] Therefore, in order to accurately distinguish whether any image is an intelligently synthesized image or a real shot image, the present application may introduce the specific model structure of the category determination model. Figure 4As shown, the category determination model in the present application can be composed of an encoder network, a routing network and multiple expert networks connected in parallel.

[0072] The encoder network and each expert network may be a deep neural network stacked by multiple convolutional layers, used to extract corresponding image features. The routing network may be a multi-layer perceptron network stacked by multiple fully connected layers and an activation function, used to configure corresponding weights for each expert network. Moreover, the output layer in the category determination model is used to perform weighted processing on the image features extracted by each expert network using the corresponding weights of each expert network to obtain the corresponding category determination features.

[0073] Specifically, after the current image is input into the diffusion model, a corresponding white noise image can be generated in the forward denoising process. Then, in the reverse denoising process, the white noise image can be continuously denoised under the guidance of the category determination model. At this time, the input of the category determination model can be a new denoised image obtained after each denoising of the white noise image.

[0074] Then, for the new denoised image obtained after each denoising of the white noise image, the present application can first input the denoised image into the encoder network in the above-mentioned category determination model, and perform corresponding feature processing on the denoised image through the encoder network to extract the initial features of the denoised image. Then, the initial features of the denoised image are respectively input into the routing network and each expert network, and the routing network performs corresponding feature analysis on the initial features of the denoised image to assign corresponding weights to each expert network. Moreover, by performing multi-level feature processing on the initial features of the denoised image by each expert network, the detailed image features extracted by each expert network can be obtained.

[0075] Furthermore, the output layer in the category determination model can be used to utilize the weights assigned to each expert network by the routing network to perform corresponding weighted processing on the detailed image features extracted by each expert network, thereby obtaining the category determination features of the image to be denoised, and determining the classification gradient corresponding to the image to be denoised, so as to subsequently utilize the classification gradient to guide the diffusion model to denoise the image to be denoised in the direction of the target category during the reverse denoising process, until a clear image completely free of noise is obtained as the reconstructed image under the target category.

[0076] In some possible implementations, in order to achieve accurate determination of the category to which any image belongs by the category determination model, it is first necessary to accurately obtain the training samples of the category determination model to ensure accurate training of the category determination model. Therefore, the present application can determine the training samples of the category determination model in the following manner: for each historical image in the image set, the category to which the historical image belongs is used as the corresponding category label; the historical image is input into the forward diffusion network of the diffusion model to obtain a noise image at least in one time step, and each noise image and the category label are combined into a training sample pair of the category determination model.

[0077] Specifically, after the current image is input into the diffusion model, a corresponding white noise image can be generated in the forward denoising process, and the white noise image is continuously denoised in the reverse denoising process to generate a reconstructed image under the target category. The main purpose of the category determination model in the present application is to determine whether the intermediate image obtained after each denoising of the white noise image is an intelligent synthetic image or a real shot image in the reverse denoising process of the diffusion model. It can be seen that the image that the category determination model faces is usually an intermediate image of the white noise image that still has a certain noise component after each denoising. Then, in order to ensure the accurate training of the category determination model, it can be known that the training samples of the category determination model should also be noise images with certain noise components.

[0078] Therefore, the present application can collect a large number of historical images to form the image set in the present application, and mark the category to which each historical image belongs. Among them, each historical image in the image set is a clear image with no noise at all.

[0079] For each historical image in the image set, the present application can first determine the category to which the historical image belongs as the corresponding category label. Moreover, the historical image is input into the forward diffusion network of the above-mentioned diffusion model, and noise can be continuously added to the historical image through the forward diffusion network, so that the data distribution of the historical image gradually diffuses to the white noise state, and the corresponding white noise image is obtained. At this time, the present application can use the intermediate image with a certain amount of noise obtained after each noise addition of the historical image as the noise image of the historical image at each time step in the forward noise addition process.

[0080] Thus, for each historical image, the present application can obtain a noise image of the historical image at at least one time step in the forward noise addition process. Then, the noise image obtained at each time step is used as the corresponding sample image, and the category label corresponding to the historical image is used as the corresponding sample label, and the two constitute a training sample pair of the category determination model. The training sample can include multiple sample images and the same sample label.

[0081] In the same manner as above, for each historical image, a training sample pair can be determined for the category determination model, thereby accurately obtaining a category determination model training sample set to ensure accurate training of the category determination model.

[0082] Figure 5 FIG. 1 is a block diagram of an image detection device according to an embodiment of the present application. Figure 5 As shown, the apparatus 500 may include:

[0083] An image reconstruction module 510 is used to input the current image into a pre-trained diffusion model, and obtain a reconstructed image of a target category under the guidance of a pre-trained category determination model, wherein the target category is any one of an intelligent synthesis category and a real shooting category;

[0084] The image category detection module 520 is used to determine the category to which the current image belongs according to the difference features between the current image and the reconstructed image.

[0085] In some implementations, the diffusion model includes a forward diffusion network and a backward denoising network, and the backward denoising network is configured with at least a target category.

[0086] In some implementations, the image reconstruction module 510 may include:

[0087] A forward noise adding unit, used for inputting the current image into the forward diffusion network to obtain a corresponding white noise image;

[0088] The backward denoising unit is used to input the white noise image into the backward denoising network, and use the pre-trained category determination model to guide the white noise image to be continuously denoised in the direction of the target category to obtain a reconstructed image under the target category.

[0089] In some implementations, the backward denoising unit may be specifically used to:

[0090] Inputting the white noise image into the backward denoising network as an initial image to be denoised;

[0091] Inputting the image to be denoised into the pre-trained category determination model to obtain a category determination feature of the image to be denoised, so as to determine a classification gradient corresponding to the image to be denoised;

[0092] The classification gradient is used to guide the backward denoising network to denoise the image to be denoised in the direction of the target category to obtain a new image to be denoised, and a new denoising is continued in the direction of the target category until the new image to be denoised is free of noise, and the latest image to be denoised is used as the reconstructed image under the target category.

[0093] In some possible implementations, the category determination model is composed of an encoder network and a routing network and multiple expert networks connected in parallel; the encoder network and each of the expert networks are deep neural networks composed of multiple convolutional layers stacked together, which are used to extract corresponding image features; the routing network is a multi-layer perceptron network composed of multiple fully connected layers and an activation function stacked together, which is used to configure corresponding weights for each expert network.

[0094] In some possible implementations, the output layer in the category determination model is used to perform weighted processing on the image features extracted by the expert network using the weights corresponding to the expert network to obtain corresponding category determination features.

[0095] In some implementations, the training samples of the category determination model are determined by a sample acquisition module. The sample acquisition module can be used to:

[0096] For each historical image in the image set, the category to which the historical image belongs is used as a corresponding category label;

[0097] The historical image is input into the forward diffusion network of the diffusion model to obtain a noise image at least one time step, and each noise image and the category label are combined into a training sample pair of the category determination model.

[0098] In the embodiment of the present application, for any image, the current image can be input into the pre-trained diffusion model, and guided by the pre-trained category determination model, the current image is diffused toward any target category of the intelligent synthesis category and the real shooting category configured by the diffusion model to obtain a reconstructed image under the target category. Then, by analyzing the difference features between the current image and the reconstructed image, the category to which the current image belongs can be determined, thereby achieving accurate detection of the category to which any image belongs, accurately distinguishing between intelligent synthesis images and real shooting images, avoiding illegal use of intelligent synthesis images on the Internet, and ensuring information security when using images.

[0099] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 5 The device 500 shown can execute any method embodiment provided in the present application, and the aforementioned and other operations and / or functions of each module in the device 500 are respectively for implementing the corresponding processes in each method of the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0100] The above describes the device 500 of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0101] Figure 6 A schematic block diagram of an electronic device provided in an embodiment of the present application.

[0102] like Figure 6 As shown, the electronic device 600 may include:

[0103] The memory 610 and the processor 620, the memory 610 is used to store the computer program and transmit the program code to the processor 620. In other words, the processor 620 can call and run the computer program from the memory 610 to implement the method in the embodiment of the present application.

[0104] For example, the processor 620 may be configured to execute the above method embodiments according to instructions in the computer program.

[0105] In some embodiments of the present application, the processor 620 may include but is not limited to:

[0106] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0107] In some embodiments of the present application, the memory 610 includes but is not limited to:

[0108] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0109] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 610 and executed by the processor 620 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0110] like Figure 6 As shown, the electronic device may also include:

[0111] The transceiver 630 may be connected to the processor 620 or the memory 610 .

[0112] The processor 620 may control the transceiver 630 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 630 may include a transmitter and a receiver. The transceiver 630 may further include an antenna, and the number of antennas may be one or more.

[0113] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0114] The embodiment of the present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the embodiment of the present application also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0115] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0116] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0117] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0118] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0119] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image detection method, characterized in that: include: Inputting the current image into a pre-trained diffusion model, and obtaining a reconstructed image of a target category under the guidance of a pre-trained category determination model, wherein the target category is any one of an intelligent synthesis category and a real shooting category; The category to which the current image belongs is determined according to the difference features between the current image and the reconstructed image.

2. The method according to claim 1, characterized in that: The diffusion model includes a forward diffusion network and a backward denoising network, and the backward denoising network is at least configured with a target category.

3. The method according to claim 2, characterized in that The current image is input into the pre-trained diffusion model, and under the guidance of the pre-trained category determination model, a reconstructed image of the target category is obtained, including: Inputting the current image into the forward diffusion network to obtain a corresponding white noise image; The white noise image is input into the backward denoising network, and the pre-trained category determination model is used to guide the white noise image to be continuously denoised in the direction of the target category to obtain a reconstructed image under the target category.

4. The method according to claim 3, characterized in that The step of inputting the white noise image into the backward denoising network, and using the pre-trained category determination model to guide the white noise image to be continuously denoised in the direction of the target category to obtain a reconstructed image under the target category includes: Inputting the white noise image into the backward denoising network as an initial image to be denoised; Inputting the image to be denoised into the pre-trained category determination model to obtain a category determination feature of the image to be denoised, so as to determine a classification gradient corresponding to the image to be denoised; The classification gradient is used to guide the backward denoising network to denoise the image to be denoised in the direction of the target category to obtain a new image to be denoised, and a new denoising is continued in the direction of the target category until the new image to be denoised is free of noise, and the latest image to be denoised is used as the reconstructed image under the target category.

5. The method according to claim 1, characterized in that The category determination model is composed of an encoder network, a routing network and a plurality of expert networks connected in parallel; The encoder network and each of the expert networks are deep neural networks formed by stacking multiple convolutional layers, and are used to extract corresponding image features; The routing network is a multi-layer perceptron network formed by stacking multiple fully connected layers and an activation function, and is used to configure corresponding weights for each expert network.

6. The method according to claim 5, characterized in that The output layer in the category determination model is used to perform weighted processing on the image features extracted by the expert network using the weights corresponding to the expert network to obtain corresponding category determination features.

7. The method according to claim 1, characterized in that The training samples of the category determination model are determined in the following manner: For each historical image in the image set, the category to which the historical image belongs is used as a corresponding category label; The historical image is input into the forward diffusion network of the diffusion model to obtain a noise image at least one time step, and each noise image and the category label are combined into a training sample pair of the category determination model.

8. An image detection device, characterized in that: include: An image reconstruction module, used to input the current image into a pre-trained diffusion model, and obtain a reconstructed image under a target category under the guidance of a pre-trained category determination model, wherein the target category is any one of an intelligent synthesis category and a real shooting category; The image category detection module is used to determine the category to which the current image belongs based on the difference features between the current image and the reconstructed image.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the image detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the image detection method according to any one of claims 1 to 7.

Images