A method for constructing a generated image detection model based on incremental learning and a detection method

Abstract

The application discloses a kind of based on incremental learning's generation image detection model's construction method and detection method, model construction method includes: constructing image dataset, dataset includes several subsets, subset includes generation image and real image;Select the specified real generation image pair of maximum number in image dataset subset, input basic feature extractor and carry out pre-training processing, obtain loss function, based on loss function to basic feature extractor is optimized, obtains pre-training feature extractor;With pre-training feature extractor as incremental stage feature extractor, on this basis, incremental training is carried out, the loss function of incremental stage is designed, and the pre-training feature extractor of the t stage is trained by the loss function of incremental stage, until the remaining subset in image dataset is trained, and generation image detection model is obtained.The application not only improves the accuracy of generation image detection, but also has the ability of continuous learning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing in deep learning, and in particular to a construction method and detection method of a generated image detection model based on incremental learning. BACKGROUND

[0002] With the rapid development of artificial intelligence (AI), the generated model has made significant progress in generating images. A person without the skill of synthesizing images only needs to use an excellent generated image model to input some key text information, and can quickly generate a realistic image related to the text description. This unprecedented ease of use will largely cause the proliferation of generated images. At present, AI generated images have reached a realistic level that is difficult for human eyes to distinguish. However, this advanced technology has a large number of potential risks. For example, the widely spread AI generated image of the Pentagon on fire on Twitter deceived several important news media organizations, which in turn led to a serious decline in the US stock market.

[0003] With the continuous improvement of infrastructure, the development of artificial intelligence is booming, and the generated model is emerging, and the generated image distribution is various. Many researchers are committed to researching a general generated image detector, trying to detect all generated images generated by the generator, especially for unknown generators. Many studies have shown that there are obvious differences between the images generated by the generator and the natural images in the low-level pixel features, and the model can easily obtain a decision boundary that distinguishes them. However, for different generator architectures, such a decision boundary is different. Therefore, designing a general generated image detector across generators is a difficult and challenging task.

[0004] However, the general detector also has some shortcomings. For example, it cannot perceive future changes in real time, and there is a lot of uncertainty in detecting novel generators. Therefore, it is also important to design a detector with incremental learning function. SUMMARY

[0005] To solve the above technical problems, the present application provides a construction method and detection method of a generated image detection model based on incremental learning, which improves the accuracy of generated image detection.

[0006] In one aspect, to achieve the above purpose, the present application provides a construction method of a generated image detection model based on incremental learning, comprising:

[0007] Constructing an image data set, the data set comprising a plurality of subsets, the subsets comprising generated images and real images;

[0008] The subset with the largest number of specified real generated image pairs in the image dataset is selected, input into the basic feature extractor for pre-training processing, a loss function is obtained, the basic feature extractor is optimized based on the loss function, and a pre-training feature extractor is obtained;

[0009] The pre-training feature extractor is used as an incremental stage feature extractor, and incremental training is performed thereon, a loss function of the incremental stage is designed, and the pre-training feature extractor of the t-th stage is trained through the loss function of the incremental stage until the remaining subsets in the image dataset are trained, and a generated image detection model is obtained.

[0010] Preferably, the pre-training processing process comprises:

[0011] The subset with the largest number of specified real generated image pairs in the image dataset is selected, the processed subset is input into the basic feature extractor for feature extraction, global embedding vector representations of generated images and real images are obtained, real images and generated images are respectively used as anchor points, and a three-tuple loss function is constructed with Euclidean distance as a measurement criterion.

[0012] Preferably, the method for obtaining the global embedding vector representations of the generated images and the global vector representations of the real images is:

[0013]

[0014]

[0015] wherein, respectively represent the i-th real image and the j-th generated image, E represents the basic feature extractor, v i , e j respectively represent global embedding vector representations of the i-th real image and the j-th generated image after the basic feature extractor, represents a D-dimensional vector, is a global vector representation of a real image, is a global embedding vector representation of a generated image.

[0016] Preferably, the method for obtaining the loss function is:

[0017] L pretrain (v,e)=[α-d(v,v)+d(v,e)] + +[α-d(e,e)+d(e,v)] +

[0018] In the formula, [x] + ≡max(x,0), α is a threshold value, and L pretrain(v, e) is a loss function, d(v, v) is the Euclidean distance between each pair of vectors in set v, d(v, e) is the Euclidean distance between each pair of vectors in set v and set e, d(e, e) is the Euclidean distance between each pair of vectors in set e, and d(e, v) is the Euclidean distance between each pair of vectors in set e and set v.

[0019] Preferably, obtaining the pre-trained feature extractor comprises:

[0020] Based on the loss function, the base feature extractor is optimized by a back propagation algorithm to obtain a pre-trained feature extractor.

[0021] Preferably, after obtaining the pre-trained feature extractor, it further comprises:

[0022] Using the pre-trained feature extractor to calculate prototypes in combination with selected subsets and storing the calculation results in a knowledge base, specifically:

[0023]

[0024] wherein μ r , μ f represent prototypes of real images and generated images respectively, n1 is the number of real images, n2 is the number of generated images, and n1 = n2, represent the embedding representation of the i-th real image and the embedding representation of the i-th generated image respectively.

[0025] Preferably, the incremental training comprises:

[0026] Shuffling the remaining subsets after the pre-training phase is completed, selecting the first subset, and using the pre-trained feature extractor of the pre-training phase to calculate prototypes for the first subset, and using the calculated prototypes and the historical prototypes stored in the knowledge base to do nearest distance matching, and selecting the historical prototype with the smallest distance as the anchor point.

[0027] wherein if it is the first incremental phase, the pre-trained feature extractor is used to calculate prototypes; if it is the t-th incremental phase, the feature extractor of the t-1 phase and the subset of the t phase are used to calculate prototypes.

[0028] Preferably, the loss function of the incremental phase comprises a triplet loss of positive samples and negative samples, a triplet loss of a selected positive sample prototype from the knowledge base as an anchor point, and a triplet loss of a selected negative sample prototype from the knowledge base as an anchor point.

[0029] Preferably, the loss function of the incremental phase is:

[0030] L increment= [a + d(mu r,v i )-d(mu r,e i )] + +[a + d(mu f ,e i )-d(mu f ,v i )] + [ a + d(v i ,v i )-d(v i ,e i )] + +[a + d(e i ,e i )-d(e i ,v i )] +

[0031] where L increment is the loss function of the incremental stage, a is the threshold value, d represents the calculation of the Euclidean distance between each pair of vectors in two vector sets, v i and e i represent the embedding vector set of the real image in the i-th subset and the embedding vector set of the generated image in the i-th subset respectively, mu r is the historical prototype of the real image, mu f is the historical prototype of the generated image, d(mu r ,v i ) is the Euclidean distance between each pair of vectors in the real image prototype and the vector set v i , d(mu r ,e i ) is the Euclidean distance between each pair of vectors in the real image prototype and the vector set e i , d(mu f ,e i ) is the Euclidean distance between each pair of vectors in the generated image prototype and the vector set e i , d(mu f ,v i ) is the Euclidean distance between each pair of vectors in the generated image prototype and the vector set v i , d(v i ,v i ) is the Euclidean distance between each pair of vectors in the embedding vector set v i , d(v i ,e i ) is the Euclidean distance between each pair of vectors in the embedding vector set v i and the embedding vector set e i , d(e i ,e i ) is the Euclidean distance between each pair of vectors in the embedding vector set e iEuclidean distances between each pair of vectors in the embedding vector set e i i is an embedding vector set v i and an embedding vector set v i Euclidean distances between each pair of vectors in the embedding vector set e

[0032] In another aspect, to achieve the above object, the application further provides a generation image detection method based on incremental learning, comprising:

[0033] collecting a to-be-predicted image;

[0034] inputting the to-be-predicted image into a constructed generation image detection model to obtain an embedding vector, performing nearest neighbor matching on the embedding vector and a vector stored in a knowledge base to obtain a prototype image closest in distance, and outputting a detection result.

[0035] Compared with the prior art, the application has the following advantages and technical effects:

[0036] (1) Compared with the general cross-generator generation image detection method, it cannot perceive future changes in real time and is full of uncertainty when detecting novel generators. The detection method with incremental learning ability of the application can continuously and effectively learn new knowledge and gradually improve its recognition ability.

[0037] (2) The general cross-generator generation image detection method needs to collect a large number of images for training at one time and requires a large amount of computing resources, while the generation image detector based on incremental learning can complete training with only a small amount of data and requires less computing resources.

[0038] (3) For the catastrophic forgetting problem in incremental learning, the application re-designs the triplet loss function to ensure that the distance between similar samples is narrowed and the distance between dissimilar samples is widened in the process of continuous incremental learning.

[0039] (4) The application can improve the accuracy of generation image detection, reduce the error degree of model prediction, and the model has the ability of continuous learning. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated herein in their entirety. The application illustratively described herein suitably can be practiced in the absence of any element or step not specifically disclosed. In the accompanying drawings:

[0041] Figure 1 is a flowchart of a generation image detection method based on incremental learning according to an embodiment of the application;

[0042] Figure 2 ​A prediction method flowchart when a use model of an embodiment of the present application is used as a feature extractor;

[0043] Figure 3 A generated image and a real image example graph for use of an embodiment of the present application. DETAILED DESCRIPTION

[0044] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0045] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0046] The present application proposes a generated image detection model construction method based on incremental learning, as shown in Figures 1-2 , comprising:

[0047] Constructing an image data set, the data set including a plurality of subsets, the subsets including generated images and real images;

[0048] Selecting a subset with the most specified real generated image pairs in the image data set, inputting a basic feature extractor for pre-training processing, obtaining a loss function, optimizing the basic feature extractor based on the loss function, and obtaining a pre-trained feature extractor;

[0049] Taking the pre-trained feature extractor as an incremental stage feature extractor, performing incremental training on this basis, designing a loss function for the incremental stage, and training the pre-trained feature extractor for the t stage through the loss function for the incremental stage until the remaining subsets in the image data set are trained, and obtaining a generated image detection model.

[0050] The present application can improve the accuracy of generated image detection, reduce the error degree of model prediction, and the model has the ability of continuous learning.

[0051] Further, the image data set is constructed, comprising:

[0052] Obtaining a data set, the data set including a plurality of subsets, the subsets including generated images and real images, wherein the generated images and the real images are generated by different generators. For example, Figure 3 .

[0053] Further, the pre-training process comprises:

[0054] A subset with the most specified real generated image pairs in the image dataset is selected, and the processed subset is input into the base feature extractor for feature extraction to obtain global embedding vector representations of the generated images and global embedding vector representations of the real images. The real images and the generated images are taken as anchor points respectively, and a Euclidean distance is taken as a measurement criterion to construct a triplet loss function.

[0055] In this embodiment, the input images in the subset are scaled to 224x224 pixel size, and the images are standardized according to image pixel mean and variance. The standardization method is as follows:

[0056]

[0057] wherein x is an image pixel value, μ is an image pixel mean, σ is an image pixel variance, and f(x) is a standardized image.

[0058] Further, the processed subset is input into the base feature extractor for feature extraction to obtain global embedding vector representations of the generated images and global vector representations of the real images. The specific method is as follows:

[0059]

[0060]

[0061] wherein, respectively represent the i-th real image and the j-th generated image, E represents the base feature extractor, and v i , e j respectively represent global embedding vector representations of the i-th real image and the j-th generated image after the base feature extractor, represents a D-dimensional vector, is a global vector representation of the real image, is a global embedding vector representation of the generated image. In this embodiment, the base feature extractor is a VIT (Vision Transformer).

[0062] The Euclidean distance between adjacent vectors in the subset is calculated, and the formula is as follows:

[0063] d(X, Y) = ||X-Y||2

[0064] In the formula, d(X, Y) is the Euclidean distance between adjacent vectors, and X and Y are different vectors in the subset.

[0065] Further, the method for obtaining the loss function is as follows:

[0066] L pretrain (v, e) = [α-d(v, v) + d(v, e)]+ +[a-d(e,e)+d(e,v)] +

[0067] where [x] + ≡max(x,0), a is a threshold, L pretrain (v,e) is a loss function, d(v,v) is the Euclidean distance between two vectors in set v, d(v,e) is the Euclidean distance between two vectors in set v and set e, d(e,e) is the Euclidean distance between two vectors in set e, and d(e,v) is the Euclidean distance between two vectors in set e and set v.

[0068] Using the loss function, the model is optimized by a back propagation algorithm to obtain a pre-trained feature extractor.

[0069] Further, after obtaining the pre-trained feature extractor, the method further comprises:

[0070] calculating prototypes using the pre-trained feature extractor in combination with the selected subsets and storing the calculation results in a knowledge base, specifically:

[0071]

[0072] wherein μ r , μ f represent prototypes of real images and generated images respectively, n1 is the number of real images, n2 is the number of generated images, and n1=n2, represent the embedding representation of the i-th real image and the embedding representation of the i-th generated image.

[0073] Further, the incremental training comprises:

[0074] shuffling the order of the remaining subsets after the pre-training phase is completed, selecting the first subset, and using the pre-trained feature extractor of the pre-training phase to calculate prototypes for the first subset, and using the calculated prototypes and the historical prototypes stored in the knowledge base to do nearest distance matching, and selecting the historical prototype with the smallest distance as an anchor point.

[0075] wherein if it is the first incremental phase, the pre-trained feature extractor is used to calculate prototypes; if it is the t-th incremental phase, the feature extractor of the t-1 phase and the subset of the t phase are used to calculate prototypes.

[0076] Specifically, in order to show that the training order of the subsets does not affect the model results, the training order of the subsets is shuffled before training, and the shuffled subsets are denoted as represents the k-th real picture of the i-th subset, represents the lth generated image of the ith subset. The remaining subsets are selected step by step for incremental training. The input image is scaled to 224x224 pixel size and standardized.

[0077] Further, the loss function of the incremental stage is designed by using the real image historical prototype and the generated image historical prototype selected from the knowledge base, and the pre-training feature extractor of the tth stage is trained by the loss function of the incremental stage until the training of all subsets is completed, and an image detection model is obtained.

[0078] The loss function of the incremental stage includes the triplet loss of positive samples and negative samples, the triplet loss of the positive sample prototype selected from the knowledge base as an anchor point, and the triplet loss of the negative sample prototype selected from the knowledge base as an anchor point, and is specifically:

[0079]

[0080]

[0081] d(X,Y)=||X-Y||2

[0082] In the formula, is the kth real image of the ith subset, v ik is the embedding vector of the kth real image of the ith subset, is the lth generated image of the ith subset, e il is the embedding vector of the lth generated image of the ith subset.

[0083] The three parts of the loss are weighted and summed to form the final loss function of the incremental stage:

[0084] L increment =[α+d(μr,v i )-d(μr,e i )] + +[α+d(μ f ,e i )-d(μ f ,v i )] + [α+d(v i ,v i )-d(v i ,e i )] + +[α+d(e i ,e i )-d(e i ,v i )] +

[0085] wherein L incrementFor the loss function of the incremental stage, a is a threshold value, d represents the Euclidean distance between each pair of vectors in the two vector sets, and v i i respectively represent the embedding vector set of the real image and the embedding vector set of the generated image in the i-th subset.

[0086] The model designed in the present application can continuously and effectively use the image set generated by the existing generator, constantly learn new knowledge, and gradually improve its recognition ability. In the face of an infinite possible future, incremental learning can ensure that the model perceives new changes at all times, and thus enrich its own knowledge base.

[0087] The present embodiment also provides a detection method of generated images based on incremental learning, comprising:

[0088] collecting an image to be detected;

[0089] inputting the image to be detected into the generated image detection model constructed, obtaining an embedding vector, performing nearest neighbor matching between the embedding vector and the vector stored in the knowledge base, obtaining a prototype image closest in distance, and outputting a detection result.

[0090] Further, the image to be detected is scaled and inputted into the image detection model to obtain an embedding vector, and the embedding vector and the vector stored in the knowledge base are used for nearest neighbor matching, and the class to which the prototype closest in distance belongs is the class to be detected.

[0091] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.​

Claims

1. A method for constructing a generative image detection model based on incremental learning, characterized in that, include: Construct an image dataset, which includes several subsets, including generated images and real images; Select the subset with the largest number of specified real generated image pairs from the image dataset, input it into the basic feature extractor for pre-training, obtain the loss function, and optimize the basic feature extractor based on the loss function to obtain the pre-trained feature extractor; Using the pre-trained feature extractor as the feature extractor for the incremental stage, incremental training is performed on this basis. The loss function for the incremental stage is designed, and the pre-trained feature extractor for the t-th stage is trained through the loss function for the incremental stage until the remaining subset in the image dataset is trained, thereby obtaining the generated image detection model. The method for obtaining the loss function is as follows: In the formula, , For the threshold, For loss function, Let v be the pairwise Euclidean distance between vectors in set v. Let V be the pairwise Euclidean distance between vectors in sets V and E. Let e ​​be the pairwise Euclidean distance between vectors in set e. Let be the pairwise Euclidean distance between vectors in sets e and v; Performing the incremental training includes: After the pre-training phase is completed, the remaining subsets are shuffled, the first subset is selected, and the pre-trained feature extractor of the pre-training phase is used to perform prototype calculation on the first subset. The calculated prototypes are matched with the historical prototypes stored in the knowledge base, and the historical prototype with the smallest distance is selected as the anchor point. Specifically, if the current stage is the first incremental stage, a pre-trained feature extractor is used to calculate the prototype; if the current stage is the t-th incremental stage, the feature extractor of stage t-1 and the subset of stage t are used to calculate the prototype. The loss function for the incremental phase is: in, For the loss function in the incremental phase, For the threshold, This indicates the calculation of the Euclidean distance between any two vectors in two sets of vectors. Let represent the set of embedding vectors for the real images in the i-th subset and the set of embedding vectors for the generated images in the i-th subset, respectively. As a historical prototype of real images, To generate historical prototypes of images, For real image prototypes and collections The pairwise Euclidean distance between the midvectors For real image prototypes and collections The pairwise Euclidean distance between the midvectors To generate image prototypes and sets The pairwise Euclidean distance between the midvectors To generate image prototypes and sets The pairwise Euclidean distance between the midvectors For the set of embedded vectors The pairwise Euclidean distance between the midvectors For the set of embedded vectors and embedding vector set The pairwise Euclidean distance between the midvectors. For the set of embedded vectors The pairwise Euclidean distance between the midvectors For the set of embedded vectors and embedding vector set The pairwise Euclidean distance between vectors.

2. The method for constructing a generative image detection model based on incremental learning according to claim 1, characterized in that, The pre-training process includes: Select the subset with the largest number of specified real generated image pairs from the image dataset, input the processed subset into the basic feature extractor for feature extraction, and obtain the global embedding vector representation of the generated image and the global embedding vector representation of the real image. Using the real image and the generated image as anchor points and Euclidean distance as the metric, construct a triplet loss function.

3. The method for constructing a generative image detection model based on incremental learning according to claim 2, characterized in that, The method for obtaining the global embedding vector representation of the generated image and the global vector representation of the real image is as follows: in, , Let i represent the i-th real image and j-th generated image, respectively, and E represent the basic feature extractor. , Let represent the global embedding vectors of the i-th real image and the j-th generated image after passing through the basic feature extractor, respectively. Represents a D-dimensional vector. This is the global vector representation of the real image. This is the global embedding vector representation for generating the image.

4. The method for constructing a generative image detection model based on incremental learning according to claim 1, characterized in that, Obtaining the pre-trained feature extractor includes: Based on the loss function, the basic feature extractor is optimized using the backpropagation algorithm to obtain a pre-trained feature extractor.

5. The method for constructing a generative image detection model based on incremental learning according to claim 4, characterized in that, After obtaining the pre-trained feature extractor, the following steps are also included: The prototype is computed using the pre-trained feature extractor in combination with a selected subset, and the computation result is stored in a knowledge base, specifically as follows: in, Let n1 represent the prototype of the real image and the prototype of the generated image, respectively. Let n1 be the number of real images and n2 be the number of generated images, and n1 = n2. Let represent the embedding representation of the i-th real image and the embedding representation of the i-th generated image, respectively.

6. The method for constructing a generative image detection model based on incremental learning according to claim 1, characterized in that, The loss function in the incremental stage includes the triplet loss of positive and negative samples, the triplet loss of positive sample prototypes selected from the knowledge base as anchor points, and the triplet loss of negative sample prototypes selected from the knowledge base as anchor points.

7. A generative image detection method based on incremental learning, characterized in that, include: Acquire the image to be detected; The image to be detected is input into the generated image detection model constructed according to any one of the construction methods described in claims 1-6 to obtain an embedding vector. The embedding vector is then matched with the nearest neighbor vector stored in the knowledge base to obtain the nearest prototype image, and the detection result is output.

Images