Construction method and detection method for generating image detection model based on incremental learning

Abstract

The invention discloses a construction method and detection method of a generated image detection model based on incremental learning, and the model construction method comprises the steps: constructing an image data set, the data set comprises a plurality of subsets, and the subsets comprise generated images and real images; selecting a subset with the maximum number of specified real generated image pairs in the image data set, inputting the subset into a basic feature extractor for pre-training processing to obtain a loss function, and optimizing the basic feature extractor based on the loss function to obtain a pre-trained feature extractor; and taking the pre-training feature extractor as an increment stage feature extractor, performing increment training on the basis, designing a loss function of an increment stage, and training the pre-training feature extractor of the tth stage through the loss function of the increment stage until the remaining subsets in the image data set are trained, thereby obtaining a generated image detection model. The method not only improves the detection accuracy of the generated image, but also has the continuous learning capability.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology in deep learning, and in particular to a method for constructing a generative image detection model based on incremental learning and a detection method. Background Art

[0002] With the rapid development of artificial intelligence (AI), generative models have made significant progress in image generation. A person without synthetic image skills can simply use a good generative image model and input some key text information to quickly generate a realistic image that matches the text description. This unprecedented ease of use has largely led to the proliferation of generated images. Currently, AI-generated images have reached a level of realism that is difficult for the human eye to distinguish. However, this advanced technology carries numerous potential risks. For example, the AI-generated image of the Pentagon on fire that was widely circulated on Twitter deceived several major news media organizations and caused a severe drop in the US stock market.

[0003] With the continuous improvement of infrastructure, the development of artificial intelligence is booming. Generative models are emerging in large numbers, and the distribution of generated images is also diverse. Many researchers are committed to developing universal generated image detectors, attempting to detect generated images produced by all generators, especially those from unknown generators. Numerous studies have shown that there are significant differences in low-level pixel features between images generated by generators and natural images. Models can easily learn a decision boundary that distinguishes them. However, this decision boundary varies for different generator architectures. Therefore, designing a universal generated image detector across generators is a difficult and challenging task.

[0004] However, general-purpose detectors also have some shortcomings. For example, they cannot perceive future changes in real time and are full of uncertainty when detecting novel generators. Therefore, designing a detector with incremental learning capabilities is also of great significance. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a method for constructing a generated image detection model and a detection method based on incremental learning to improve the accuracy of generated image detection.

[0006] On the one hand, to achieve the above-mentioned purpose, the present invention provides a method for constructing a generative image detection model based on incremental learning, comprising:

[0007] Constructing an image dataset, wherein the dataset includes several subsets, and the subsets include generated images and real images;

[0008] Selecting a subset with the largest number of specified real generated image pairs from the image dataset, inputting the subset into a basic feature extractor for pre-training processing to obtain a loss function, and optimizing the basic feature extractor based on the loss function to obtain a pre-trained feature extractor;

[0009] The pre-trained feature extractor is used as the feature extractor in the incremental stage, and incremental training is performed on this basis. The loss function of the incremental stage is designed, and the pre-trained feature extractor of the tth stage is trained by the loss function of the incremental stage until the remaining subsets in the image dataset are trained to obtain a generated image detection model.

[0010] Preferably, the pre-training process includes:

[0011] A subset with the largest number of specified real-generated image pairs in the image dataset is selected, and the processed subset is input into the basic feature extractor for feature extraction to obtain a global embedding vector representation of the generated image and a global embedding vector representation of the real image. The real image and the generated image are used as anchor points respectively, and the Euclidean distance is used as the measurement criterion to construct a triplet loss function.

[0012] Preferably, the method for obtaining the global embedding vector representation of the generated image and the global vector representation of the real image is:

[0013]

[0014]

[0015] in, denote the i-th real image and the j-th generated image, E denotes the basic feature extractor, and v i , e j Respectively represent the global embedding vector representation of the i-th real image and the j-th generated image after passing through the basic feature extractor, represents a D-dimensional vector, is the global vector representation of the real image, Generates a global embedding vector representation of the image.

[0016] Preferably, the method for obtaining the loss function is:

[0017] L pretrain (v,e)=[α-d(v,v)+d(v,e)] + +[α-d(e,e)+d(e,v)] +

[0018] Where [x] + ≡max(x,0), α is the threshold, L pretrain(v,e) is the loss function, d(v,v) is the pairwise Euclidean distance between vectors in set v, d(v,e) is the pairwise Euclidean distance between vectors in set v and set e, d(e,e) is the pairwise Euclidean distance between vectors in set e, and d(e,v) is the pairwise Euclidean distance between vectors in set e and set v.

[0019] Preferably, obtaining the pre-trained feature extractor comprises:

[0020] Based on the loss function, the basic feature extractor is optimized by a back propagation algorithm to obtain a pre-trained feature extractor.

[0021] Preferably, after obtaining the pre-trained feature extractor, the method further includes:

[0022] The pre-trained feature extractor is used in combination with the selected subset to calculate the prototype, and the calculation results are stored in the knowledge base, specifically:

[0023]

[0024] Among them, μ r ,μ f They represent the prototype of the real image and the prototype of the generated image respectively, n1 is the number of real images, n2 is the number of generated images, and n1=n2, denote the embedding representation of the i-th real image and the embedding representation of the i-th generated image, respectively.

[0025] Preferably, performing the incremental training includes:

[0026] After the pre-training phase, the remaining subsets are shuffled, the first subset is selected, and the prototype is calculated for the first subset using the pre-trained feature extractor in the pre-training phase. The calculated prototype is matched with the historical prototypes stored in the knowledge base for the closest distance, and the historical prototype with the smallest distance is selected as the anchor point.

[0027] Among them, if the current stage is the first incremental stage, the pre-trained feature extractor is used to calculate the prototype; if the current stage is the t-th incremental stage, the feature extractor of the t-1 stage and the subset of the t-th stage are used to calculate the prototype.

[0028] Preferably, the loss function of the incremental stage includes triplet loss of positive samples and negative samples, triplet loss of positive sample prototypes selected from the knowledge base as anchor points, and triplet loss of negative sample prototypes selected from the knowledge base as anchor points.

[0029] Preferably, the loss function of the incremental stage is:

[0030] L increment=[α+d(μr,v i )-d(μr,e i )] + +[α+d(μ f , e i )-d(μ f , v i )] + [α+d(v i , v i )-d(v i , e i )] + +[α+d(e i , e i )-d(e i , v i )] +

[0031] Among them, L increment is the loss function of the incremental stage, α is the threshold, d represents the calculation of the Euclidean distance between two vectors in the vector set, v i , e i They represent the embedding vector set of the real image in the i-th subset and the embedding vector set of the generated image in the i-th subset, μ r is the historical prototype of the real image, μ f is the historical prototype of the generated image, d(μ r , v i ) is the real image prototype and set v i The Euclidean distance between two vectors, d(μ r , e i ) is the real image prototype and set e i The Euclidean distance between two vectors, d(μ f , e i ) is the generated image prototype and set e i The Euclidean distance between two vectors, d(μ f , v i ) is the generated image prototype and set v i The Euclidean distance between two vectors, d(v i , v i ) is the embedding vector set v i The Euclidean distance between two vectors, d(v i , e i ) is the embedding vector set v i and the embedding vector set e i The Euclidean distance between two vectors, d(e i ,e i ) is the embedding vector set e iThe Euclidean distance between two vectors, d(e i ,v i ) is the embedding vector set e i and the embedding vector set v i The Euclidean distance between two vectors.

[0032] On the other hand, to achieve the above-mentioned object, the present invention further provides a detection method for generated images based on incremental learning, comprising:

[0033] Collecting images to be predicted;

[0034] The image to be predicted is input into the constructed generative image detection model to obtain an embedding vector, the embedding vector is matched with the vector stored in the knowledge base by nearest neighbor matching to obtain the prototype image with the closest distance, and the detection result is output.

[0035] Compared with the prior art, the present invention has the following advantages and technical effects:

[0036] (1) Compared with the general cross-generator image detection method, it cannot perceive future changes in real time and is full of uncertainty when detecting novel generators. However, the detection method of the present invention has incremental learning capabilities and can continuously and effectively learn new knowledge and gradually improve its own recognition ability;

[0037] (2) The general cross-generator generated image detection method needs to collect a large number of images for training at one time, which requires a lot of computing resources, while the generated image detector based on incremental learning only needs a small amount of data to complete the training, which requires less computing resources;

[0038] (3) To address the catastrophic forgetting problem in incremental learning, the present invention redesigns the triplet loss function to ensure that in the process of continuous incremental learning, the distance between similar samples is shortened and the distance between dissimilar samples is widened.

[0039] (4) The present invention can improve the accuracy of generated image detection, reduce the error level of model prediction, and the model has the ability to continue learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0041] Figure 1 This is a flow chart of a detection method for generating an image based on incremental learning according to an embodiment of the present invention;

[0042] Figure 2This is a flow chart of a prediction method when using a model as a feature extractor according to an embodiment of the present invention;

[0043] Figure 3 These are example diagrams of generated images and real images used in the embodiments of the present invention. DETAILED DESCRIPTION

[0044] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0045] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] This paper proposes a method for constructing a generative image detection model based on incremental learning, such as Figure 1-Figure 2 ,include:

[0047] Constructing an image dataset, where the dataset includes several subsets, each of which includes generated images and real images;

[0048] Selecting a subset with the largest number of specified real generated image pairs from the image dataset, inputting the subset into a basic feature extractor for pre-training, obtaining a loss function, and optimizing the basic feature extractor based on the loss function to obtain a pre-trained feature extractor;

[0049] The pre-trained feature extractor is used as the feature extractor in the incremental stage, and incremental training is performed on this basis. The loss function of the incremental stage is designed, and the pre-trained feature extractor of the tth stage is trained by the loss function of the incremental stage until the remaining subsets in the image dataset are trained to obtain a generative image detection model.

[0050] The present invention can improve the accuracy of generated image detection, reduce the error level of model prediction, and the model has the ability to continue learning.

[0051] Furthermore, constructing an image dataset includes:

[0052] Get a dataset, which includes several subsets, each of which includes generated images and real images, where the generated images and real images are generated by different generators. Figure 3 .

[0053] Furthermore, the pre-training process includes:

[0054] A subset with the largest number of specified real-generated image pairs in the image dataset is selected, and the processed subset is input into the basic feature extractor for feature extraction to obtain a global embedding vector representation of the generated image and a global embedding vector representation of the real image. The real image and the generated image are used as anchor points respectively, and the Euclidean distance is used as the measurement criterion to construct a triplet loss function.

[0055] In this embodiment, the input images in the subset are scaled to 224×224 pixels, and the images are normalized according to the image pixel mean and variance. The normalization method is:

[0056]

[0057] Where x is the image pixel value, μ is the image pixel mean, σ is the image pixel variance, and f(x) is the normalized image.

[0058] Furthermore, the processed subset is input into the basic feature extractor for feature extraction to obtain the global embedding vector representation of the generated image and the global vector representation of the real image. The specific method is as follows:

[0059]

[0060]

[0061] in, denote the i-th real image and the j-th generated image, E denotes the basic feature extractor, and v i , e j Respectively represent the global embedding vector representation of the i-th real image and the j-th generated image after passing through the basic feature extractor, represents a D-dimensional vector, is the global vector representation of the real image, To generate a global embedding vector representation of the image. In this embodiment, the basic feature extractor is VIT (Vision Transformer).

[0062] Calculate the Euclidean distance between adjacent vectors in the subset. The formula is as follows:

[0063] d(X, Y) = ||XY||2

[0064] Where d(X, Y) is the Euclidean distance between adjacent vectors, and X and Y are different vectors in the subset.

[0065] Furthermore, the method to obtain the loss function is:

[0066] L pretrain (v,e)=[α-d(v,v)+d(v,e)]+ +[α - d(e, e) + d(e, v)] +

[0067] Where [x] + ≡max(x, 0), α is the threshold, L pretrain (v, e) is the loss function, d(v, v) is the pairwise Euclidean distance between vectors in set v, d(v, e) is the pairwise Euclidean distance between vectors in set v and set e, d(e, e) is the pairwise Euclidean distance between vectors in set e, and d(e, v) is the pairwise Euclidean distance between vectors in set e and set v.

[0068] Using this loss function, the model is optimized through the back-propagation algorithm to obtain a pre-trained feature extractor.

[0069] Furthermore, after obtaining the pre-trained feature extractor, the following steps are also included:

[0070] The pre-trained feature extractor is used in combination with the selected subset to calculate the prototype, and the calculation results are stored in the knowledge base, specifically:

[0071]

[0072] Among them, μ r , μ f They represent the prototype of the real image and the prototype of the generated image respectively, n1 is the number of real images, n2 is the number of generated images, and n1=n2, denote the embedding representation of the i-th real image and the embedding representation of the i-th generated image, respectively.

[0073] Furthermore, performing incremental training includes:

[0074] After the pre-training phase, the remaining subsets are shuffled, the first subset is selected, and the prototype is calculated for the first subset using the pre-trained feature extractor in the pre-training phase. The calculated prototype is matched with the historical prototypes stored in the knowledge base for the closest distance, and the historical prototype with the smallest distance is selected as the anchor point.

[0075] Among them, if the current stage is the first incremental stage, the pre-trained feature extractor is used to calculate the prototype; if the current stage is the t-th incremental stage, the feature extractor of the t-1 stage and the subset of the t-th stage are used to calculate the prototype.

[0076] Specifically, in order to show that the training order of the subsets does not affect the model results, the training order of the subsets is disrupted before training. The subsets after the disruption are expressed as represents the kth real picture of the i-th subset, Denotes the lth generated image of the i-th subset. Gradually select the remaining subsets for incremental training. The input image is scaled to 224×224 pixels and normalized.

[0077] Furthermore, using the real image history prototypes and the generated image history prototypes selected from the knowledge base, a loss function for the incremental stage is designed, and the pre-trained feature extractor of the tth stage is trained using the loss function of the incremental stage until all subsets are trained to obtain an image detection model;

[0078] The loss function in the incremental phase includes the triple loss of positive and negative samples, the triple loss of the positive sample prototype selected from the knowledge base as the anchor point, and the triple loss of the negative sample prototype selected from the knowledge base as the anchor point. Specifically,

[0079]

[0080]

[0081] d(X,Y)=||XY||2

[0082] Where, is the kth real image in the i-th subset, v ik is the embedding vector of the kth real image in the i-th subset, Generate the lth image for the i-th subset, e il Generate an image embedding vector for the lth image in the i-th subset.

[0083] The weighted sum of these three losses forms the loss function of the final incremental stage:

[0084] L increment =[α+d(μr,v i )-d(μr,e i )] + +[α+d(μ f , e i )-d(μ f , v i )] + [α+d(v i , v i )-d(v i , e i )] + +[α+d(e i , e i )-d(e i , v i )] +

[0085] Among them, L incrementis the loss function of the incremental stage, α is the threshold, d represents the calculation of the Euclidean distance between two vectors in the vector set, v i , e i They represent the embedding vector set of the real image and the embedding vector set of the generated image in the i-th subset respectively.

[0086] The model designed by this invention can continuously and effectively utilize the image sets generated by the existing generator, continuously learning new knowledge and gradually improving its recognition ability. In the face of an infinitely possible future, incremental learning can ensure that the model is always aware of new changes and thus enriches its knowledge base.

[0087] This embodiment also provides a detection method for generated images based on incremental learning, including:

[0088] Collecting images to be detected;

[0089] The image to be detected is input into the constructed generative image detection model to obtain the embedding vector, and the embedding vector is matched with the vector stored in the knowledge base by nearest neighbors to obtain the prototype image with the closest distance, and the detection result is output.

[0090] Furthermore, the image to be detected is scaled and input into the image detection model to obtain an embedding vector, and the embedding vector is used to perform nearest neighbor matching with the vectors stored in the knowledge base. The category to which the nearest prototype belongs is the category to be detected.

[0091] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for constructing a generative image detection model based on incremental learning, characterized in that: include: Constructing an image dataset, wherein the dataset includes several subsets, and the subsets include generated images and real images; Selecting a subset with the largest number of specified real generated image pairs in the image data set, inputting the subset into a basic feature extractor for pre-training processing, obtaining a loss function, and optimizing the basic feature extractor based on the loss function to obtain a pre-trained feature extractor; The pre-trained feature extractor is used as the feature extractor in the incremental stage, and incremental training is performed on this basis. The loss function of the incremental stage is designed, and the pre-trained feature extractor of the tth stage is trained by the loss function of the incremental stage until the remaining subsets in the image data set are trained to obtain a generated image detection model.

2. The method for constructing a generative image detection model based on incremental learning according to claim 1, characterized in that: The pre-training process includes: A subset with the largest number of specified real-generated image pairs in the image data set is selected, and the processed subset is input into the basic feature extractor for feature extraction to obtain a global embedding vector representation of the generated image and a global embedding vector representation of the real image. The real image and the generated image are used as anchor points respectively, and the Euclidean distance is used as a measurement criterion to construct a triplet loss function.

3. The method for constructing a generative image detection model based on incremental learning according to claim 2, characterized in that: The method of obtaining the global embedding vector representation of the generated image and the global vector representation of the real image is: in, denote the i-th real image and the j-th generated image respectively, E denotes the basic feature extractor, and v i , e j They represent the global embedding vector representation of the i-th real image and the j-th generated image after passing through the basic feature extractor, represents a D-dimensional vector, is the global vector representation of the real image, Generate a global embedding vector representation of the image.

4. The method for constructing a generative image detection model based on incremental learning according to claim 3, characterized in that: The method for obtaining the loss function is: L pretrain (v,e)=[α-d(v,v)+d(v,e)] + +[α-d(e,e)+d(e,v)] + Where, [x] + ≡max(x,0), α is the threshold, L pretrain (v,e) is the loss function, d(v,v) is the pairwise Euclidean distance between vectors in set v, d(v,e) is the pairwise Euclidean distance between vectors in set v and set e, d(e,e) is the pairwise Euclidean distance between vectors in set e, and d(e,v) is the pairwise Euclidean distance between vectors in set e and set v.

5. The method for constructing a generative image detection model based on incremental learning according to claim 4, characterized in that: Obtaining the pre-trained feature extractor, comprising: Based on the loss function, the basic feature extractor is optimized by a back propagation algorithm to obtain a pre-trained feature extractor.

6. The method for constructing a generative image detection model based on incremental learning according to claim 5, characterized in that: After obtaining the pre-trained feature extractor, the method further includes: The pre-trained feature extractor is used in combination with the selected subset to calculate the prototype, and the calculation result is stored in the knowledge base, specifically: Among them, μ r ,μ f They represent the prototype of the real image and the prototype of the generated image respectively, n1 is the number of real images, n2 is the number of generated images, and n1=n2, denote the embedding representation of the i-th real image and the embedding representation of the i-th generated image respectively.

7. The method for constructing a generative image detection model based on incremental learning according to claim 1, characterized in that: The incremental training comprises: The remaining subsets after the pre-training phase are shuffled, the first subset is selected, the pre-trained feature extractor of the pre-training phase is used to perform prototype calculation on the first subset, the calculated prototype is used to perform the closest distance matching with the historical prototype stored in the knowledge base, and the historical prototype with the smallest distance is selected as the anchor point; Among them, if the current stage is the first incremental stage, the pre-trained feature extractor is selected to calculate the prototype; if the current stage is the t-th incremental stage, the feature extractor of the t-1 stage and the subset of the t stage are used to calculate the prototype.

8. The method for constructing a generative image detection model based on incremental learning according to claim 7, characterized in that: The loss function of the incremental stage includes triplet loss of positive samples and negative samples, triplet loss of positive sample prototypes selected from the knowledge base as anchor points, and triplet loss of negative sample prototypes selected from the knowledge base as anchor points.

9. The method for constructing a generative image detection model based on incremental learning according to claim 1, characterized in that: The loss function of the incremental stage is: L increment =[α+d(µ r ,v i )-d(µ r ,and i )] + +[α+d(µ f ,and i )-d(µ f ,v i )] + [α+d(v i ,v i )-d(v i ,and i )] + +[α+d(e i ,and i )-gave i ,v i )] + Among them, L increment is the loss function of the incremental stage, α is the threshold, d represents the calculation of the Euclidean distance between two vectors in the two vector sets, and v i , e i They represent the embedding vector set of the real image in the ith subset and the embedding vector set of the generated image in the ith subset, μ r is the historical prototype of the real image, μ f is the historical prototype of the generated image, d(μ r ,v i ) is the real image prototype and the set v i The Euclidean distance between two vectors, d(μ r ,e i ) is the real image prototype and the set e i The Euclidean distance between two vectors, d(u f ,e i ) is the generated image prototype and set e i The Euclidean distance between two vectors, d(μ f ,v i ) is the generated image prototype and set v i The Euclidean distance between two vectors, d(v i ,v i ) is the embedding vector set v i The Euclidean distance between two vectors, d(v i ,e i ) is the embedding vector set v i and the embedding vector set e i The Euclidean distance between two vectors, d(e i ,e i ) is the embedding vector set e i The Euclidean distance between two vectors, d(e i ,v i ) is the embedding vector set e i and the embedding vector set v i The Euclidean distance between two vectors in the 10. A generated image detection method based on incremental learning, characterized in that: include: Collecting images to be detected; The image to be detected is input into the generated image detection model constructed according to any one of the construction methods described in claims 1-9 to obtain an embedding vector, the embedding vector is matched with the vector stored in the knowledge base by nearest neighbors to obtain the prototype image with the closest distance, and the detection result is output.

Images