A method, device, equipment, medium and product for generating image detection

Abstract

The present application discloses a method, apparatus, device, medium and product for generating image detection, which relates to the field of image detection. The method comprises: inputting an original image into each proxy model to obtain the output result of each proxy model; back-propagating the weighted average of the output results of each proxy model on the corresponding proxy model to obtain the gradient map of each proxy model on the original image; inputting each gradient map and the original image into a corresponding shallow feature extraction network to obtain a feature map; inputting the feature map into a cross-attention network to obtain the token value of each gradient map; aggregating each similar image type feature and the image type feature of the original image to obtain a new image type feature; inputting the new image type feature into a weight calculation network to obtain a gradient weight; calculating a weighted average value based on the token value of each gradient map and the gradient weight; and inputting the weighted average value into a classification head to determine whether the original image is a generated image. The present application can improve the accuracy of recognition results.

Description

Technical Field

[0001] The present application relates to the field of image detection, and in particular to a method, device, equipment, medium and product for generating image detection. Background Art

[0002] With the rapid development of deep learning, especially Generative Adversarial Networks (GANs) and Diffusion Models, the image generation process has become more efficient. These advanced algorithms not only generate high-resolution and highly realistic images, but also optimize the generation process, allowing users to easily and freely generate images that meet their needs.

[0003] Generative image detection is a key technology in the field of computer vision, designed to identify and determine whether an image was generated by a deep learning model. Conventional generative image detection techniques rely primarily on feature extraction and pattern recognition methods to detect images. Traditional methods require manual extraction of residual artifacts in images, a time-consuming and subject-sensitive process. In contrast, deep learning models can automatically learn and extract discriminative features, thereby improving detection accuracy and efficiency. However, due to the rapid development of deep learning model technology, artifacts appearing in the original image space are becoming increasingly rare, and detection models based on this method are increasingly failing, resulting in low recognition accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, equipment, medium and product for generating image detection, which can improve the accuracy of recognition results.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for generating an image detection, comprising:

[0007] Inputting the original image into each proxy model to obtain the output result of each proxy model; the output result is a multidimensional vector;

[0008] The weighted average of the output results of each proxy model is back-propagated on the corresponding proxy model to obtain the gradient map of each proxy model on the original image;

[0009] Inputting each gradient map and the original image into a corresponding shallow feature extraction network to obtain a feature map of each gradient map and a feature map of the original image;

[0010] For any gradient map, the feature map of the gradient map and the feature map of the original image are input into the cross attention network to obtain the token value of the gradient map;

[0011] Inputting the original image into a type feature extractor to obtain image type features of the original image;

[0012] Aggregating each similar image type feature and the image type feature of the original image to obtain a new image type feature; the similar image type feature is obtained based on the image type feature of the original image using a nearest neighbor algorithm;

[0013] Inputting the new image type feature into the weight calculation network to obtain the gradient weight of the token value of each gradient map;

[0014] Calculate the weighted average of the token values ​​of all gradient images according to the token value of each gradient image and the gradient weight of the token value of each gradient image;

[0015] The weighted average of the token values ​​of all gradient images is input into the classification head to determine whether the original image is a generated image.

[0016] Optionally, the shallow feature extraction network includes:

[0017] Four feature extraction modules connected in sequence;

[0018] The first feature extraction module includes two sequentially connected convolution modules, wherein the convolution module includes a first convolution layer, a first batch processing layer and a first ReLU activation function connected in sequence;

[0019] The second feature extraction module consists of three sequentially connected inlet stream convolutions;

[0020] The third feature extraction module consists of four sequentially connected mid-stream convolutions;

[0021] The fourth feature extraction module consists of four intermediate stream convolutions and one outlet stream convolution connected in sequence;

[0022] The inlet stream convolution and the outlet stream convolution both include a first depth-separable convolution, a second depth-separable convolution, a maximum pooling layer, a second convolution layer and a first splicing layer; the input end of the first depth-separable convolution is connected to the input end of the second convolution layer, the output end of the first depth-separable convolution is connected to the input end of the second depth-separable convolution, the output end of the second depth-separable convolution is connected to the input end of the maximum pooling layer, and the output end of the maximum pooling layer is connected to the output end of the second convolution layer through the first splicing layer;

[0023] The intermediate stream convolution includes a third depth-separable convolution, a fourth depth-separable convolution, a fifth depth-separable convolution and a second splicing layer; the input end of the third depth-separable convolution and the output end of the fifth depth-separable convolution are connected through the second splicing layer, the output end of the third depth-separable convolution is connected to the input end of the fourth depth-separable convolution, and the output end of the fourth depth-separable convolution is connected to the input end of the fifth depth-separable convolution.

[0024] Optionally, the cross-attention network includes: three attention modules; the first input end of the first attention module is used to input the feature map of the gradient map, and the second input end of the first attention module is used to input the feature map of the original map; the first input end of the second attention module is connected to the output end of the first attention module, and the second input end of the second attention module is used to input the feature map of the original map; the first input end of the third attention module is connected to the output end of the second attention module, and the second input end of the third attention module is used to input the feature map of the original map.

[0025] Optionally, the attention module includes three parallel linear mapping layers, each linear mapping layer is used to input the feature map of the gradient map and the feature map of the original image.

[0026] Optionally, the weight calculation network includes:

[0027] Two multilayer perceptrons connected in sequence; the first multilayer perceptron includes a first fully connected layer, a second Relu activation function, and a second batch processing layer connected in sequence; the second multilayer perceptron includes a second fully connected layer and a Sigmoid activation function connected in sequence.

[0028] Optionally, each similar image type feature and the image type feature of the original image are aggregated to obtain a new image type feature, specifically:

[0029] According to the formula Aggregate the similar image type features and the image type features of the original image to obtain new image type features, where: Z' cls represents the new image type feature, k represents the total number of similar image type features, a i represents the combined weight of the i-th similar image type feature, represents the i-th similar image type feature, Z cls Represents the image type characteristics of the original image, Z cls T Indicates Z cls Find the transpose, || || means find the norm.

[0030] In a second aspect, the present application provides a device for generating an image detection, comprising:

[0031] A gradient extraction module is configured to input the original image into each proxy model to obtain an output result of each proxy model; the output result is a multidimensional vector; and a weighted average of the output results of each proxy model is back-propagated on the corresponding proxy model to obtain a gradient map of each proxy model on the original image;

[0032] A detection module is configured to input each gradient map and the original image into a corresponding shallow feature extraction network to obtain a feature map of each gradient map and a feature map of the original image; for any gradient map, the feature map of the gradient map and the feature map of the original image are input into a cross attention network to obtain a token value of the gradient map;

[0033] An integrated module is configured to input the original image into a type feature extractor to obtain the image type features of the original image; aggregate the similar image type features and the image type features of the original image to obtain new image type features; the similar image type features are obtained based on the image type features of the original image using a nearest neighbor algorithm; input the new image type features into a weight calculation network to obtain the gradient weights of the token values ​​of each gradient map; calculate the weighted average of the token values ​​of all gradient maps based on the token values ​​of each gradient map and the gradient weights of the token values ​​of each gradient map; and input the weighted average of the token values ​​of all gradient maps into a classification head to determine whether the original image is a generated image.

[0034] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any one of the above-described methods for generating image detection.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any one of the above-described methods for generating image detection.

[0036] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements any one of the above-described methods for generating image detection.

[0037] According to the specific embodiments provided in this application, this application has the following technical effects:

[0038] The present application provides a generated image detection method, device, equipment, medium and product. With the continuous advancement of counterfeiting technology, the difference between real images and generated images in the image domain is getting smaller and smaller, making it difficult to distinguish them directly. Converting the image to the gradient domain first can make the originally inconspicuous differences more obvious, thereby identifying them. The present application inputs the original image into each proxy model to obtain the output results of each proxy model; the weighted average of the output results of each proxy model is back-propagated on the corresponding proxy model to obtain the gradient map of each proxy model on the original image, and the image is converted to the gradient domain for processing, so that the generated image can expand the difference with the real image in the new space, thereby better distinguishing it and improving the accuracy of the recognition result. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A flowchart of a method for generating image detection provided in one embodiment of the present application;

[0041] Figure 2 A schematic diagram of a method for generating image detection provided in one embodiment of the present application;

[0042] Figure 3 A flowchart for obtaining a gradient map provided in one embodiment of the present application;

[0043] Figure 4 A structural block diagram of the first feature extraction module provided in one embodiment of the present application;

[0044] Figure 5 A structural block diagram of a second feature extraction module provided in an embodiment of the present application;

[0045] Figure 6 A structural block diagram of a third feature extraction module provided in an embodiment of the present application;

[0046] Figure 7 A structural block diagram of a fourth feature extraction module provided in an embodiment of the present application;

[0047] Figure 8 A block diagram of the inlet flow convolution structure provided in one embodiment of the present application;

[0048] Figure 9 A block diagram of the intermediate stream convolution structure provided in one embodiment of the present application;

[0049] Figure 10 This is a block diagram of the export flow convolution structure provided by an embodiment of the present application;

[0050] Figure 11 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0052] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0053] In an exemplary embodiment, Figure 1 As shown, a method for generating image detection is provided, comprising the following steps, wherein:

[0054] Step 201: Input the original image into each proxy model to obtain the output of each proxy model; the output is a multidimensional vector. Each proxy model is pre-trained on a specific dataset. Specific proxy models include models trained for classification tasks on a large natural image dataset (ImageNet dataset), models pre-trained for low-level generation tasks such as super-resolution and denoising on high-definition image datasets, detection models (object detection models and semantic segmentation models) trained on various image datasets of different styles and categories, and many more models adopted to improve the diversity and generalization of the entire detection framework.

[0055] Step 202: Back-propagate the weighted average of the output results of each proxy model on the corresponding proxy model to obtain the gradient map of each proxy model on the original image.

[0056] Step 203: Input each gradient map and the original image into the corresponding shallow feature extraction network to obtain the feature map of each gradient map and the feature map of the original image. There are N gradient maps and one original image in total, so there are N+1 shallow feature extraction networks.

[0057] Step 204: For any gradient map, the feature map of the gradient map and the feature map of the original image are input into the cross attention network to obtain the token value of the gradient map.

[0058] Step 205: Input the original image into a type feature extractor to obtain image type features of the original image.

[0059] Step 206: Aggregate the similar image type features and the image type features of the original image to obtain new image type features; the similar image type features are obtained based on the image type features of the original image using a nearest neighbor algorithm.

[0060] Step 207: Input the new image type feature into the weight calculation network to obtain the gradient weight of the token value of each gradient map.

[0061] Step 208: Calculate the weighted average of the token values ​​of all gradient images according to the token value of each gradient image and the gradient weight of the token value of each gradient image.

[0062] Step 209: The weighted average of the token values ​​of all gradient images is input into the classification head to determine whether the original image is a generated image.

[0063] Implementing the above steps 201 to 209 can improve the accuracy of the recognition results. In the related art, models trained for specific data sets often have good performance, but once used on a new data set, the performance often drops significantly and overfitting occurs. They can only recognize generated images that are similar to those that appeared in the training data set, but cannot recognize images generated by new generation methods. This application uses a variety of proxy models based on different pre-training tasks to extract the gradient image of the image, so that the generated images generated by different types and different generation methods can reflect the corresponding differences in specific proxy tasks. The final result is obtained by integrating the models to improve the model's detection ability for different types of images and its generalization ability on new data sets.

[0064] In another exemplary embodiment of the present application, the weighted average of the output results of each proxy model is back-propagated on the corresponding proxy model to obtain the gradient map of each proxy model on the original image, specifically:

[0065] For the output result of the i-th proxy model, according to the formula Calculate the weighted average of the output results of the i-th surrogate model Among them, L i Indicates the length of the output result after converting the output result of the i-th proxy model into a one-dimensional vector, w ij Indicates the weight value of the jth number in the output result after converting the output result of the i-th proxy model into a one-dimensional vector (a learnable weight), z ij Indicates the jth number in the output result after converting the output result of the i-th proxy model into a one-dimensional vector.

[0066] The weighted average of the output results of the i-th proxy model is back-propagated on the i-th proxy model to obtain the gradient map of the i-th proxy model on the original image.

[0067] In another exemplary embodiment of the present application,

[0068] The shallow feature extraction network includes four feature extraction modules connected in sequence.

[0069] like Figure 4 As shown, the first feature extraction module includes two sequentially connected convolution modules, and the convolution module includes a first convolution layer, a first batch processing layer and a first ReLU activation function connected in sequence.

[0070] like Figure 5 As shown, the second feature extraction module consists of three inlet stream convolutions connected sequentially.

[0071] like Figure 6 As shown, the third feature extraction module consists of four sequentially connected mid-stream convolutions.

[0072] like Figure 7 As shown, the fourth feature extraction module includes four intermediate flow convolutions and one outlet flow convolution connected in sequence.

[0073] like Figure 8 and Figure 10 As shown, the inlet flow convolution and the outlet flow convolution both include a first depth-separable convolution, a second depth-separable convolution, a maximum pooling layer, a second convolution layer and a first splicing layer; the input end of the first depth-separable convolution is connected to the input end of the second convolution layer, the output end of the first depth-separable convolution is connected to the input end of the second depth-separable convolution, the output end of the second depth-separable convolution is connected to the input end of the maximum pooling layer, and the output end of the maximum pooling layer is connected to the output end of the second convolution layer through the first splicing layer.

[0074] like Figure 9 As shown, the intermediate stream convolution includes a third depth-separable convolution, a fourth depth-separable convolution, a fifth depth-separable convolution and a second splicing layer; the input end of the third depth-separable convolution and the output end of the fifth depth-separable convolution are connected through the second splicing layer, the output end of the third depth-separable convolution is connected to the input end of the fourth depth-separable convolution, and the output end of the fourth depth-separable convolution is connected to the input end of the fifth depth-separable convolution.

[0075] In another exemplary embodiment of the present application, the cross attention network includes: three attention modules; the first input end of the first attention module is used to input the feature map of the gradient map, and the second input end of the first attention module is used to input the feature map of the original image; the first input end of the second attention module is connected to the output end of the first attention module, and the second input end of the second attention module is used to input the feature map of the original image; the first input end of the third attention module is connected to the output end of the second attention module, and the second input end of the third attention module is used to input the feature map of the original image. The first attention module uses the feature map of the original image as Q and the feature map of the gradient map as K and V. The subsequent two attention modules use the feature map of the original image as Q and the output value of the previous network as K and V.

[0076] In another exemplary embodiment of the present application, the attention module includes three parallel linear mapping layers, each of which is used to input the feature map of the gradient map and the feature map linear mapping layer of the original image to map the features to a specified space.

[0077] In another exemplary embodiment of the present application, the weight calculation network includes:

[0078] Two multilayer perceptrons connected in sequence; the first multilayer perceptron includes a first fully connected layer (output dimension is 2N), a second Relu activation function and a second batch processing layer connected in sequence; the second multilayer perceptron includes a second fully connected layer (output dimension is N) and a Sigmoid activation function connected in sequence. The Sigmoid activation function is used to map features to weights of token values ​​of N gradient maps between 0 and 1.

[0079] In another exemplary embodiment of the present application, each similar image type feature and the image type feature of the original image are aggregated to obtain a new image type feature, specifically:

[0080] According to the formula Aggregate the similar image type features and the image type features of the original image to obtain new image type features, where: Z' cls represents the new image type feature, k represents the total number of similar image type features, a i represents the combined weight of the i-th similar image type feature, represents the i-th similar image type feature, Z cls Represents the image type characteristics of the original image, Z cls T Indicates Z cls Find the transpose, || || means find the norm.

[0081] In another exemplary embodiment of the present application, the above five depth-wise separable convolutions each include a depth-wise convolution layer, a point-by-point convolution layer and a ReLU activation function connected in sequence.

[0082] In another exemplary embodiment of the present application, the type feature extractor is an image encoder in a contrastive language-image pre-training (CLIP) model that performs self-supervised pre-training on a large number of text-image pairs, which is used to extract image type features.

[0083] In another exemplary embodiment of the present application, specifically, in training a type feature extractor, the type feature extractor is trained to extract type features of an image. The image type here specifically refers to a stylized type that is unrelated to the semantic content of the image, such as a real photo, a game model, or a fine art drawing. This method uses the image feature encoder in the CLIP model as the type feature extractor. By fine-tuning the pre-trained model using supervised contrastive learning, the features of images of the same or similar types are brought closer together, while the features of images of different types are further apart.

[0084] In another exemplary embodiment of the present application, in the process of alternating training of detectors (the detector includes a shallow feature extraction network, a cross-attention network, a weight calculation network and a classification head) and feature weights (the weights applied in step 202), the weights of the type feature extractor will be fixed, and the remaining models (detectors) will be trained alternately with the feature weight values. First, the feature weights will be randomized, and then the remaining models will be supervised trained using a binary classification loss function. After a certain number of training rounds, all models are frozen. For each proxy model, a random noise will be added to the feature weight, and the corresponding gradient weight will be set to 1, and the remaining gradient weights will be set to 0. Compare the performance difference before and after adding noise. If it is better, keep the noise, otherwise remove it. Repeat this operation several times for the feature weights of each proxy model and continue to train other models. Repeat the alternating training process until the preset training rounds are reached.

[0085] The binary classification loss function used is specifically the cross entropy loss function:

[0086]

[0087] Among them, y i Indicates whether the true i-th sample image is a generated image, Indicates whether the predicted i-th sample image is a generated image, and BS represents the total number of sample images.

[0088] In another exemplary embodiment of the present application, similar image type features are obtained by processing a type feature memory bank using a nearest neighbor algorithm based on the image type features of the original image. The type feature memory bank stores features of different types of real images extracted by the type feature extractor in a fixed-size first-in, first-out queue. During the aforementioned alternating training process, when a fixed-weight type feature extractor is used to extract image type features, the memory bank is simultaneously updated to extract type features from the real image. These features are then compared with the features stored in the memory bank. If no similarity exists, the features are added to the memory bank to obtain a type feature memory bank.

[0089] This application has the following technical effects:

[0090] 1. Effectively improve the performance of model detection of generated images through a new detection framework.

[0091] 2. Perform computations on the proxy model and obtain the corresponding gradient representation through backpropagation, thereby converting the image from the source domain to the gradient domain. This increases the difference between the real image and the generated image, improving the accuracy of the detection results.

[0092] 3. This application uses a variety of proxy models for different pre-training tasks. By extracting image type features and comparing them with memory library features, it integrates multiple different results, thereby enhancing the accuracy of the model and its generalization performance on new datasets.

[0093] 4. Use the CLIP model to extract image type features, and use the nearest neighbor method to fuse similar features stored in the memory bank to replace the original image features. This is used to calculate the gradient weights, allowing the weight calculation network to adapt to the image type, thereby improving the accuracy and generalization of the model.

[0094] Based on the same inventive concept, embodiments of the present application also provide a generated image detection device for implementing the generated image detection method described above. The implementation solution provided by this device is similar to the implementation solution described in the above method. Therefore, the specific limitations of one or more embodiments of the generated image detection device provided below can be found in the above-mentioned limitations of the generated image detection method and will not be repeated here.

[0095] In an exemplary embodiment, a device for generating an image detection is provided, comprising:

[0096] The gradient extraction module is used to input the original image into each proxy model to obtain the output results of each proxy model; the output results are multidimensional vectors; the weighted average of the output results of each proxy model is back-propagated on the corresponding proxy model to obtain the gradient map of each proxy model on the original image.

[0097] The detection module is used to input each gradient map and the original image into the corresponding shallow feature extraction network to obtain the feature map of each gradient map and the feature map of the original image; for any gradient map, the feature map of the gradient map and the feature map of the original image are input into the cross attention network to obtain the token value of the gradient map.

[0098] An integrated module is configured to input the original image into a type feature extractor to obtain the image type features of the original image; aggregate the similar image type features and the image type features of the original image to obtain new image type features; the similar image type features are obtained based on the image type features of the original image using a nearest neighbor algorithm; input the new image type features into a weight calculation network to obtain the gradient weights of the token values ​​of each gradient map; calculate the weighted average of the token values ​​of all gradient maps based on the token values ​​of each gradient map and the gradient weights of the token values ​​of each gradient map; and input the weighted average of the token values ​​of all gradient maps into a classification head to determine whether the original image is a generated image.

[0099] In an exemplary embodiment, Figure 2 and Figure 3 As shown, the image data is defined as I∈R H×W×C , where H is the height of the image, W is the width of the image, and C is the dimension of the image, which defaults to 3. In the first step, the image is first input into the gradient extraction module to obtain the corresponding gradient map. The proxy model includes a variety of models that perform different pre-training tasks on different datasets. Let F i , i∈[1,N]. For each specific agent model F i , input the image into it to get z i =F i (I), z i is the output of the corresponding task. Different proxy tasks will result in different proxy model outputs. i The dimensions of will also be different, so for each proxy model, an additional set of learnable feature weights is required That is, the weight applied in step 202 above, where L i z i The length after straightening (converting to a one-dimensional vector). z i After w i After weighted average, we get use In the proxy model F i Back propagation can be performed on the original image I to obtain its gradient map I g-i . z i is a length of L iThe length of the vector varies depending on the agent task, but only one value can be used to find the gradient, so it is necessary to perform a weighted average of the values ​​of different dimensions of the vector.

[0100] Original image I and N generated gradient maps I g-i The input detection module is used to obtain the corresponding features for subsequent integration module detection. The shallow feature extraction network takes the image as input and obtains the corresponding feature map: Z i ∈R h×w×c ,i∈[0,N], where h,w,c represent the height, width and dimension of the feature map respectively, and Z0 represents the feature map corresponding to the original image. Each proxy model F i Extracted gradient map I g-i The corresponding feature map Z i Combine the feature map Z0 of the original image to form a feature map pair Z0-Z i Input into the subsequent cross attention network. The cross attention network first straightens the feature map into a c-dimensional vector Z of length h×w+1 according to the spatial dimension. i '∈R (h×w+1)×c , where token is added to the feature as a self-learning value (in the attention module, the input is the h×w blocks of the image, and the output is h×w blocks and a token value of the same size. Generally, only the token value is used as the output result. In order to ensure the consistency of input and output dimensions, a zero-initialized token and h×w vectors are added to the input of the first attention module and input into the model together). Then the original image feature Z0' is used as the query, and the gradient map feature Z i ' as Key and Value. The specific calculation method is as follows, using the matrix W Q , W K , W V Convert the features to Q, K, V, and then perform attention calculation:

[0101] Q=W Q T Z0'

[0102] K=W K T Z i '

[0103] V=W v T Z i '

[0104] Attn(Q,K,V)=Softmax(Q T K)V T

[0105] Attn(Q,K,V) is the output of the cross attention network, Softmax() represents the Softmax function, Q T Indicates the transpose of Q, V T Indicates the transposition of V. Since there are three attention modules in the cross attention network, in order to pass the information of the previous attention module to the next module, Z in the next two attention modules i 'Replace with the attention value Attn(Q,K,V) in the previous module.

[0106] Since there are N feature map pairs Z0-Z i , so in the end we will get N token values, which need to be input into the integration module to get the final result. In the integration module, the type feature extractor will extract the type feature Z of the original image cls Then Z cls Compare it with the rest of the features in the memory bank to get the closest Z cls k features of These k features will be combined into a new type feature Z' according to the following formula cls :

[0107]

[0108]

[0109] Z' cls The input weight calculation network obtains N corresponding gradient weights, each weight is between 0 and 1. The above N token values ​​will also be weighted averaged by these weights: token i Represents the token value of the gradient map of the i-th proxy model on the original image, and finally the token avg The data is sent to the classification head to obtain the detection results of the model.

[0110] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 11As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store generated image detection data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for generating image detection is implemented.

[0111] Those skilled in the art will understand that Figure 11 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application and does not constitute a limitation on the computer device to which the solution of the present application is applied. A specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned method embodiments when executing the computer program.

[0112] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above-mentioned method embodiments when executed by a processor.

[0113] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the above method embodiments are implemented.

[0114] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0115] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0116] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0117] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0118] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for generating image detection, characterized in that: The method for generating image detection comprises: Inputting the original image into each proxy model to obtain the output result of each proxy model; the output result is a multidimensional vector; The weighted average of the output results of each proxy model is back-propagated on the corresponding proxy model to obtain the gradient map of each proxy model on the original image; Inputting each gradient map and the original image into a corresponding shallow feature extraction network to obtain a feature map of each gradient map and a feature map of the original image; For any gradient map, the feature map of the gradient map and the feature map of the original image are input into the cross attention network to obtain the token value of the gradient map; Inputting the original image into a type feature extractor to obtain image type features of the original image; Aggregating each similar image type feature and the image type feature of the original image to obtain a new image type feature; the similar image type feature is obtained based on the image type feature of the original image using a nearest neighbor algorithm; Inputting the new image type feature into the weight calculation network to obtain the gradient weight of the token value of each gradient map; Calculate the weighted average of the token values ​​of all gradient images according to the token value of each gradient image and the gradient weight of the token value of each gradient image; The weighted average of the token values ​​of all gradient images is input into the classification head to determine whether the original image is a generated image.

2. The generated image detection method according to claim 1, characterized in that The shallow feature extraction network includes: Four feature extraction modules connected in sequence; The first feature extraction module includes two sequentially connected convolution modules, wherein the convolution module includes a first convolution layer, a first batch processing layer and a first ReLU activation function connected in sequence; The second feature extraction module consists of three sequentially connected inlet stream convolutions; The third feature extraction module consists of four sequentially connected mid-stream convolutions; The fourth feature extraction module consists of four intermediate stream convolutions and one outlet stream convolution connected in sequence; The inlet stream convolution and the outlet stream convolution both include a first depth-separable convolution, a second depth-separable convolution, a maximum pooling layer, a second convolution layer and a first splicing layer; the input end of the first depth-separable convolution is connected to the input end of the second convolution layer, the output end of the first depth-separable convolution is connected to the input end of the second depth-separable convolution, the output end of the second depth-separable convolution is connected to the input end of the maximum pooling layer, and the output end of the maximum pooling layer is connected to the output end of the second convolution layer through the first splicing layer; The intermediate stream convolution includes a third depth-separable convolution, a fourth depth-separable convolution, a fifth depth-separable convolution and a second splicing layer; the input end of the third depth-separable convolution and the output end of the fifth depth-separable convolution are connected through the second splicing layer, the output end of the third depth-separable convolution is connected to the input end of the fourth depth-separable convolution, and the output end of the fourth depth-separable convolution is connected to the input end of the fifth depth-separable convolution.

3. The generated image detection method according to claim 1, characterized in that: The cross attention network includes: three attention modules; the first input end of the first attention module is used to input the feature map of the gradient map, and the second input end of the first attention module is used to input the feature map of the original map; the first input end of the second attention module is connected to the output end of the first attention module, and the second input end of the second attention module is used to input the feature map of the original map; the first input end of the third attention module is connected to the output end of the second attention module, and the second input end of the third attention module is used to input the feature map of the original map.

4. The generated image detection method according to claim 3, characterized in that: The attention module includes three parallel linear mapping layers, each of which is used to input the feature map of the gradient map and the feature map of the original image.

5. The generated image detection method according to claim 1, characterized in that: The weight calculation network includes: Two multilayer perceptrons connected in sequence; the first multilayer perceptron includes a first fully connected layer, a second Relu activation function, and a second batch processing layer connected in sequence; the second multilayer perceptron includes a second fully connected layer and a Sigmoid activation function connected in sequence.

6. The generated image detection method according to claim 1, characterized in that: Aggregate the similar image type features and the image type features of the original image to obtain new image type features, specifically: According to the formula Aggregate the similar image type features and the image type features of the original image to obtain new image type features, where: Z cls represents the new image type feature, k represents the total number of similar image type features, a i represents the combined weight of the i-th similar image type feature, represents the i-th similar image type feature, Z cls Represents the image type characteristics of the original image, Z cls T Indicates Z cls Find the transpose, |||| means find the norm.

7. A device for generating an image detection, characterized in that: The image detection device comprises: A gradient extraction module is configured to input the original image into each proxy model to obtain an output result of each proxy model; the output result is a multidimensional vector; and a weighted average of the output results of each proxy model is back-propagated on the corresponding proxy model to obtain a gradient map of each proxy model on the original image; A detection module is configured to input each gradient map and the original image into a corresponding shallow feature extraction network to obtain a feature map of each gradient map and a feature map of the original image; for any gradient map, the feature map of the gradient map and the feature map of the original image are input into a cross attention network to obtain a token value of the gradient map; An integrated module is configured to input the original image into a type feature extractor to obtain the image type features of the original image; aggregate the similar image type features and the image type features of the original image to obtain new image type features; the similar image type features are obtained based on the image type features of the original image using a nearest neighbor algorithm; input the new image type features into a weight calculation network to obtain the gradient weights of the token values ​​of each gradient map; calculate the weighted average of the token values ​​of all gradient maps based on the token values ​​of each gradient map and the gradient weights of the token values ​​of each gradient map; and input the weighted average of the token values ​​of all gradient maps into a classification head to determine whether the original image is a generated image.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image generation detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the generated image detection method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the generated image detection method according to any one of claims 1 to 6 is implemented.

Images