Image authenticity detection method, system and device based on pre-trained visual model

Abstract

This invention provides an image authenticity detection method, system, and device based on a pre-trained visual model. The method utilizes a pre-trained visual model to extract features from input images, which are then fed into a linear layer network for binary classification, outputting the image authenticity probability. During training, in addition to real images and AI-generated images in the training set, AI reconstructions of real images are added as stronger AI-generated images. Real images and reconstructions are randomly mixed at the image patch level. Contrastive learning on patch-level features during training enhances the model's local learning of image authenticity features. Furthermore, a low-rank adaptation (Lora) fine-tuning method is employed instead of full-scale fine-tuning to improve the model's training efficiency and effectiveness. Through these methods, this invention introduces local image authenticity information into the pre-trained visual model, effectively improving the accuracy and generalization ability of image authenticity detection.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and machine learning, specifically to an image authenticity detection method, system, and device based on a pre-trained visual model. Background Technology

[0002] With the development of generative models such as Generative Adversarial Networks (GANs) and Diffusion Models, generating high-quality fake images has become increasingly easy. These generated images are visually almost indistinguishable from real images, posing a significant challenge to image authenticity detection. Existing image authenticity detection methods fall into two main categories. One approach involves classification using linear layers of an MLP based on features from a pre-trained model. The drawback of this approach is its reliance on the native features of the pre-trained model, which is typically an image model used for instance recognition and segmentation, or a multimodal model for image-text alignment; the model itself lacks the concept of real / fake images. The other approach trains a lightweight network, such as a convolutional network, from scratch to distinguish between real and generated images. The drawback of this approach is that the network is only trained on relatively small real / generated detection datasets, limiting its generalization ability.

[0003] Pre-trained visual models refer to deep learning models pre-trained on large-scale datasets. These models possess rich visual features and are typically used in downstream image segmentation or classification tasks. In the field of generated content detection, real images and AI-generated images are somewhat separable in the visual feature space of pre-trained visual models because these models are generally trained on large-scale real image datasets, while AI-generated images belong to out-of-domain data and have a distribution inconsistent with real images. However, with advancements in image generation technology, such as the emergence of new models like MidJourney and Stable DiffusionXL, the quality of generated images has improved significantly, and directly using the visual features of pre-trained visual models has certain limitations. Therefore, many researchers have continuously proposed fine-tuning pre-trained visual models to adapt them to generated content detection tasks.

[0004] Fine-tuning of pre-trained models refers to further training a pre-trained model for a specific task and dataset to adapt it to new task requirements. In the field of image authenticity detection, to make the trained visual model more suitable for image authenticity features, some researchers have attempted to incorporate image authenticity information into the text and use contrastive learning to align the image and text information. However, such methods attempt to directly learn the authenticity information of the entire image, and the feature granularity is insufficient to identify the authenticity information of details, leaving room for improvement. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned shortcomings in the prior art and to provide an image authenticity detection method based on a pre-trained visual model.

[0006] The specific technical solution adopted in this invention is as follows:

[0007] In a first aspect, the present invention provides an image authenticity detection method based on a pre-trained visual model, comprising:

[0008] S1. Obtain a labeled training dataset containing real images and AI-generated images, and perform AI reconstruction on the real images in the training dataset. Add the obtained AI-reconstructed images to the training dataset and label them as AI-generated images. At the same time, perform image patch-level replacement and combination on the real images and the corresponding AI-reconstructed images, and add the generated combined images to the training dataset and label them as AI-generated images.

[0009] S2. Load the Transformer-based pre-trained visual model as the feature extraction part and freeze the model weights. After the feature extraction part, build a linear layer to perform a binary classification task to identify whether the image is generated by AI. Decompose each query matrix and value matrix in the pre-trained visual model into two low-rank matrices and unfreeze the parameters of the decomposed low-rank matrices as learnable parameters in the pre-trained visual model.

[0010] S3. The image authenticity detection model composed of a pre-trained visual model and a linear layer is iteratively trained using the training dataset. In each iteration, the input image first extracts image-level features and image patch-level features through the pre-trained visual model, and then the linear layer outputs a category prediction based on the image-level features. Finally, the learnable parameters in the pre-trained visual model and the linear layer are updated by calculating the weighted sum of the image-level cross-entropy loss and the image patch-level contrast loss.

[0011] S4. For the image to be detected, input it into the trained image authenticity detection model. The linear layer obtains the prediction result of whether the image to be detected belongs to the AI-generated image based on the image-level features extracted by the pre-trained visual model.

[0012] Preferably, the original real images and AI-generated images contained in the training dataset are further augmented to increase the sample size; the data augmentation process includes one or more of random cropping, rotation, color jittering, and flipping.

[0013] Preferably, the AI ​​reconstruction uses the Stable-Diffusion model.

[0014] Preferably, when generating the combined image, the real image and the corresponding AI-reconstructed image are divided into image blocks of the same size, and then some image blocks in one image are replaced with image blocks in the same position in another image to generate the combined image.

[0015] Preferably, the pre-trained visual model adopts the CLIP model or the DINOv2 model.

[0016] Preferably, the rank of the low-rank matrix is ​​2 to 8, and more preferably 4.

[0017] Preferably, the image-level cross-entropy loss takes the form of:

[0018]

[0019] The image patch-level contrast loss takes the form of:

[0020]

[0021] The total loss function obtained by weighting the two losses is in the form of:

[0022] L=λ·L cross-entropy +(1-λ)·L contrastive

[0023] Where, p i y represents the probability that the i-th sample image in the current training batch output by the linear layer belongs to the AI-generated image; i This represents the label of the i-th sample image in the current training batch. A label value of 0 indicates that the sample image is a real image, and a label value of 1 indicates that the sample image is an AI-generated image; M is the size of the current training batch; D w (i) 2 denoted as the image patch level feature distance value between the i-th pair of image patches, Y represents the source consistency flag value of the i-th pair of image patches, which is 1 if both image patches in the i-th pair of image patches are from real images or AI-generated images, and 0 otherwise; λ represents the weight hyperparameter.

[0024] Preferably, the weight hyperparameter λ is set to 0.7.

[0025] Preferably, the output dimension of the linear layer is 2, representing the probability that the input image belongs to the real image and the AI-generated image, respectively. The two-dimensional probability distribution output by the linear layer needs to be normalized to the 0-1 range by the Softmax layer.

[0026] Preferably, the iterative training process employs a batch training and dynamic learning rate adjustment strategy.

[0027] Secondly, the present invention provides an image authenticity detection system based on a pre-trained visual model, comprising:

[0028] The dataset generation module is used to obtain a labeled training dataset containing real images and AI-generated images, and to perform AI reconstruction on the real images in the training dataset. The obtained AI-reconstructed images are added to the training dataset and labeled as AI-generated images. At the same time, the real images and the corresponding AI-reconstructed images are replaced and combined at the image patch level, and the resulting combined images are also added to the training dataset and labeled as AI-generated images.

[0029] The model setup module is used to load a Transformer-based pre-trained visual model as the feature extraction part and freeze the model weights. After the feature extraction part, a linear layer is built to perform a binary classification task to identify whether an image is generated by AI. Each query matrix and value matrix in the pre-trained visual model is decomposed into two low-rank matrices, and the parameters of the decomposed low-rank matrices are unfrozen as learnable parameters in the pre-trained visual model.

[0030] The model training module is used to iteratively train the image authenticity detection model composed of a pre-trained visual model and a linear layer using the training dataset. In each iteration, the input image first extracts image-level features and image patch-level features through the pre-trained visual model, and then the linear layer outputs a category prediction based on the image-level features. Finally, the learnable parameters in the pre-trained visual model and the linear layer are updated by calculating the weighted sum of the image-level cross-entropy loss and the image patch-level contrast loss.

[0031] The detection module takes an image to be detected, inputs it into a trained image authenticity detection model, and uses a linear layer to obtain a prediction result on whether the image to be detected belongs to an AI-generated image based on the image-level features extracted by the pre-trained visual model.

[0032] Thirdly, the present invention provides a computer electronic device, which includes a memory and a processor;

[0033] The memory is used to store computer programs;

[0034] The processor is configured to, when executing the computer program, implement the image authenticity detection method based on a pre-trained visual model as described in any of the first aspects above.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] This invention comprehensively considers the accuracy and generalization ability of image authenticity detection, and constructs a pre-trained model fine-tuning strategy based on contrastive learning and data augmentation, thus planning a solution with the lowest cost for image authenticity detection. Compared with existing technologies, considering the challenges such as the continuous improvement of AI-generated image quality and the inconsistency between the distribution of real images and AI-generated images, this invention does not require additional parameters during inference. However, through data augmentation and contrastive learning, this method significantly improves the generalization and robustness of the model, achieving an in-domain accuracy of 98% and an overall average accuracy of 95%, which is 5% higher than previous methods. Attached Figure Description

[0037] Figure 1 This is a schematic diagram illustrating the steps of an image authenticity detection method based on a pre-trained visual model.

[0038] Figure 2 This is a schematic diagram of the iterative training process;

[0039] Figure 3 This is a schematic diagram of the framework for image authenticity detection. Detailed Implementation

[0040] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in various embodiments of the present invention can be combined accordingly without mutual conflict.

[0041] The "image authenticity detection" mentioned in this invention refers to the process of analyzing a given image using computer vision and machine learning techniques to determine whether it was generated or modified by artificial intelligence. The image authenticity detection framework of this invention uses a pre-trained visual model to extract features from the input image, obtaining a high-dimensional image feature vector. This extracted feature vector is then input into a linear layer network for a binary classification task, outputting the image's authenticity probability. However, this invention adds AI reconstructions of real images as a stronger AI-generated image, in addition to the real images and AI-generated images in the training set. Real images and reconstructions are randomly mixed at the image patch level. During training, comparative learning on image patch-level features strengthens the model's learning of local image authenticity features. Simultaneously, a low-rank adaptation (Lora) fine-tuning method is used instead of full-scale fine-tuning to improve the model's training efficiency and effectiveness. Through these methods, this invention introduces local image authenticity information into the pre-trained visual model, effectively improving the accuracy and generalization ability of image authenticity detection, and is suitable for real-time detection of large-scale image data. The implementation of this invention is described in detail below.

[0042] like Figure 1 As shown, this invention provides an image authenticity detection method based on a pre-trained visual model, which includes the following steps S1 to S4. The specific implementation of each step is described in detail below.

[0043] S1. Obtain a labeled training dataset containing real images and AI-generated images, and perform AI reconstruction on the real images in the training dataset. Add the obtained AI-reconstructed images to the training dataset and label them as AI-generated images. At the same time, perform image patch-level replacement and combination on the real images and the corresponding AI-reconstructed images, and add the generated combined images to the training dataset and label them as AI-generated images.

[0044] In embodiments of the present invention, the original real images and AI-generated images contained in the training dataset need to undergo data augmentation processing to expand the sample size and increase the generalization ability of the model. Data augmentation processing includes one or more of random cropping, rotation, color jittering, and flipping.

[0045] In embodiments of the present invention, the Stable-Diffusion model can be used to perform AI reconstruction of real images as described above.

[0046] In an embodiment of the present invention, the specific method for generating the combined image is as follows: the real image and the corresponding AI reconstructed image are divided into image blocks of the same size, and then some image blocks in one image are replaced with image blocks in the same position in another image to generate a combined image.

[0047] It should be noted that the specific replacement ratio can be adjusted according to actual needs when performing image patch replacement. In the embodiments of the present invention, 50% of the image patches in each real image need to be replaced by image patches in the AI-reconstructed image. It is particularly important to note that when performing image patch replacement, the image patch to be replaced and the replacement image patch need to be located in the same position in the image. The AI-reconstructed image used to replace the real image is output based on the real image using a Stable-Diffusion model. This replacement combination process can be achieved by randomly generating image masks and then combining them according to the masks.

[0048] S2. Load a Transformer-based pre-trained visual model as the feature extraction part and freeze the model weights. After the feature extraction part, construct a linear layer to perform a binary classification task to identify whether an image is generated by AI. Decompose each query matrix and value matrix in the pre-trained visual model into two low-rank matrices and unfreeze the parameters of the decomposed low-rank matrices as learnable parameters in the pre-trained visual model.

[0049] In embodiments of the present invention, the pre-trained visual model can be implemented using a CLIP model or a DINOv2 model. CLIP and DINOv2 models are existing technologies and will not be described in detail here.

[0050] In this embodiment, all learnable parameters in the pre-trained visual model, except for the query matrix and value matrix, are frozen and do not participate in subsequent iterations of training optimization. Only the query matrix and value matrix participate in the optimization process. However, the query matrix and value matrix are not directly learned and optimized; instead, they need to be decomposed into two low-rank matrices to participate in optimization. Taking the query matrix Q as an example, it can be decomposed into matrix A and matrix B, i.e., Q = A * B, where the ranks of matrices A and B are lower than the rank of Q. In this embodiment, the ranks of the low-rank matrices A and B can be set to 2 to 8, preferably 4. Assuming the dimension of the query matrix Q is 1024 * 1024, it is preferable to decompose it into a 1024 * 4 matrix A and a 4 * 1024 matrix B. Thus, by unfreezing matrices A and B and setting them as learnable parameters, they can replace the original query matrix Q and participate in the learning optimization process in subsequent training. Similarly, each value matrix V can also be decomposed into a 1024*4 learnable matrix A and a 4*1024 learnable matrix B, which can then participate in the learning optimization during the subsequent training process.

[0051] In addition to the query matrix and the low-rank matrix obtained from value matrix decomposition in the pre-trained visual model serving as learnable parameters, learnable parameters also exist in the linear layer constructed after feature extraction. Therefore, in the iterative training process of this invention, it is necessary to optimize the query matrix and the low-rank matrix obtained from value matrix decomposition in the pre-trained visual model, as well as the learnable parameters in the linear layer.

[0052] In embodiments of the present invention, a linear layer is used to determine the authenticity of the input image; this task is a binary classification task. Therefore, the output dimension of the linear layer is 2, representing the probabilities that the input image belongs to a real image and an AI-generated image, respectively. Specifically, the information at the first position in the linear layer output represents the probability that the image is real, and the information at the second position represents the probability that the image is AI-generated. Furthermore, the two-dimensional probability distribution output by the linear layer needs to be normalized to the 0-1 interval using a Softmax layer.

[0053] S3. Using the training dataset obtained in S1, iteratively train the image authenticity detection model consisting of a pre-trained visual model and a linear layer. In each iteration, the input image first extracts image-level features and image patch-level features through the pre-trained visual model, then the linear layer outputs a category prediction based on the image-level features, and finally updates the learnable parameters in the pre-trained visual model and the linear layer by calculating the weighted sum of the image-level cross-entropy loss and the image patch-level contrast loss.

[0054] In an embodiment of the present invention, the above-mentioned image-level cross-entropy loss takes the form of:

[0055]

[0056] In an embodiment of the present invention, the above-mentioned image patch-level contrast loss needs to be calculated by sampling all image patches in the current training batch in pairs. Each pair of image patches can come from the same image or from different images. The contrast loss takes the form of:

[0057]

[0058] In an embodiment of the present invention, the total loss function obtained by weighting the above two losses is in the form of:

[0059] L=λ·L cross-entropy +(1-λ)·L contrastive

[0060] Where, p i y represents the probability that the i-th sample image in the current training batch output by the linear layer belongs to the AI-generated image; iThis represents the label of the i-th sample image in the current training batch. A label value of 0 indicates that the sample image is a real image, and a label value of 1 indicates that the sample image is an AI-generated image; M is the size of the current training batch; D w (i) 2 represents the image patch-level feature distance value between the i-th pair of image patches, and Y represents the source consistency flag value of the i-th pair of image patches. If both image patches in the i-th pair of image patches originate from real images or AI-generated images, the value is 1; otherwise, it is 0. In this embodiment of the invention, the image patch-level feature distance value can be calculated using cosine distance. N is the number of image patch pairs sampled from the current training batch. In this embodiment, N is set to the maximum number of pairwise pairings of image patches from all images in the current training batch, meaning that all image patch pairings participate in the calculation of the contrastive loss. λ represents the weight hyperparameter. This weight hyperparameter λ can be optimized and adjusted according to actual conditions. In this embodiment of the invention, λ = 0.7.

[0061] The total loss function described above is a linear combination of cross-entropy loss and contrastive loss. During the iterative optimization process, the optimizer can update the learnable parameters in the pre-trained visual model and the linear layers based on the backpropagation gradient of the total loss function obtained in each iteration.

[0062] In embodiments of the present invention, batch training and dynamic learning rate adjustment strategies can be employed during iterative training. A maximum number of iterations G can be set. After each iteration, it is determined whether the maximum number of iterations G has been reached. If the current iteration count exceeds the maximum number of iterations G, the iteration stops, the model parameters are saved, and model training is complete; otherwise, the next iteration continues. The iteration process can be found in [reference needed]. Figure 2 As shown.

[0063] S4. For the image to be detected, input it into a pre-trained visual model and a linear layer. The pre-trained visual model extracts image-level features and image patch-level features from the image to be detected. The image-level features are then input into a binary classification linear layer to obtain a prediction result of whether the image to be detected belongs to an AI-generated image. This detection process is as follows: Figure 3 As shown.

[0064] This invention presents a general image authenticity detection method. The pre-trained visual model used in this method can theoretically be any visual model pre-trained on a large scale. Through patch-level real / fake image confusion and comparative learning of patch-level features, the visual features of the pre-trained visual model can be transferred to image authenticity detection tasks, significantly improving the generalization and accuracy of the pre-trained visual model in image authenticity detection. Furthermore, the practical value of this method increases with the improvement of the pre-trained visual model. This invention can be effectively applied to image authenticity detection tasks and allows generated images that did not appear during training to appear during testing, thus possessing broad application prospects.

[0065] It should be noted that the method steps S1 to S4 described above can essentially be implemented in the form of a computer program.

[0066] Therefore, based on the same inventive concept, the present invention also provides an image authenticity detection system based on a pre-trained visual model, corresponding to the image authenticity detection method based on a pre-trained visual model provided in the above embodiments. This system includes:

[0067] The dataset generation module is used to obtain a labeled training dataset containing real images and AI-generated images, and to perform AI reconstruction on the real images in the training dataset. The obtained AI-reconstructed images are added to the training dataset and labeled as AI-generated images. At the same time, the real images and the corresponding AI-reconstructed images are replaced and combined at the image patch level, and the resulting combined images are also added to the training dataset and labeled as AI-generated images.

[0068] The model setup module is used to load a Transformer-based pre-trained visual model as the feature extraction part and freeze the model weights. After the feature extraction part, a linear layer is built to perform a binary classification task to identify whether an image is generated by AI. Each query matrix and value matrix in the pre-trained visual model is decomposed into two low-rank matrices, and the parameters of the decomposed low-rank matrices are unfrozen as learnable parameters in the pre-trained visual model.

[0069] The model training module is used to iteratively train the image authenticity detection model composed of a pre-trained visual model and a linear layer using the training dataset. In each iteration, the input image first extracts image-level features and image patch-level features through the pre-trained visual model, and then the linear layer outputs a category prediction based on the image-level features. Finally, the learnable parameters in the pre-trained visual model and the linear layer are updated by calculating the weighted sum of the image-level cross-entropy loss and the image patch-level contrast loss.

[0070] The detection module takes an image to be detected, inputs it into a trained image authenticity detection model, and uses a linear layer to obtain a prediction result on whether the image to be detected belongs to an AI-generated image based on the image-level features extracted by the pre-trained visual model.

[0071] In the image authenticity detection system based on the pre-trained visual model in the above embodiment, the specific process executed by each module can also be referred to the specific steps of S1 to S4 mentioned above, and will not be repeated here.

[0072] Similarly, based on the same inventive concept, the present invention also provides a computer electronic device corresponding to the image authenticity detection method based on a pre-trained visual model provided in the above embodiments, which includes a memory and a processor;

[0073] The memory is used to store computer programs;

[0074] The processor is configured to implement the image authenticity detection method based on the pre-trained visual model as described above when executing the computer program.

[0075] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0076] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0077] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0078] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.

[0079] The image authenticity detection method based on the pre-trained visual model described in S1 to S4 above will be tested on a specific dataset to demonstrate its technical effectiveness.

[0080] Example

[0081] This embodiment employs two pre-trained visual models: CLIP and DINOv2. To demonstrate the generality of this embodiment, reconstruction is performed using both Stable-Diffusion v2 (sdv2) and Stable-Diffusion v1 (sdv1). Therefore, this embodiment designs four experimental groups: DINOv2+sdv2, CLIP+sdv2, DINOv2+sdv1, and CLIP+sdv1. These groups are tested on the GenImage dataset (Zhu M, Chen H, Yan Q, et al. Genimage: A million-scale benchmark for detecting ai-generated image[J]. Advances in Neural Information Processing Systems, 2024, 36.), and trained on the Stable-Diffusion 1.4 dataset according to the testing methods described in the paper. Accuracy on other methods such as Midjourney and Big GAN is also tested. The results of the four experimental groups are compared in Table 1.

[0082] Table 1. Algorithm Comparison Experimental Data

[0083] Generate method name sd14 sd15 mj adm wukong glide vqdm gan avg DINOv2+sdv2 97.94 97.94 95.97 92.06 98.02 95.07 95.47 86.70 94.89 CLIP+sdv2 98.36 98.03 97.97 94.05 98.19 94.12 97.93 92.51 96.39 DINOv2+sdv1 98.97 98.71 89.27 91.97 98.80 94.17 97.82 92.73 95.30 CLIP+sdv1 98.77 98.51 91.19 90.79 98.72 89.03 98.75 95.30 95.13

[0084] The above results demonstrate that this invention, by adding augmented data that obfuscates real and generated images at the image patch granularity and performing contrastive learning at the image patch granularity, can enhance the model's recognition of locally augmented image features. This invention can transfer the visual features of a pre-trained visual model to image realism feature tasks, significantly improving the generalization and robustness of the pre-trained visual model across various generation methods.

[0085] The embodiments described above are merely some preferred implementations of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. An image authenticity detection method based on a pre-trained visual model, characterized in that, include: S1. Obtain a labeled training dataset containing real images and AI-generated images, and perform AI reconstruction on the real images in the training dataset. Add the obtained AI-reconstructed images to the training dataset and label them as AI-generated images. At the same time, perform image patch-level replacement and combination on the real images and the corresponding AI-reconstructed images, and add the generated combined images to the training dataset and label them as AI-generated images. S2. Load the Transformer-based pre-trained visual model as the feature extraction part and freeze the model weights. After the feature extraction part, build a linear layer to perform a binary classification task to identify whether the image is an AI-generated image. Decompose each query matrix and value matrix in the pre-trained visual model into two low-rank matrices and unfreeze the parameters of the decomposed low-rank matrices as learnable parameters in the pre-trained visual model. S3. The image authenticity detection model composed of a pre-trained visual model and a linear layer is iteratively trained using the training dataset. In each iteration, the input image first extracts image-level features and image patch-level features through the pre-trained visual model, and then the linear layer outputs a category prediction based on the image-level features. Finally, the learnable parameters in the pre-trained visual model and the linear layer are updated by calculating the weighted sum of the image-level cross-entropy loss and the image patch-level contrast loss. S4. For the image to be detected, input it into the trained image authenticity detection model. The linear layer obtains the prediction result of whether the image to be detected belongs to the AI-generated image based on the image-level features extracted by the pre-trained visual model.

2. The image authenticity detection method based on a pre-trained visual model as described in claim 1, characterized in that, The training dataset originally contained real images and AI-generated images, which were then augmented to increase the sample size; the data augmentation process included one or more of random cropping, rotation, color jittering, and flipping.

3. The image authenticity detection method based on a pre-trained visual model as described in claim 1, characterized in that, The AI ​​reconstruction uses the Stable-Diffusion model.

4. The image authenticity detection method based on a pre-trained visual model as described in claim 1, characterized in that, When generating the combined image, the real image and the corresponding AI-reconstructed image are divided into image blocks of the same size. Then, some image blocks in one image are replaced with image blocks in the same position in another image to generate the combined image.

5. The image authenticity detection method based on a pre-trained visual model as described in claim 1, characterized in that, The pre-trained visual model uses either the CLIP model or the DINOv2 model.

6. The image authenticity detection method based on a pre-trained visual model as described in claim 1, characterized in that, The rank of the low-rank matrix ranges from 2 to 8.

7. The image authenticity detection method based on a pre-trained visual model as described in claim 1, characterized in that, The image-level cross-entropy loss takes the form of: ； The image patch-level contrast loss takes the form of: ； The total loss function obtained by weighting the two losses is in the form of: ； in, This represents the probability that the i-th sample image in the current training batch output by the linear layer belongs to an AI-generated image; This represents the label of the i-th sample image in the current training batch. A label value of 0 indicates that the sample image is a real image, and a label value of 1 indicates that the sample image is an AI-generated image; M is the size of the current training batch. Indicates the first For the image patch-level feature distance values ​​between image patches, Indicates the first For the source consistency flag value of the image patch, if the first... If both image patches in an image patch are derived from real images or both image patches are derived from AI-generated images, the value is 1; otherwise, it is 0. This represents the weight hyperparameter.

8. The image authenticity detection method based on a pre-trained visual model as described in claim 1, characterized in that, The iterative training process employs a batch training and dynamic learning rate adjustment strategy.

9. An image authenticity detection system based on a pre-trained visual model, characterized in that, include: The dataset generation module is used to obtain a labeled training dataset containing real images and AI-generated images, and to perform AI reconstruction on the real images in the training dataset. The obtained AI-reconstructed images are added to the training dataset and labeled as AI-generated images. At the same time, the real images and the corresponding AI-reconstructed images are replaced and combined at the image patch level, and the resulting combined images are also added to the training dataset and labeled as AI-generated images. The model setup module is used to load a Transformer-based pre-trained visual model as the feature extraction part and freeze the model weights. After the feature extraction part, a linear layer is built to perform a binary classification task to identify whether an image is generated by AI. Each query matrix and value matrix in the pre-trained visual model is decomposed into two low-rank matrices, and the parameters of the decomposed low-rank matrices are unfrozen as learnable parameters in the pre-trained visual model. The model training module is used to iteratively train the image authenticity detection model composed of a pre-trained visual model and a linear layer using the training dataset. In each iteration, the input image first extracts image-level features and image patch-level features through the pre-trained visual model, and then the linear layer outputs a category prediction based on the image-level features. Finally, the learnable parameters in the pre-trained visual model and the linear layer are updated by calculating the weighted sum of the image-level cross-entropy loss and the image patch-level contrast loss. The detection module takes an image to be detected, inputs it into a trained image authenticity detection model, and uses a linear layer to obtain a prediction result on whether the image to be detected belongs to an AI-generated image based on the image-level features extracted by the pre-trained visual model.

10. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the image authenticity detection method based on a pre-trained visual model as described in any one of claims 1 to 8.

Images