AI-generated image blind box detection method and device based on semantic consistency differences

Abstract

The present invention discloses an AI-generated image blind box detection method and device based on semantic consistency differences. The method includes: generating corresponding descriptive text for the image to be detected based on a pre-trained multimodal large model BLIP with frozen parameters; using the multimodal model CLIP to embed the image-text pair consisting of the image to be detected and its corresponding descriptive text into a joint representation space, and extracting image-text features. The detection model is constructed through two-stage ruminant training: in the first stage, the image and text features are spliced ​​and input into the first multi-layer perceptron MLP1, and the AI ​​generation confidence is output; in the second stage, the confidence is used as a control coefficient, and the cosine similarity of the image and text features is adaptively adjusted in combination with the loss function proposed in this scheme, and iterative optimization is performed in the latent space to generate the controlled joint features and input them into the second multi-layer perceptron MLP2 for final judgment. This method does not require the foreknowledge of the image source model, has strong versatility and detection accuracy, and is suitable for AI-generated image recognition in complex scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of image blind box detection, and in particular to a method and device for detecting blind boxes in AI-generated images based on semantic consistency differences. Background Art

[0002] With the development of generative AI technologies like generative adversarial networks (GANs) and diffusion models, AI-generated images have been widely used in scenarios such as smart cities, smart healthcare, and smart retail. By modeling the distribution of raw data, AI-generated models can automatically generate image content with similar characteristics to the original samples. For example, in smart shopping scenarios, they can generate simulated outfit effects to assist user decision-making.

[0003] However, with the continuous improvement in the performance of AI image generation models and the proliferation of open-source tools, the realism of the generated images has approached that of real images, making it difficult for the human eye to accurately distinguish between real and fake images. Furthermore, uncontrolled misuse of these models could potentially be used to generate malicious deepfakes, leading to security risks such as misleading public opinion and identity theft. Therefore, it is imperative to develop a universal, accurate, and robust AI image detection method to address the challenges posed by multi-source generated content. Summary of the Invention

[0004] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0005] Therefore, to solve the above technical problems, the present invention provides the following technical solutions: a method and device for detecting blind boxes of AI-generated images based on semantic consistency differences, which is used to reliably determine whether an image is AI-generated; wherein, the detection method includes the following steps:

[0006] Use the pre-trained multimodal large model BLIP to generate corresponding descriptive text for the image to be detected. Each image has a descriptive text corresponding to it;

[0007] Use the pre-trained multimodal model CLIP to embed features of images and their descriptive text, obtaining image features and text features based on the CLIP image-text alignment space. The cosine similarity between the image and text features represents the degree of semantic similarity between the image and the text (i.e., its descriptive text);

[0008] Based on the two-stage rumination training strategy proposed in this solution, the image and text features extracted by the above pre-trained large model are processed to train the detector (i.e., the first multi-layer perceptron MLP1 and the second multi-layer perceptron MLP2); wherein:

[0009] Phase 1: The image and text features are concatenated to form a joint representation, which is input into the first multilayer perceptron MLP1 to obtain the confidence that the image is generated by AI. The loss function is then calculated based on the output of the first multilayer perceptron MLP1 and the true label of the sample to train the first multilayer perceptron MLP1.

[0010] The first multilayer perceptron MLP1 adopts a fully connected neural network structure, which includes:

[0011] 1. First linear layer: input dimension 1536, output dimension 768;

[0012] 2. ReLU activation function layer;

[0013] 3. Second linear layer: input dimension 768, output dimension 1;

[0014] 4. Sigmoid activation function layer;

[0015] The functions of the first multilayer perceptron MLP1 include:

[0016] In the first training phase, the primary classifier outputs a confidence probability value in the interval (0, 1) to indicate the likelihood that the input image is generated by AI.

[0017] In the subsequent training phase 2, when the parameters are frozen, the output probability value is used as the control coefficient of the adaptive loss function to dynamically determine the feature adjustment range; specifically:

[0018] When the output probability of the first multi-layer perceptron MLP1 is highly consistent with the true label (i.e., the classification confidence is high), the adjustment amplitude of the image and text features in the second stage is reduced accordingly;

[0019] When the output probability of the first multilayer perceptron MLP1 deviates significantly from the true label (i.e., the classification confidence is low), the second stage will implement differential adjustment based on the absolute value of the deviation. The larger the deviation, the stronger the feature vector adjustment intensity.

[0020] In the subsequent reasoning process, the first multilayer perceptron MLP1 does not participate in the forward calculation.

[0021] The second stage: The confidence level output from the first stage is used as the adjustment coefficient, and the loss function proposed by this method is used to adjust the image and text features, so that the similarity of the image and text features of the positive samples increases and the similarity of the image and text features of the negative samples decreases. The adjusted image and text features are restored to the original space and spliced ​​again to form a new joint representation. The new detection output is input to the second multi-layer perceptron MLP2, and the loss function is calculated based on the output and the true label of the sample to train the second multi-layer perceptron MLP2.

[0022] Determine whether the input image is AI-generated based on the output of the second multi-layer perceptron MLP2 in the second stage;

[0023] The second multilayer perceptron MLP2 has the same hierarchical structure as the first multilayer perceptron MLP1:

[0024] 1. First linear layer: input dimension 1536, output dimension 768;

[0025] 2. ReLU activation function layer;

[0026] 3. Second linear layer: input dimension 768, output dimension 1;

[0027] 4. Sigmoid activation function layer;

[0028] The function of the second multilayer perceptron MLP2 is:

[0029] In the second training phase, as the final classifier, the normalized probability value is output through the Sigmoid function;

[0030] In the subsequent reasoning process, its output value is used as the basis for judgment, specifically:

[0031] Probability value > 0.5: The image is determined to be AI-generated;

[0032] Probability value < 0.5: judged to be a real image;

[0033] Probability value = 0.5: indicates uncertainty.

[0034] As a preferred solution of the AI-generated image blind box detection method based on semantic consistency differences described in the present invention, the image to be detected is generated by multiple different generation models and only contains image modal information but not any text modal information.

[0035] As a preferred solution of the AI-generated image blind box detection method based on semantic consistency difference described in the present invention, wherein: a pre-trained multimodal model BLIP is used to generate descriptive text for the image to be detected, and a pre-trained multimodal model CLIP is used to jointly encode the image and the corresponding descriptive text; the method includes:

[0036] Freeze the parameters of the multimodal large model BLIP, input the images to be tested into the model one by one, and obtain the descriptive text corresponding to each image;

[0037] Freeze the parameters of the multimodal model CLIP, form an image-text pair with each image to be detected and its corresponding descriptive text, and input it into the image-text alignment space of the multimodal model CLIP for feature embedding to obtain the embedded features used to calculate the similarity between the image and the text.

[0038] As a preferred solution of the AI-generated image blind box detection method based on semantic consistency differences described in the present invention, the image features and text features are spliced ​​on a dimension of the feature vector with a length of 1 in the CLIP image-text alignment space to form a joint representation; after splicing, the length of the dimension remains 1, and the length of the other spliced ​​dimension is twice the total length of the dimension before splicing.

[0039] As a preferred solution of the AI-generated image blind box detection method based on semantic consistency difference of the present invention, the two-stage rumination training includes:

[0040] Using the joint representation to train the first multilayer perceptron MLP1, the output of the first multilayer perceptron MLP1 and the true label of the sample are used to calculate the loss function and train the first multilayer perceptron MLP1;

[0041] The output of the first multilayer perceptron MLP1 and the joint representation are input into the second multilayer perceptron MLP2 to obtain the detection output of the two-stage training; the loss function proposed in this method is calculated based on the output and the true label of the sample, and the second multilayer perceptron MLP2 is trained by iteratively updating the image and text features in the latent space;

[0042] The iteration of the image-text features in the latent space is used to adjust according to the sample label: when the sample label is positive, the cosine similarity of the image-text features of the positive sample is increased; when the sample label is negative, the cosine similarity of the image-text features of the negative sample is reduced;

[0043] The output of the second multi-layer perceptron MLP2 is used as the basis for the final determination of whether the image to be detected is generated by AI.

[0044] In AI-generated images, due to the fundamental limitations of the generator, it is often difficult to fully express semantic information in the generated images. Therefore, the cosine similarity between real images and their descriptive text is always higher than the cosine similarity between AI-generated images and their descriptive text. This is an invariant feature shared across different generative models. Therefore, by adaptively amplifying and reducing the cosine similarity of image and text features through ruminant training, the ease of extraction of this invariant feature can be amplified, thereby enabling the second multi-layer perceptron to detect accurate and reliable detection results.

[0045] The present invention also provides an AI-generated image blind box detection device based on semantic consistency differences, which is applied to the above-mentioned AI-generated image blind box detection method based on semantic consistency differences. The device includes:

[0046] The generation module is used to generate matching descriptive text for each image to be detected using the pre-trained multimodal model BLIP;

[0047] The acquisition module is used to input the image and its corresponding descriptive text into the pre-trained multimodal model CLIP for feature extraction, obtaining the embedding vectors of the image features and text features, where the cosine similarity of the image and text features represents the degree of semantic similarity between the image and text;

[0048] The scaling module is used to adaptively scale up or down the cosine similarity between image and text features to generate a joint representation for detection;

[0049] The detection module is used to determine whether the image to be detected is AI-generated based on the joint representation.

[0050] Beneficial effects of the present invention:

[0051] 1. This solution can perform blind box detection on images produced by various algorithms, such as GAN and diffusion models, without the need to pre-identify the image source or the generation model type. It is suitable for various complex application scenarios.

[0052] 2. This solution extracts deep semantic features through the CLIP-based image-text alignment space and combines it with descriptive text generated by BLIP. This allows it to capture subtle semantic differences between real images and AI-generated images at a cross-modal level, countering the generator's ever-increasing image fidelity.

[0053] 3. This solution introduces a two-stage rumination training strategy: the first stage generates classification confidence, and the second stage iteratively optimizes the similarity of image and text features in the latent space based on the confidence and an adaptive loss function, significantly amplifying the semantic differences between positive samples and narrowing negative samples, effectively improving the final classification accuracy.

[0054] 4. This solution uses the confidence level output from the first stage to dynamically adjust the cosine similarity of image and text features, flexibly controlling the scaling of positive and negative sample features to ensure the model's discriminative ability and overall stability on fuzzy samples.

[0055] 5. This solution relies entirely on pre-trained large models to automatically generate text and extract features, eliminating the need for manual design of feature templates or rules. This reduces manual intervention and improves the scalability and maintainability of the method.

[0056] 6. The detection device proposed in this scheme is implemented in a modular form. Each functional module - generation, acquisition, scaling, and detection - has clear logic and can be quickly deployed by loading software programs on common computing platforms and terminal devices, facilitating large-scale promotion and application.

[0057] 7. This solution is based on the numerical control of image-text cosine similarity and a two-stage training process. It can intuitively reflect the semantic consistency difference between real and generated images, and provide a quantifiable explanation basis for the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0059] Figure 1 This is a flow chart of Example 1 of the present invention.

[0060] Figure 2 This is a flowchart of the graphic feature control of the two-stage rumination training in Example 2 of the present invention;

[0061] Figure 3 This is an overall architecture diagram of Example 3 of the present invention;

[0062] In the figure: 310, generation module; 320, acquisition module; 330, scaling module; 340, detection module; DETAILED DESCRIPTION

[0063] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0064] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0065] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0066] Example 1

[0067] Reference Figure 1 , which is the first embodiment of the present invention, provides a method and device for detecting blind boxes of AI-generated images based on semantic consistency differences, which can reliably distinguish real images from AI-generated images; this method uses a pre-trained multimodal large model BLIP to generate descriptive text for each image to be detected, and then uses the pre-trained multimodal model CLIP to extract image and text features, and adaptively adjusts the cosine similarity of image and text features through two-stage rumination training, and finally the second stage MLP determines the authenticity of the image.

[0068] Specifically, the specific implementation steps of the above detection method are as follows:

[0069] S101: Image-text pair construction;

[0070] Based on a pre-trained multimodal large BLIP model with frozen parameters, a text description is generated for each image to be detected. Each image is given a descriptive text that is semantically consistent with its content (for example, "a brown dog standing in the grass");

[0071] The images to be detected can be generated by different types of AI image generation models and do not contain textual modal information. This method does not require pre-identification of the generation model type and is a universal blind box detection method. The selection of the multimodal large model BLIP and the length limit of the generated text (e.g., 10-20 words) can be flexibly configured according to actual needs and are not restricted in this embodiment.

[0072] S102: Image and text feature extraction;

[0073] Each image to be detected and its corresponding descriptive text form an image-text pair, and the pairs are input into a pre-trained multimodal model CLIP with frozen parameters for feature encoding. The model output includes embedded representations of image features and text features, and the cosine similarity is used to measure the degree of consistency between the image and text in the semantic space. Different versions of the pre-trained parameters of the multimodal model CLIP can be selected, and this embodiment does not limit them.

[0074] S103: Two-stage rumination training;

[0075] During the training phase, a two-stage ruminative training strategy is used to build an AI image detection model, specifically including:

[0076] Phase 1: The CLIP-encoded image and text features are concatenated into a joint representation in the vector dimension and input into the first multilayer perceptron (MLP1). The output is a confidence score indicating whether the image is AI-generated. The loss function is calculated using this output and the true sample label to train the MLP1 model.

[0077] The first multilayer perceptron MLP1 adopts a fully connected neural network structure, which includes:

[0078] 1. First linear layer: input dimension 1536, output dimension 768;

[0079] 2. ReLU activation function layer;

[0080] 3. Second linear layer: input dimension 768, output dimension 1;

[0081] 4. Sigmoid activation function layer;

[0082] The functions of the first multilayer perceptron MLP1 include:

[0083] In the first training phase, the primary classifier outputs a confidence probability value in the interval (0, 1) to indicate the likelihood that the input image is generated by AI.

[0084] In the subsequent training phase 2, when the parameters are frozen, the output probability value is used as the control coefficient of the adaptive loss function to dynamically determine the feature adjustment range; specifically:

[0085] When the output probability of the first multi-layer perceptron MLP1 is highly consistent with the true label (i.e., the classification confidence is high), the adjustment amplitude of the image and text features in the second stage is reduced accordingly;

[0086] When the output probability of the first multilayer perceptron MLP1 deviates significantly from the true label (i.e., the classification confidence is low), the second stage will implement differential adjustment based on the absolute value of the deviation. The larger the deviation, the stronger the feature vector adjustment intensity.

[0087] In the subsequent reasoning process, the first multilayer perceptron MLP1 does not participate in the forward calculation.

[0088] The second stage: Based on the confidence level output by the first multi-layer perceptron MLP1, the cosine similarity between the image and text features is adaptively adjusted. The lower the confidence level of the sample, the greater the adjustment of the image and text feature similarity. The adjustment strategy is as follows:

[0089] If the first stage classification is wrong and the sample is a positive example, then the similarity of its image and text features will be amplified;

[0090] If the sample is a negative example, the similarity between its image and text features is reduced;

[0091] If the classification is correct, the adjustment will be relatively small;

[0092] The second multilayer perceptron MLP2 has the same hierarchical structure as the first multilayer perceptron MLP1:

[0093] 1. First linear layer: input dimension 1536, output dimension 768;

[0094] 2. ReLU activation function layer;

[0095] 3. Second linear layer: input dimension 768, output dimension 1;

[0096] 4. Sigmoid activation function layer;

[0097] The function of the second multilayer perceptron MLP2 is:

[0098] In the second training phase, as the final classifier, the normalized probability value is output through the Sigmoid function;

[0099] In the subsequent reasoning process, its output value is used as the basis for judgment, specifically:

[0100] Probability value > 0.5: The image is determined to be AI-generated;

[0101] Probability value < 0.5: judged to be a real image;

[0102] Probability value = 0.5: indicates uncertainty.

[0103] After the adjusted image-text features complete feature update in the latent space, they are mapped back to the original space and residually fused with the initial features to obtain the final adjusted image-text features. This feature pair is concatenated again to form a new joint representation, which is passed as input to the second multilayer perceptron (MLP2). The loss value between the two labels and the true label is calculated to further optimize MLP2.

[0104] It should be noted that this two-stage strategy is only applied to the model training stage; in the actual detection process, only the regulated image and text features are input into the second multi-layer perceptron (the second multi-layer perceptron MLP2), and the first multi-layer perceptron does not participate in the detection.

[0105] S104: Detection and determination;

[0106] Based on the output result of the second multi-layer perceptron MLP2, determine whether the input image is generated by AI.

[0107] The present embodiment provides an AI-generated image blind box detection method based on semantic consistency differences. First, based on the text generation function of the frozen pre-trained multimodal large model BLIP, a descriptive text is generated for the image to be detected, and a picture-text pair is formed with the original image. Feature embedding is performed together through the frozen pre-trained multimodal model CLIP to generate picture-text features. The picture-text features are then spliced ​​together to form a joint representation and input into the first multi-layer perceptron to obtain a confidence score. Based on this confidence score, the cosine similarity of the picture-text features can be adaptively adjusted in the second stage to amplify the similarity between the real picture and its descriptive text and reduce the similarity between the AI-generated picture and its descriptive text. The adjusted picture-text features are then combined into a new joint representation, and the new joint representation is input into the second multi-layer perceptron to accurately obtain the detection result of whether the picture is an AI-generated picture. In AI-generated images, due to the fundamental limitations of the generator, it is often difficult to fully express semantic information in the generated images. Therefore, the cosine similarity between real images and their descriptive text is always higher than the cosine similarity between AI-generated images and their descriptive text. This is an invariant feature shared across different generative models. Therefore, by adaptively amplifying and reducing the cosine similarity of image and text features through ruminant training, the ease of extraction of this invariant feature can be amplified, thereby enabling the second multi-layer perceptron to detect accurate and reliable detection results.

[0108] Example 2

[0109] Reference Figure 2 , which is the second embodiment of the present invention. This embodiment differs from the first embodiment in that: based on the above embodiment, this embodiment further optimizes the graphic feature regulation process of the above two-stage rumination training, including:

[0110] S201, Phase 1: Joint representation construction and confidence output;

[0111] Based on the frozen parameter pre-trained multimodal large model CLIP, the image to be detected and its corresponding text description are encoded separately to obtain image and text features;

[0112] Subsequently, the image and text features are spliced ​​in a dimension of length 1 to form a joint image and text representation, which is input into the first multi-layer perceptron (the first multi-layer perceptron MLP1), and the classification confidence of whether the output image is generated by AI is , , A value close to 0.5 indicates that the model is uncertain whether the image is AI-generated, a value close to 1 indicates that the model tends to judge it as an AI-generated image, and a value close to 0 indicates that the model tends to judge it as a real image.

[0113] S202, the second stage: similarity control and feature iteration;

[0114] Classification confidence according to the output of the first multi-layer perceptron MLP1 , combined with sample labels (0 is a real picture, 1 is an AI-generated picture), adaptively calculate the scaling factor between image and text features , the adaptive control coefficient is , used to adjust the semantic consistency of image and text features;

[0115] like and misclassified ( ),but , amplify the cosine similarity of positive samples;

[0116] like and misclassified ( ),but , reduce the cosine similarity of positive samples;

[0117] Specifically, after mapping the original features of the image and text to the latent space, the features are iteratively updated based on the loss function proposed in this scheme. The loss function is expressed as:

[0118] ;

[0119] in, is the cross entropy loss function, is the adaptive control coefficient, 、 are the feature vectors after image and text adjustment respectively, To calculate the cosine similarity;

[0120] The above loss function calculates the gradient through backpropagation, guiding the image-text feature similarity of positive samples to gradually increase in the latent space, while the similarity of negative samples to gradually decrease.

[0121] After completing the feature iteration, the regulated image-text features are mapped back to the original space and connected with the original feature residual to retain the initial semantic information to form the final regulated image-text features.

[0122] S203, joint representation construction and second multi-layer perceptron MLP2 input;

[0123] Concatenate the regulated image features and text features in a dimension of length 1 to construct the final joint image and text representation;

[0124] In specific implementation, for example, in an embodiment, the vectors may be spliced ​​based on the following formula: ;

[0125] in, is the joint representation vector of image and text, 、 are the feature vectors after image and text adjustment respectively.

[0126] The joint representation is input into the second multilayer perceptron (MLP2) as the basis for the final detection.

[0127] S204, detection decision based on the second multi-layer perceptron MLP2;

[0128] The output of the second multi-layer perceptron is activated by the Sigmoid function to finally determine whether the image to be detected is AI generated;

[0129] Specifically, the output of the second perceptron is to After activation by the Sigmoid function, the output is mapped to the interval (0, 1), with 0.5 as the demarcation threshold. If the output after activation is in the interval (0, 0.5), the detection result is a real picture; if the output after activation is in the interval (0.5, 1), the detection result is an AI-generated picture;

[0130] Specifically:

[0131] Indicates that the image is likely to be generated by AI;

[0132] Indicates that the probability of the picture being a real picture is high;

[0133] Indicates that the classification result is ambiguous.

[0134] Example 3

[0135] Reference Figure 3 , which is the third embodiment of the present invention. This embodiment differs from the first embodiment in that: to cooperate with the above-mentioned method embodiment, this embodiment also provides an AI-generated image blind box detection device based on semantic consistency difference, which is applied to the above-mentioned AI-generated image blind box detection method based on semantic consistency difference; the device is suitable for deployment in an AI-generated image detection system based on semantic consistency difference, and can be implemented by running on a device with computing capability in a software manner. The functions of the device described in this embodiment are executed by the processor in the computing device by calling the computer program instructions stored in the non-volatile memory and loading them into the memory, thereby forming a corresponding logic module.

[0136] like Figure 3 As shown, the device includes: a generation module 310, an acquisition module 320, a scaling module 330 and a detection module 340, and its specific functions are as follows:

[0137] A generation module 310 is configured to generate descriptive text that matches the semantic content of each image to be detected based on the pre-trained multimodal model BLIP;

[0138] The acquisition module 320 is configured to input the image to be detected and its corresponding descriptive text as an image-text pair into a pre-trained multimodal model CLIP to extract image-text embedding features that can be recognized by a computer; wherein the cosine similarity of the image-text features is used to measure the similarity between the image and the text in the semantic space;

[0139] Scaling module 330 is used to adaptively adjust the above-mentioned image-text embedded features based on semantic consistency differences to generate a joint representation for image-text judgment. During the adjustment process, the image-text feature similarity of positive samples is dynamically amplified and the image-text feature similarity of negative samples is reduced according to the detection confidence of the previous stage. The adjusted image-text features are then restored to the original space and concatenated with the original features through residual connection to form a joint feature representation.

[0140] The detection module 340 is used to determine whether the input image is an AI-generated image based on the above joint representation.

[0141] The device can be used to perform Figure 1 The technical implementation principles and processes of each step in the method embodiment shown are consistent and will not be repeated here.

[0142] The following are optional implementations of this device:

[0143] a. Optionally, the generating module 310 is specifically configured to generate a semantic description text for the image to be detected based on the multimodal large model BLIP with frozen parameters;

[0144] b. Optionally, the acquisition module 320 includes:

[0145] Used to embed features of images based on the multimodal model CLIP with frozen parameters to obtain image embedding features;

[0146] Used to embed features of descriptive text based on the same model and obtain text embedding features;

[0147] c. Optionally, the scaling module 330 specifically includes:

[0148] for determining a control coefficient for adjusting the range of the image and text features based on the output of the first multi-layer perceptron;

[0149] It is used to adjust the image and text features in the latent space according to the control coefficient, and perform residual connection between the adjusted features and the initial features;

[0150] Used to splice the adjusted image and text features into a joint image and text representation.

[0151] d. Optionally, the detection module 340 specifically includes:

[0152] Inputting the image-text joint representation into a second multilayer perceptron to obtain a preliminary detection output;

[0153] The output is mapped to a probability value through Sigmoid function activation;

[0154] By setting a threshold, it is determined whether the image is AI-generated.

[0155] The modules of the device described in this embodiment can be physical structural units or logical functional modules running on a computing platform. Their deployment can be centralized or distributed according to specific application requirements. Modules can interact with each other through function calls, message passing, or shared memory.

[0156] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An AI-generated image blind box detection method based on semantic consistency differences, characterized by: The method includes: Use the pre-trained multimodal large model BLIP to generate corresponding descriptive text for the image to be detected; Use the pre-trained multimodal model CLIP to embed features of images and their descriptive text, obtaining image and text features based on the CLIP image-text alignment space. The cosine similarity between image and text features indicates the degree of semantic similarity between the image and text. The above-mentioned image and text features are processed based on a two-stage rumination training strategy to train a multi-layer perceptron, where: Phase 1: The image and text features are concatenated into a joint representation, which is then fed into the first multilayer perceptron (MLP1) to determine the confidence that the image is AI-generated. A loss function is then calculated based on the output of the first multilayer perceptron (MLP1) and the true sample labels to train the first multilayer perceptron (MLP1). The second stage: The confidence level output from the first stage is used as the adjustment coefficient, and the image and text features are adjusted using the loss function, so that the similarity of the image and text features of the positive sample increases and the similarity of the image and text features of the negative sample decreases. The adjusted image and text features are restored to the original space and spliced ​​again to form a new joint representation. The new detection output is input to the second multi-layer perceptron MLP2, and the loss function is calculated based on the output and the true label of the sample to train the second multi-layer perceptron MLP2. Determine whether the input image is AI-generated based on the output of the second multi-layer perceptron MLP2 in the second stage; Two-phase rumination training includes: Using the joint representation to train the first multilayer perceptron MLP1, the output of the first multilayer perceptron MLP1 and the true label of the sample are used to calculate the loss function and train the first multilayer perceptron MLP1; The output of the first multilayer perceptron MLP1 and the joint representation are input into the second multilayer perceptron MLP2 to obtain the detection output of the two-stage training; the loss function is calculated based on the output and the true label of the sample, and the second multilayer perceptron MLP2 is trained by iteratively updating the image and text features in the latent space; The iteration of the image-text features in the latent space is used to adjust according to the sample label: when the sample label is positive, the cosine similarity of the image-text features of the positive sample is increased; when the sample label is negative, the cosine similarity of the image-text features of the negative sample is reduced; The output of the second multi-layer perceptron MLP2 is used as the basis for the final determination of whether the image to be detected is generated by AI.

2. The method for detecting blind boxes in AI-generated images based on semantic consistency differences according to claim 1, characterized in that: The images to be detected are generated by a variety of different generative models and only contain image modality information without any text modality information.

3. The method for detecting blind boxes in AI-generated images based on semantic consistency differences according to claim 2, characterized in that: Generate descriptive text for the image to be detected using a pre-trained multimodal model BLIP, and jointly encode the image and the corresponding descriptive text using a pre-trained multimodal model CLIP. The method includes: Freeze the parameters of the multimodal large model BLIP, input the images to be tested into the model one by one, and obtain the descriptive text corresponding to each image; Freeze the parameters of the multimodal model CLIP, form an image-text pair with each image to be detected and its corresponding descriptive text, and input it into the image-text alignment space of the multimodal model CLIP for feature embedding to obtain the embedded features used to calculate the similarity between the image and the text.

4. The method for detecting blind boxes in AI-generated images based on semantic consistency differences according to claim 3, characterized in that: In the CLIP image-text alignment space, the image features and text features are spliced ​​on a dimension of the feature vector with a length of 1 to form a joint representation; after splicing, the length of this dimension remains 1, and the length of the other spliced ​​dimension is twice the total length of the dimension before splicing.

5. A device for detecting blind boxes in AI-generated images based on semantic consistency differences, applied to the method for detecting blind boxes in AI-generated images based on semantic consistency differences described in any one of claims 1 to 4, characterized in that: include, A generation module (310) is used to generate matching descriptive text for each image to be detected using a pre-trained multimodal model BLIP; An acquisition module (320) is used to input the image and its corresponding descriptive text into a pre-trained multimodal model CLIP for feature extraction, and obtain embedding vectors of image features and text features, wherein the cosine similarity of the image and text features represents the degree of semantic similarity between the image and the text; A scaling module (330) for adaptively scaling up or down the cosine similarity between image and text features to generate a joint representation for detection; The detection module (340) is used to determine whether the image to be detected is generated by AI based on the joint representation.

Images