Method, device and equipment for detecting forged image

Abstract

The invention discloses a forged image detection method, device and equipment. Comprising the steps of collecting images and description texts thereof, and constructing a training data set; extracting image features and text features in the training data set through a CLIP model; constructing constraints based on the image features and the text features, establishing an optimization target by adopting a conditional information bottleneck method, and training a forged image detection model to obtain a trained forged image detection model; inputting a to-be-identified image into the forged image detection model, and outputting a detection value; and under the condition that the detection value is greater than a preset threshold value, determining that the to-be-identified image is a forged image. On the basis of the strong feature expression ability of the pre-training model, the problem of feature redundancy is solved by using a conditional information bottleneck method, the enhancement effect of the text mode on the detection ability of the forged image is effectively utilized, the detection performance of balance and strong generalization ability on different types of forged images is realized, and meanwhile, the detection efficiency is improved. And the detection accuracy of the forged image is improved.

Description

Technical Field

[0001] This application relates to the field of image detection technology. Specifically, it relates to a method, apparatus, and device for detecting forged images. Background Art

[0002] With the rapid development of image generation models, the generated fake images are becoming more and more realistic, which has caused a series of serious social problems. For example, the widespread dissemination of false news and misinformation has led to a significant decline in the public's trust in the media. In addition, since the fake images generated by different types of generation models often have their own unique characteristics, many existing detection methods perform poorly when faced with unseen fake images and have limited generalization ability. Summary of the Invention

[0003] Embodiments of this application provide a method, apparatus, and device for detecting forged images to at least solve the technical problems of poor generalization ability and low detection accuracy of false image detection models in related technologies.

[0004] According to one aspect of the embodiments of this application, a method for detecting forged images is provided, including:

[0005] Collect images and their description texts to construct a training data set;

[0006] Extract image features and text features in the training data set through the CLIP model; construct constraints based on the image features and text features, and use the conditional information bottleneck method to establish an optimization objective, and train a forged image detection model to obtain a trained forged image detection model;

[0007] Input the image to be recognized into the forged image detection model and output a detection value;

[0008] If the detection value is greater than a preset threshold, determine that the image to be recognized is a forged image.

[0009] In one implementation, collecting images and their description texts to construct a training data set includes:

[0010] Collect a large number of real images and their corresponding description texts, as well as forged images and their corresponding description texts;

[0011] Preprocess the collected images and their description texts to obtain the constructed training data set.

[0012] In one implementation, extracting image features in the training data set through the CLIP model and constructing constraints based on the image features includes:

[0013] Extract image features through the image encoder of the CLIP model;

[0014] Enhance the image features, and use a linear layer to extract and compress the enhanced image features to obtain compressed intermediate features;

[0015] Reparametrize based on the compressed intermediate features to generate hidden layer representations;

[0016] Establish a conditional information bottleneck constraint based on the hidden layer representations.

[0017] In one implementation, the conditional information bottleneck method is used to establish an optimization objective, including:

[0018] Construct an optimization objective based on the conditional information bottleneck constraint, where the optimization objective includes minimizing the conditional mutual information between the features and the image and maximizing the mutual information between the features and the category under the condition of given text and category.

[0019] In one implementation, the text features in the training dataset are extracted through a CLIP model, and constraints are constructed based on the text features, including:

[0020] Extract text features through the text encoder of the CLIP model, and divide the text features into real image text features and forged image text features;

[0021] Apply a linear layer to project the real image text features and the forged image text features to obtain projected text features;

[0022] Perform weighted summation on the projected real image text features and the projected forged image text features respectively to obtain weighted pooled text features;

[0023] Fuse and orthogonalize the weighted pooled text features to obtain a conditional constraint with both text modality and category modality.

[0024] In one implementation, fusing and orthogonalizing the weighted pooled text features to obtain a conditional constraint with both text modality and category modality includes:

[0025] Perform weighted fusion on the weighted pooled text features of the current training round and the weighted pooled text features of the historical training rounds to obtain fused text features;

[0026] Use the Schmidt orthogonalization method to orthogonalize the fused text features as a conditional constraint with both text modality and category modality.

[0027] In one implementation, the forged image detection model further includes:

[0028] Use a linear layer and a Sigmoid function as a classifier to calculate the probability value of a forged image.

[0029] In one embodiment, it further includes:

[0030] Obtain the distribution mean of features under the conditions of a given image and text;

[0031] Obtain the target distribution mean of features under the conditions of a given text and category;

[0032] Use the square of the Euclidean distance to measure the difference between the two means, and obtain the constructed loss function.

[0033] According to another aspect of the embodiments of the present application, there is provided a forged image detection device, including:

[0034] A training module, configured to collect images and their description texts, construct a training data set; extract image features and text features in the training data set through a CLIP model; construct constraints based on the image features and text features, and establish an optimization target by using the conditional information bottleneck method to train a forged image detection model to obtain a trained forged image detection model;

[0035] A detection module, configured to input an image to be recognized into the forged image detection model and output a detection value; and determine that the image to be recognized is a forged image when the detection value is greater than a preset threshold.

[0036] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to execute the above-mentioned forged image detection method through the computer program.

[0037] The technical solutions provided by the embodiments of the present application may include the following beneficial effects:

[0038] The forged image detection method based on conditional information bottleneck and pre-trained model disclosed in the embodiments of the present application, on the basis of the powerful feature expression ability of the pre-trained model, uses the conditional information bottleneck method to solve the problem of feature redundancy, and effectively utilizes the enhancement effect of the text modality on the forged image detection ability in view of the characteristics of the CLIP multi-modal feature space. This method not only achieves balanced and strong generalization ability detection performance on different types of forged images, but also analyzes the reason why the text modality can improve the accuracy of forged image detection in the CLIP feature space, and improves the accuracy of image detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application, and do not constitute an improper limitation to the present application. In the drawings:

[0040] Figure 1 is a flowchart of a method for detecting forged images according to an embodiment of the present application;

[0041] Figure 2 is the difference between the multi-modal conditional information bottleneck method used in the present application and the traditional information bottleneck method according to an embodiment of the present application;

[0042] Figure 3 is a flowchart of a forged image detection model according to an embodiment of the present application;

[0043] Figure 4 is a schematic diagram of a device for detecting forged images according to an embodiment of the present application;

[0044] Figure 5 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0045] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0046] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0047] Currently, artificial intelligence-generated image detection models can be mainly divided into three categories: methods based on pre-trained models, reconstruction-based diffusion methods, and frequency-based methods. Among them, methods based on pre-trained models, especially methods using the CLIP model, demonstrate significant advantages due to their strong feature extraction capabilities and generalization performance.

[0048] Although the CLIP-based artificial intelligence-generated image detection method has strong generalization ability, there is a key problem that has been overlooked: Although the features of CLIP are highly expressive, they may exhibit redundancy.

[0049] The information bottleneck method provides a powerful theoretical and practical framework to address this redundancy problem, enabling the extraction of compressed and generalizable representations. Additionally, this method aims to utilize the strong multimodal capabilities of CLIP to enhance its ability to detect forged images.

[0050] However, experimental results show that using only the corresponding prompts as conditions leads to poor performance. To solve this problem, this method first conducts text analysis experiments and, based on the conclusions of the experiments, proposes a dynamic pooling text feature calculation method to improve the generated image detection method based on conditional information bottleneck and CLIP.

[0051] The following combines the attached Figures 1-3 to introduce in detail the method for detecting forged images in the embodiments of this application. As Figure 1 shown, the method mainly includes the following steps:

[0052] S101 Collect images and their descriptive texts to construct a training dataset.

[0053] In one implementation, first, collect a large number of real images and their corresponding descriptive texts, as well as forged images and their corresponding descriptive texts. Ensure that the dataset covers various types of forged images, such as images generated by different generative models, to enhance the generalization ability of the model.

[0054] Furthermore, preprocess the collected images and their descriptive texts to obtain the constructed training dataset.

[0055] For example, first preprocess the images: Resize the images to unify the input size. Normalize the images by scaling the pixel values to the range [0,1] or other appropriate ranges. Images can also be enhanced (such as rotated, cropped, flipped, etc.) as needed to expand the dataset and improve the robustness of the model.

[0056] Then preprocess the text: Tokenize the descriptive text to convert the text into a sequence of words. Convert the tokenized text into numericalized embedding vectors, and the text can also be truncated or padded to make its length consistent to fit the model input.

[0057] Pair the preprocessed images and texts to form training samples.

[0058] S102 extracts the image features and text features in the training dataset through the CLIP model; constructs constraints based on the image features and text features, and uses the conditional information bottleneck method to establish an optimization objective, trains the forged image detection model, and obtains the trained forged image detection model.

[0059] Furthermore, train the forged image detection model.

[0060] First, extract the text features and image features in the training dataset through the CLIP model, and then process the text features and image features respectively.

[0061] Specifically, extract the text features through the text encoder of the CLIP model, and divide the text features into real image text features and forged image text features according to the image category. Apply a linear layer to project the real image text features and forged image text features to obtain the projected text features t=(t r ,t f ) T .

[0062] Furthermore, in order to utilize the inherent "bias" and hint semantics in the CLIP feature space, dynamic text orthogonality is used in the method to obtain the weighted pooled text features t. Dynamic text orthogonality includes a pooling process and a fusion process.

[0063] In the pooling process, perform weighted summation on the projected real image text features and forged image text features respectively to obtain the weighted pooled text features.

[0064] Furthermore, perform fusion and orthogonality processing on the weighted pooled text features to obtain the conditional constraints with both text modality and category modality.

[0065] Specifically, in batch fusion, perform weighted fusion on the weighted pooled text features of the current training round and the weighted pooled text features of the historical training rounds to obtain the fused text features. The calculation method is as follows:

[0066]

[0067] where L p is a hyperparameter that controls the global pooling scale, B is the input batch size, and * represents the group corresponding to real images or fake images.

[0068] In the orthogonality stage, use the Schmidt orthogonality method to perform orthogonality processing on the fused text features as the conditional constraints with both text modality and category modality.

[0069] It can be understood that in the forged image detection model, adding a text processing branch and constructing conditional constraints with text modality and category modality has the following effects:

[0070] Multi-modal feature fusion: By combining image features with text features, the model can utilize the information of both modalities to enhance the detection ability for forged images. The text features provide additional semantic information about the image content, helping the model better understand the context of the image.

[0071] Introduction of category information: By dividing the text features into real-image text features and fake-image text features and applying a linear layer to obtain projected text features, the model can introduce category information. This helps the model learn the feature representations for distinguishing real and forged images.

[0072] Dynamic text orthogonalization: The dynamic text orthogonalization process (including pooling and fusion) enables the model to extract more representative and discriminative text features. Through the Schmidt orthogonalization process, redundant information in the text features is removed, making the features more independent and enhancing the generalization ability of the model.

[0073] Construction of conditional constraints: Construct conditional constraints with text modality and category modality, enabling the model to consider both text information and category information when learning image features. This helps the model better capture the semantic association between images and texts while satisfying the conditional information bottleneck constraint, enhancing the detection accuracy for forged images.

[0074] Therefore, adding a text processing branch and constructing conditional constraints with text modality and category modality enable the model to fully utilize multi-modal information and enhance the detection ability for forged images. This process provides the model with richer semantic information and category information, helping to enhance the generalization ability and accuracy of the model.

[0075] Furthermore, image features are extracted through the image encoder of the CLIP model.

[0076] Image features can be enhanced by introducing compound Gaussian perturbations into image I:

[0077]

[0078] where μ I and σ I are the mean and standard deviation of I, and Σ s is a hyperparameter used to control the enhancement intensity.

[0079] Furthermore, a linear layer is used to extract and compress the enhanced image features to obtain compressed intermediate features; the main purpose of the linear layer is to further extract and compress the enhanced image features to obtain a more compact and expressive intermediate feature representation μ e and Σ eThese intermediate features serve as the basis for subsequent reparameterization steps.

[0080] Furthermore, reparameterization is performed based on the compressed intermediate features to generate hidden layer representations. Reparameterization is a crucial step in generating the final hidden layer representation Z, which introduces random noise combined with the intermediate feature representation μ generated by the linear layer e and Σ e to generate a hidden layer representation Z with randomness and generalization ability:

[0081] z = μ e + Σ e ∈

[0082] Finally, a conditional information bottleneck constraint is established based on the hidden layer representation.

[0083] In one implementation, the conditional information bottleneck method is used to establish an optimization objective, including: constructing an optimization objective based on the conditional information bottleneck constraint, where the optimization objective includes minimizing the conditional mutual information between the features and the image under the given text and category, while maximizing the mutual information between the features and the category.

[0084] The conditional information bottleneck constraint is an optimization objective aimed at balancing the compressibility and relevance of the feature Z through information - theoretic methods. Establish the conditional information bottleneck optimization objective:

[0085] min I(Z; X|T,Y) - βI(Z; Y);

[0086] The goal is to minimize the conditional mutual information I(Z; X|T,Y) while maximizing the mutual information I(Z; Y).

[0087] Where: I(Z; X|T,Y) represents the redundant information between the feature Z and the image X under the given text T and category Y. I(Z; Y) represents the correlation between the feature Z and the category Y. β is a hyperparameter used to balance these two objectives.

[0088] Transform the optimization objective and use variational approximation to obtain the formula:

[0089]

[0090] Where r(·) is the variational approximation, e(·) and c(·) represent the encoder and classifier in the model respectively, and <·> represents E p(x,y,t)p(z∣x,t) .

[0091] Optimize the above objective by constructing a loss function (such as LMMD) so that the model learns a feature representation Z that satisfies the conditional information bottleneck constraint during training.

[0092] In one implementation, obtain the distribution mean of features under the conditions of a given image and text; obtain the target distribution mean of features under the conditions of a given text and category; use the square of the Euclidean distance to measure the difference between the two means, and obtain the constructed loss function:

[0093]

[0094] where, calculate the mean μ of the feature distribution output by the encoder e(z∣x,t) e(z∣x,t) , which represents the distribution of the feature z under the conditions of the given image x and text t. Calculate the mean μ of the variational approximation distribution r(z∣t,y) r(z∣t,y) , which represents the target distribution of the feature z under the conditions of the given text t and category y.

[0095] Use the square of the Euclidean distance to measure the difference between the two mean vectors. This loss function makes the feature distribution output by the encoder closer to the target distribution by minimizing the mean difference between the two distributions. The loss function ensures that while the feature z satisfies the conditional information bottleneck constraint, it can better serve the forgery image detection task by minimizing the mean difference between the feature distribution output by the encoder and the target distribution.

[0096] It can be understood that the conditional constraint constructed based on text features refers to the additional constraint conditions introduced in the model to guide the feature learning process. In the forgery image detection model, the conditional constraint is realized by the pooled text features after Schmidt orthogonalization. These text features contain information of both the text modality and the category modality, and are used to enhance the semantic relevance of the feature Z.

[0097] In one implementation, the forgery image detection model further includes: using a linear layer and a Sigmoid function as a classifier to calculate the probability value of the forgery image.

[0098] Train the model based on the model structure, constraints, optimization objectives, loss functions, etc. given in the embodiments of the present application. After the model training is completed, save the parameters and structure of the model for subsequent use. Use the test set to perform final testing and evaluation on the model to further verify the performance and generalization ability of the model in actual applications.

[0099] S103 Input the image to be recognized into the forgery image detection model and output the detection value.

[0100] S104 When the detection value is greater than the preset threshold, determine that the image to be recognized is a forgery image.

[0101] In an exemplary embodiment, input the image to be recognized into the forgery image detection model. The model will automatically extract the features of the image and perform analysis. The description text may not be input when the model is applied.

[0102] Further, the model calculates a detection value based on the features of the input image, representing the possibility that the image is a forged image. The detection value output by the model is compared with a preset threshold. If the detection value is greater than the preset threshold, the model determines that the image is a forged image. If the detection value is less than or equal to the preset threshold, the model determines that the image is a genuine image. The specific value of the preset threshold is not specifically limited in this application and can be set according to the actual situation. For example, when the threshold is 0.5, if the output detection value is greater than 0.5, the image is determined to be a forged image, and if the output detection value is less than 0.5, the image is determined to be a genuine image.

[0103] To facilitate understanding of the method provided by the embodiments of this application, the following will be further described with reference to the Figure 2 and 3 drawings.

[0104] As Figure 2 shown, the differences between the multi-modal conditional information bottleneck method used in this method and the traditional information bottleneck method are shown.

[0105] (a) In the standard Information Bottleneck (IB) method, both minI(Z; X) and maxI(Z; Y) are constrained. Although minimizing I(Z; X) may lose Z related to Y; (b) In the Conditional Information Bottleneck (CIB), this problem is solved by restricting the given Y, that is, by minI(Z; X|Y) and maxI(Z; Y); (c) The Venn diagram in (c) illustrates the relationship between X, Y, and Z of IB and CIB under ideal conditions); (d) In (d), it is described that in the case of given \T and \Y, constraints are imposed on minI(Z; X|T,Y) and maxI(Z; Y), and Schmidt orthogonalization is used to eliminate the redundancy in T. (e) In (e), it is explained that in the ideal case of this method, compared with (c), the model can learn additional information about Y in Z.

[0106] As Figure 3 shown, the method first uses the text encoder and image encoder of CLIP to extract text features and image features respectively. For the image processing branch, first, a compound Gaussian perturbation is introduced into the image I, second, reparameterization is used to generate the hidden layer representation Z, and finally, a conditional information bottleneck constraint is established for z. The model uses a linear layer and a Sigmoid function as the classifier to obtain the probability

[0107] For the text processing branch, the text features corresponding to real images and fake images are processed separately. The input text features are divided into real-image text features and fake-image text features according to the image category, and a linear layer is applied to obtain the projected text features. In order to utilize the inherent "bias" and prompt semantics in the CLIP feature space, dynamic text orthogonality is used in the method to obtain the weighted pooled text feature t. Dynamic text orthogonality includes a pooling process and a fusion process. In the pooling process, the text is projected into the latent variable space, and the weighted sum of the text features corresponding to real images and the text features corresponding to fake images is calculated separately. The fusion includes batch fusion and orthonormalization. In batch fusion, the model uses the pooled text features to perform weighted fusion with the text features in the historical batches. In the orthonormalization stage, the Schmidt orthonormalization is used in the method to orthonormalize the fused features, introducing category information and removing irrelevant information.

[0108] Based on the pre-trained model, the forged image detection method of this application solves the feature redundancy problem through the information bottleneck method and uses multi-modal (image + text) features to enhance the detection ability. This method not only improves the generalization performance of the model, but also deeply analyzes the improvement effect of the text modality on the detection accuracy, achieving efficient detection of different types of forged images.

[0109] According to another aspect of the embodiments of this application, there is also provided a forged image detection device for implementing the above-mentioned forged image detection method. As Figure 4 shown, the device includes:

[0110] A training module 401, configured to collect images and their description texts, construct a training data set; extract image features and text features in the training data set through a CLIP model; construct constraints based on the image features and text features, establish an optimization objective using the conditional information bottleneck method, and train a forged image detection model to obtain a trained forged image detection model;

[0111] A detection module 402, configured to input the image to be recognized into the forged image detection model and output a detection value; determine that the image to be recognized is a forged image when the detection value is greater than a preset threshold.

[0112] It should be noted that when the above-mentioned forged image detection device executes the forged image detection method, only the above-mentioned division of each functional module is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the above-mentioned forged image detection device provided by the embodiments and the forged image detection method embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0113] According to another aspect of the embodiments of the present application, there is also provided an electronic device corresponding to the method for detecting forged images provided in the foregoing embodiments to execute the method for detecting forged images described above.

[0114] Please refer to Figure 5 , which shows a schematic diagram of an electronic device provided in some embodiments of the present application. As Figure 5 shown, the electronic device includes: a processor 500, a memory 501, a bus 502, and a communication interface 503. The processor 500, the communication interface 503, and the memory 501 are connected through the bus 502; a computer program that can run on the processor 500 is stored in the memory 501, and when the processor 500 runs the computer program, it executes the method for detecting forged images provided in any of the foregoing embodiments of the present application.

[0115] Among them, the memory 501 may include a high-speed random access memory (RAM: Random Access Memory), and may also include a non-volatile memory, such as at least one disk memory. Through at least one communication interface 503 (which can be wired or wireless), a communication connection is realized between this system network element and at least one other network element, and the Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0116] The bus 502 may be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Among them, the memory 501 is used to store programs. After receiving an execution instruction, the processor 500 executes the program. The method for detecting forged images disclosed in any implementation manner of the foregoing embodiments of the present application can be applied to the processor 500 or implemented by the processor 500.

[0117] The processor 500 may be an integrated circuit chip with the ability to process signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor 500 or instructions in the form of software. The above-mentioned processor 500 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 501, and the processor 500 reads the information in the memory 501 and combines its hardware to complete the steps of the above method.

[0118] The electronic device provided by the embodiments of the present application and the method for detecting forged images provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run, or implemented by them.

[0119] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0120] The above embodiments only represent several implementation manners of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

Claims

1. A method for detecting forged images, characterized in that, Including: Collect images and their description texts to construct a training dataset; Extract image features and text features in the training dataset through the CLIP model; Construct constraints based on the image features and text features, adopt the conditional information bottleneck method to establish an optimization objective, train a forged image detection model, and obtain a trained forged image detection model; Input the image to be recognized into the forged image detection model and output a detection value; When the detection value is greater than a preset threshold, determine that the image to be recognized is a forged image.

2. The method according to claim 1, wherein Collect images and their description texts to construct a training dataset, including: Collect a large number of real images and their corresponding description texts, as well as forged images and their corresponding description texts; Preprocess the collected images and their description texts to obtain the constructed training dataset.

3. The method according to claim 1, wherein Extract image features in the training dataset through the CLIP model, and construct constraints based on the image features, including: Extract image features through the image encoder of the CLIP model; Enhance the image features, use a linear layer to extract and compress the enhanced image features to obtain compressed intermediate features; Perform reparameterization based on the compressed intermediate features to generate hidden layer representations; Establish a conditional information bottleneck constraint based on the hidden layer representations.

4. The method according to claim 3, characterized in that, Adopt the conditional information bottleneck method to establish an optimization objective, including: Construct an optimization objective based on the conditional information bottleneck constraint. The optimization objective includes minimizing the conditional mutual information between the features and the images under the given text and category, while maximizing the mutual information between the features and the category.

5. The method according to claim 1, wherein Extract text features in the training dataset through the CLIP model, and construct constraints based on the text features, including: Extract text features through the text encoder of the CLIP model, and divide the text features into real image text features and forged image text features; Apply a linear layer to project the real image text features and the forged image text features to obtain projected text features; Perform weighted summation on the projected real image text features and the forged image text features respectively to obtain weighted pooled text features; Perform fusion and orthogonalization processing on the weighted pooled text features to obtain a conditional constraint with both text modality and category modality.

6. The method according to claim 5, wherein Perform fusion and orthogonalization processing on the weighted pooled text features to obtain a conditional constraint with both text modality and category modality, including: Perform weighted fusion on the weighted pooled text features of the current training round and the weighted pooled text features of the historical training rounds to obtain fused text features; Use the Schmidt orthogonalization method to perform orthogonalization processing on the fused text features as a conditional constraint with both text modality and category modality.

7. The method according to claim 1, wherein The forged image detection model further includes: Use a linear layer and a Sigmoid function as a classifier to calculate the probability value of a forged image.

8. The method according to claim 1, wherein Also including: Obtain the distribution mean of the features under the given image and text; Obtain the target distribution mean of the features under the given text and category; Use the square of the Euclidean distance to measure the difference between the two means to obtain the constructed loss function.

9. A forged image detection device, characterized in that, Including: A training module, configured to collect images and their description texts, and construct a training dataset; extract image features and text features in the training dataset through a CLIP model; Construct constraints based on the image features and text features, establish an optimization objective using the conditional information bottleneck method, train a forged image detection model, and obtain a trained forged image detection model; A detection module, configured to input an image to be recognized into the forged image detection model and output a detection value; and determine that the image to be recognized is a forged image when the detection value is greater than a preset threshold.

10. An electronic device, characterized in that, It includes a processor and a memory storing program instructions, and the processor is configured to execute the method for detecting a forged image according to any one of claims 1 to 8 when executing the program instructions.

Images