Artificial intelligence generated image fairness detection method and system

Abstract

The invention provides an artificial intelligence generated image fairness detection method and system, and relates to the technical field of image processing, and the method comprises the steps: inputting a natural image into a pre-trained image encoder to obtain natural semantics, combining an AI generated image with the natural image, inputting the combined image into the image encoder to obtain original image semantics, and outputting the original image semantics; inputting a to-be-trained fair adapter network model to carry out residual fusion, obtaining nature and generating enhanced semantics; fusing the natural semantics and the generated enhanced semantics to generate a first mixed sample, fusing the natural semantics and the naturally enhanced semantics to generate a second mixed sample, and obtaining a mixed sample; inputting the mixed sample into a classification head to calculate a fair adapter loss function, and obtaining a trained fair adapter network model; and inputting the enhanced semantics into a to-be-trained classification network model to obtain image classification semantics, and generating a classification result. According to the invention, the brand-new fairness auxiliary module and classification module are constructed, so that the image fairness detection performance can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a fairness detection method and system for artificial intelligence generated images. Background Art

[0002] The high-quality realistic images generated by generative models pose a significant challenge to identifying these images. So far, data-driven deep neural networks have been shown to be the most effective forensic tool to address these challenges. However, they may overfit certain semantics, resulting in considerable inconsistency in detection performance between different contents of generated samples, leading to low image fairness detection performance.

[0003] Therefore, it is urgent to provide a solution to improve the above problems. Summary of the invention

[0004] The purpose of the present invention is to provide an artificial intelligence generated image fairness detection method and system to improve the problem of low image fairness detection performance in the prior art.

[0005] In a first aspect, the present invention provides an artificial intelligence generated image fairness detection method using the following technical solutions: Select multiple natural images and input them into a pre-trained image encoder for encoding to obtain natural semantics; The AI-generated image and the natural image are combined and input into the pre-trained image encoder for encoding to obtain the original image semantics, and the original image semantics are input into the fair adapter network model to be trained for residual fusion to obtain enhanced semantics, wherein the enhanced semantics include: natural enhanced semantics and generated enhanced semantics; Fusing natural semantics with generated enhanced semantics to generate a first mixed sample, fusing natural semantics with natural enhanced semantics to generate a second mixed sample, and performing feature fusion on the first mixed sample and the second mixed sample to obtain a mixed sample; Inputting the mixed sample into a classification head to obtain a fair adapter loss function, and adjusting the parameters of the fair adapter network model to be trained based on minimization of the fair adapter loss function to obtain a trained fair adapter network model; The enhanced semantics are input into the classification network model to be trained to obtain the image classification semantics, the image classification semantics are input into the classification head to obtain the classification loss function, the classification network model to be trained is adjusted based on minimization of the classification loss function to obtain the trained classification network model, and the classification result is generated.

[0006] Optionally, the natural semantics consists of multiple natural image semantics, the original image semantics consists of natural original image semantics and generated original image semantics, and the image classification semantics consists of natural image classification semantics and generated image classification semantics.

[0007] Optionally, the process of inputting the original image semantics into the fair adapter network model to be trained to perform residual fusion to obtain enhanced semantics includes: ; in, Represents the generation of enhanced semantics, Represents natural enhancement semantics, represents the fair adapter network model to be trained, Indicates the semantics of generating the original image, Represents natural original image semantics.

[0008] Optionally, the mathematical expressions of natural semantics and original image semantics are: ; ; in, Represents natural semantics, Represents the original image semantics, Represents the semantics of the natural original image after being processed by the image encoder, represents the semantics of natural images after being processed by the encoder, Represents the semantics of the original image generated after being processed by the image encoder. Indicates the generated image, represents the real image, Indicates that the image encoder performs encoding operations. Indicates A natural image.

[0009] Optionally, the fair adapter loss function is used to adjust the prediction value of mixed samples from AI-generated images and natural images in each image category, where The mathematical expression of the fair adapter loss function is: ; ; Among them, It is a mixed sample The classification head, are the labels of real and generated images, Indicates the The fair adapter loss function corresponding to the image categories is represents the cross entropy calculation operation, represents the fair adapter loss function, Indicates the number of image categories, Indicates the parameters to be adjusted. .

[0010] The mathematical expression of the parameter adjustment is: ; in, The previously calculated loss value for this category.

[0011] Optionally, the classification loss function is used to learn the classification features of a pair of positive and negative samples processed by the classification adapter network, wherein the mathematical expression of the classification loss function is: ; in, Represents the semantics of generating image classification The classification head, represents the classification loss function, represents the cross entropy calculation operation, are the labels of real images and generated images.

[0012] In a second aspect, the present invention provides an artificial intelligence generated image fairness detection system, comprising: A fairness auxiliary module, used for performing residual fusion on the original image semantics to obtain enhanced semantics, and combining the enhanced semantics with natural semantics to generate a mixed sample, wherein the mixed sample consists of enhanced semantics of a single category and original image semantics from other categories; A classification module is used to extract image classification semantics from the enhanced semantics, and classify AI-generated images and natural images based on the image classification semantics to obtain classification results.

[0013] The beneficial effect of an artificial intelligence generated image fairness detection system provided by the present invention is that the present invention proposes two new modules, specifically including a fairness auxiliary module and a classification module. The fairness of the model is improved by the fairness auxiliary module and the parameters of the fairness adapter network are updated; the classification module is used to detect whether the image is generated, and images with poor detection effects are retrained, thereby improving the fairness of image detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of an artificial intelligence generated image fairness detection method provided by the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0016] The embodiment of the present invention provides a method for detecting fairness of an image generated by artificial intelligence, comprising the following steps: S1, select multiple natural images and input them into the pre-trained image encoder for encoding to obtain natural semantics; S2, combining the AI-generated image and the natural image and inputting them into the pre-trained image encoder for encoding to obtain the original image semantics, and inputting the original image semantics into the fair adapter network model to be trained for residual fusion to obtain enhanced semantics, wherein the enhanced semantics include: natural enhanced semantics and generated enhanced semantics; S3, fusing natural semantics with generated enhanced semantics to generate a first mixed sample, fusing natural semantics with natural enhanced semantics to generate a second mixed sample, and performing feature fusion on the first mixed sample and the second mixed sample to obtain a mixed sample; S4, inputting the mixed sample into the classification head to obtain a fair adapter loss function, and adjusting the parameters of the fair adapter network model to be trained based on minimization of the fair adapter loss function to obtain a trained fair adapter network model; S5. Input the enhanced semantics into the classification network model to be trained to obtain image classification semantics, input the image classification semantics into the classification head to obtain the classification loss function, adjust the parameters of the classification network model to be trained based on minimization of the classification loss function to obtain the trained classification network model, and generate a classification result.

[0017] In some embodiments, when executing step S1, the natural semantics consists of multiple natural image semantics, the original image semantics consists of natural original image semantics and generated original image semantics, and the image classification semantics consists of natural image classification semantics and generated image classification semantics.

[0018] Specifically, the mathematical expressions of natural semantics and original image semantics are: ; ; in, Represents natural semantics, Represents the original image semantics, Represents the semantics of the natural original image after being processed by the image encoder, represents the semantics of natural images after being processed by the encoder, Represents the semantics of the original image generated after being processed by the image encoder. Indicates the generated image, represents the real image, Indicates that the image encoder performs encoding operations. Indicates A natural image.

[0019] In some embodiments, the process of inputting the original image semantics into the fair adapter network model to be trained to perform residual fusion to obtain enhanced semantics in step S2 includes: ; in, Represents the generation of enhanced semantics, Represents natural enhancement semantics, represents the fair adapter network model to be trained, Indicates the semantics of generating the original image, Represents natural original image semantics.

[0020] In some embodiments, when executing step S4, the mathematical expression of the fair adapter loss function is: ; ; Among them, It is a mixed sample The classification head, are the labels of real and generated images, Indicates The fair adapter loss function corresponding to the image categories is represents the cross entropy calculation operation, represents the fair adapter loss function, Indicates the number of image categories, Indicates the parameters to be adjusted. ; The mathematical expression of the parameter adjustment is: ; in, The previously calculated loss value for this category.

[0021] In some embodiments, when executing step S5, the classification loss function is used to learn the classification features of a pair of positive and negative samples processed by the classification adapter network, wherein the mathematical expression of the classification loss function is: ; in, Represents the semantics of generating image classification The classification head, represents the classification loss function, represents the cross entropy calculation operation, are the labels of real images and generated images.

[0022] The embodiment of the present invention also provides an artificial intelligence generated image fairness detection system, including: A fairness auxiliary module, used for performing residual fusion on the original image semantics to obtain enhanced semantics, and combining the enhanced semantics with natural semantics to generate a mixed sample, wherein the mixed sample consists of enhanced semantics of a single category and original image semantics from other categories; A classification module is used to extract image classification semantics from the enhanced semantics, and classify AI-generated images and natural images based on the image classification semantics to obtain classification results.

[0023] Specifically, S natural images of different types are , , a single AI-generated image and a single natural image are input into the image encoder for encoding to obtain natural semantics and original image semantics respectively, the original image semantics are input into the fair adapter network model for residual processing to obtain enhanced semantics, the enhanced semantics are fused and spliced ​​with the natural semantics to obtain mixed samples, the mixed samples are input into the head to obtain the fair adapter loss function, and the fair adapter network model is trained, then the enhanced semantics are input into the classification network model to obtain image classification semantics, the image classification semantics are input into the head to obtain the classification loss function, and the virtual image and the real image are output.

[0024] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.

Claims

1. A fairness detection method for artificial intelligence generated images, characterized in that: include: Select multiple natural images and input them into a pre-trained image encoder for encoding to obtain natural semantics; The AI-generated image and the natural image are combined and input into the pre-trained image encoder for encoding to obtain the original image semantics, and the original image semantics are input into the fair adapter network model to be trained for residual fusion to obtain enhanced semantics, wherein the enhanced semantics include: natural enhanced semantics and generated enhanced semantics; Fusing natural semantics with generated enhanced semantics to generate a first mixed sample, fusing natural semantics with natural enhanced semantics to generate a second mixed sample, and performing feature fusion on the first mixed sample and the second mixed sample to obtain a mixed sample; Inputting the mixed sample into a classification head to obtain a fair adapter loss function, and adjusting the parameters of the fair adapter network model to be trained based on minimization of the fair adapter loss function to obtain a trained fair adapter network model; The enhanced semantics are input into the classification network model to be trained to obtain the image classification semantics, the image classification semantics are input into the classification head to obtain the classification loss function, the classification network model to be trained is adjusted based on minimization of the classification loss function to obtain the trained classification network model, and the classification result is generated.

2. The method for detecting fairness of artificial intelligence generated images according to claim 1, characterized in that: The natural semantics consists of multiple natural image semantics, the original image semantics consists of natural original image semantics and generated original image semantics, and the image classification semantics consists of natural image classification semantics and generated image classification semantics.

3. The method for detecting fairness of artificial intelligence generated images according to claim 1, characterized in that: The process of inputting the original image semantics into the fair adapter network model to be trained to perform residual fusion to obtain enhanced semantics includes: ； in, Represents the generation of enhanced semantics, Represents natural enhancement semantics, represents the fair adapter network model to be trained, Indicates the semantics of generating the original image, Represents natural original image semantics.

4. The method for detecting fairness of artificial intelligence generated images according to claim 2, characterized in that: The mathematical expressions of natural semantics and original image semantics are: ； ； in, Represents natural semantics, Represents the original image semantics, Represents the semantics of the natural original image after being processed by the image encoder, represents the semantics of natural images after being processed by the encoder, Represents the semantics of the original image generated after being processed by the image encoder. Indicates the generated image, represents the real image, Indicates that the image encoder performs encoding operations. Indicates A natural image.

5. The method for detecting fairness of artificial intelligence generated images according to claim 1, characterized in that: The fair adapter loss function is used to adjust the prediction value of mixed samples from AI-generated images and natural images in each image category, where The mathematical expression of the fair adapter loss function is: ； ； Among them, It is a mixed sample The classification head, are the labels of real and generated images, Indicates The fair adapter loss function corresponding to the image categories is represents the cross entropy calculation operation, represents the fair adapter loss function, Indicates the number of image categories, Indicates the parameters to be adjusted. ; The mathematical expression of the parameter adjustment is: ； in, The previously calculated loss value for this category.

6. The method for detecting fairness of artificial intelligence generated images according to claim 1, characterized in that: The classification loss function is used to learn the classification features of a pair of positive and negative samples processed by the classification adapter network, wherein the mathematical expression of the classification loss function is: ； in, Represents the semantics of generating image classification The classification head, represents the classification loss function, represents the cross entropy calculation operation, are the labels of real images and generated images.

7. An artificial intelligence generated image fairness detection system, characterized in that: include: A fairness auxiliary module, used for performing residual fusion on the original image semantics to obtain enhanced semantics, and combining the enhanced semantics with natural semantics to generate a mixed sample, wherein the mixed sample consists of enhanced semantics of a single category and original image semantics from other categories; A classification module is used to extract image classification semantics from the enhanced semantics, and classify AI-generated images and natural images based on the image classification semantics to obtain classification results.

Images