Method and system for recognizing fraudulent images based on ethics of generative artificial intelligence

Abstract

The application is suitable for the technical field of scene modeling, and provides a fraud image recognition method and system based on generative artificial intelligence ethics, which comprises the following steps: acquiring a to-be-detected image, extracting digital watermark information embedded in the to-be-detected image, determining that the to-be-detected image is a real image if the extraction is successful and the verification is passed, and ending the process; if the extraction is not successful and the verification is passed, inputting the to-be-detected image into a trained dual-vision encoder model respectively to obtain fusion features; inputting the fusion features into a trained large language model decoder to generate and output an explainability detection result for the to-be-detected image; and constructing a hierarchical fake identification system of dual-dimension feature detection and explainable decoding to realize efficient and comprehensive identification of generative artificial intelligence fake images.

Description

Technical Field

[0001] This invention belongs to the field of scene modeling technology, and in particular relates to a method for identifying fraudulent images based on generative artificial intelligence ethics. Background Technology

[0002] With the rapid development of generative artificial intelligence technology, especially the widespread application of generative adversarial networks, variational autoencoders and diffusion models, highly realistic synthetic images and deepfake videos have emerged in large numbers on the Internet; how to identify these images has become an urgent need in the field of digital content security.

[0003] Current image recognition technologies are mainly divided into two categories: active defense and passive detection. Active defense technologies establish a source-trusted architecture by embedding digital watermarks or metadata signatures. However, this type of technology is highly dependent on the active cooperation of content creators, and it is impossible to trace images that have been widely disseminated without watermarks. At the same time, digital watermarks are easily damaged by common image processing operations such as compression and cropping, and metadata is often stripped and tampered with during dissemination, making it difficult to guarantee integrity and reliability.

[0004] Passive detection techniques identify forgery traces by analyzing the statistical features of the image itself, including detection based on low-level visual features (such as inconsistent noise distribution and abnormal frequency domain coefficients) and high-level semantic features (such as consistency of light and shadow direction and correctness of perspective relationships). However, existing methods face multiple challenges: single-dimensional detection architectures lack generalization ability and are easily circumvented by novel generative algorithms; detection models generally exhibit "black box" characteristics, only outputting probability values ​​without interpretable judgment criteria. To address these issues, existing technologies urgently need improvement. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for identifying fraudulent images based on generative artificial intelligence ethics, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for identifying fraudulent images based on generative artificial intelligence ethics, the method comprising:

[0008] Obtain the original image, perform digital watermark embedding processing on the original image, and generate an image to be distributed with watermark marking;

[0009] Obtain the image to be detected, extract the digital watermark information embedded in the image to be detected, and if the extraction and verification are successful, determine that the image to be detected is a real image and end the process;

[0010] If extraction and verification fail, the images to be detected are input into the trained dual-vision encoder model to obtain fused features.

[0011] The fused features are input into the trained large language model decoder to generate and output interpretability detection results for the image to be detected. The interpretability detection results include the authenticity judgment label of the image to be detected and the corresponding natural language description information, which is used to indicate the specific basis for determining that the image is fake.

[0012] As a further aspect of the present invention: the steps of acquiring the original image, performing digital watermark embedding processing on the original image, and generating a watermarked image to be distributed include:

[0013] The original image is acquired, and a two-dimensional discrete cosine transform is performed on the original image to obtain the frequency domain coefficient matrix;

[0014] Select the mid-frequency region coefficients from the frequency domain coefficient matrix;

[0015] The intermediate frequency coefficients are embedded using amplitude modulation based on the watermark bit sequence;

[0016] Perform an inverse discrete cosine transform on the modified frequency domain coefficient matrix to generate an image to be distributed with a watermark.

[0017] As a further aspect of the present invention: the dual-vision encoder model includes a first visual encoder and a second visual encoder;

[0018] The first visual encoder is used to extract high-level semantic features of the image to be detected, and the high-level semantic features are used to characterize whether the image content conforms to physical logic;

[0019] The second visual encoder is used to extract low-level artifact features of the image to be detected, and the low-level artifact features are used to characterize the generation traces that exist in the image in the pixel domain or frequency domain.

[0020] After obtaining high-level semantic features and low-level artifact features, the high-level semantic features and low-level artifact features are fused to obtain fused features.

[0021] As a further aspect of the present invention: the working process of the first visual encoder includes:

[0022] Perform semantic segmentation on the input image to locate object regions and their corresponding shadow regions;

[0023] Extract the principal direction vector for each shaded region;

[0024] Calculate the angle difference between all principal direction vectors;

[0025] The mean and variance of the included angle difference are calculated. When the variance exceeds a preset threshold, the variance is used as one of the elements of the advanced semantic feature.

[0026] As a further aspect of the present invention: the working process of the second visual encoder includes:

[0027] High-pass filtering is applied to the input image to obtain the residual image;

[0028] Perform a Fast Fourier Transform on the residual image to obtain its frequency domain representation;

[0029] The frequency domain representation is divided into a fixed number of frequency bands;

[0030] Calculate the energy value of each frequency band to form a frequency band energy vector;

[0031] Calculate the Euclidean distance between the frequency band energy vector and the frequency domain statistical model of the real image, and use the Euclidean distance as one element of the low-level artifact feature.

[0032] As a further aspect of the present invention: the dual-vision encoder model and the large language model decoder are obtained through joint training, specifically as follows:

[0033] Obtain a fraudulent image dataset, which includes sample images and their corresponding authenticity labels;

[0034] The parameters of the first and second visual encoders are frozen. The features extracted by the dual encoders from the sample image are fused and then input into the initial large language model decoder for supervised fine-tuning. The large language model decoder is trained to generate a natural language explanation that matches the realism label.

[0035] The direct preference optimization algorithm is used to optimize the decoder of the large language model after supervised fine-tuning, so as to improve the accuracy and naturalness of the model output.

[0036] The present invention also provides a fraud image recognition system based on generative artificial intelligence ethics, the system comprising:

[0037] The watermark image generation module is used to acquire the original image, perform digital watermark embedding processing on the original image, and generate a watermarked image to be distributed.

[0038] The watermark image detection module is used to acquire the image to be detected, extract the digital watermark information embedded in the image to be detected, and if the extraction and verification are successful, the image to be detected is determined to be a real image and the process ends.

[0039] The feature extraction module is used to input the image to be detected into the trained dual-vision encoder model to obtain fused features if the feature extraction and verification are unsuccessful.

[0040] The feature interpretation module is used to input the fused features into the trained large language model decoder, generate and output the interpretability detection result for the image to be detected; the interpretability detection result includes the authenticity judgment label of the image to be detected and the corresponding natural language description information, the natural language description information is used to indicate the specific basis for judging the image as fake.

[0041] As a further aspect of the present invention: the watermark image generation module includes:

[0042] The frequency domain transformation unit is used to acquire the original image and perform a two-dimensional discrete cosine transform on the original image to obtain the frequency domain coefficient matrix.

[0043] Regional positioning unit, used to select mid-frequency regional coefficients in the frequency domain coefficient matrix;

[0044] A watermark embedding unit is used to perform amplitude modulation embedding on the intermediate frequency coefficients according to the watermark bit sequence;

[0045] The inverse transform unit is used to perform an inverse discrete cosine transform on the modified frequency domain coefficient matrix to generate an image to be distributed with a watermark.

[0046] As a further aspect of the present invention: the dual-vision encoder model includes a first visual encoder and a second visual encoder;

[0047] The first visual encoder is used to extract high-level semantic features of the image to be detected, and the high-level semantic features are used to characterize whether the image content conforms to physical logic;

[0048] The second visual encoder is used to extract low-level artifact features of the image to be detected, and the low-level artifact features are used to characterize the generation traces that exist in the image in the pixel domain or frequency domain.

[0049] After obtaining high-level semantic features and low-level artifact features, the high-level semantic features and low-level artifact features are fused to obtain fused features.

[0050] As a further aspect of the present invention: the working process of the first visual encoder includes:

[0051] Perform semantic segmentation on the input image to locate object regions and their corresponding shadow regions;

[0052] Extract the principal direction vector for each shaded region;

[0053] Calculate the angle difference between all principal direction vectors;

[0054] The mean and variance of the included angle difference are calculated. When the variance exceeds a preset threshold, the variance is used as one of the elements of the advanced semantic feature.

[0055] Compared with the prior art, the beneficial effects of the present invention are:

[0056] This invention achieves efficient and comprehensive identification of generative AI-forged images by constructing a hierarchical authentication system with dual-dimensional feature detection and interpretable decoding. First, a digital watermark embedding and verification mechanism enables rapid release of genuine images and source-level trust management, improving system processing efficiency. When the watermark fails or is missing, advanced semantic features based on physical logical consistency and low-level artifact features based on frequency domain statistical deviation are introduced for collaborative detection. This dual approach characterizes image authenticity from both the physical constraint and statistical distribution layers, enhancing the generalization ability to different generation techniques. Feature fusion is achieved through an attention mechanism, combined with a large language model to output authenticity judgments and natural language evidence, breaking through the traditional "black box" judgment method and significantly improving the transparency and interpretability of the detection results. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.

[0058] Figure 1 This is a flowchart of a method for identifying fraudulent images based on generative artificial intelligence ethics.

[0059] Figure 2 This is a block diagram of the composition structure of a fraud image recognition system based on generative artificial intelligence ethics. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0061] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0062] Figure 1 This is a flowchart illustrating a method for identifying fraudulent images based on generative artificial intelligence ethics. In one embodiment of the technical solution of this invention, a method for identifying fraudulent images based on generative artificial intelligence ethics is provided, the method comprising:

[0063] Step S100: Obtain the original image, perform digital watermark embedding processing on the original image, and generate an image to be distributed with watermark marking;

[0064] Digital watermarking is a technique that embeds hidden information into digital media, such as images. The embedded watermark information is usually invisible to the human eye, but can be extracted and verified using specific algorithms to identify the image's origin, copyright, or integrity. This step aims to establish a trustworthy architecture from the image's source. For example, digital watermark information can be encoded as a specific binary sequence and directly appended to the image file's metadata, or it can be embedded by making minor adjustments to certain pixel values ​​of the image, making the watermark information visually imperceptible.

[0065] Step S200: Obtain the image to be detected, extract the digital watermark information embedded in the image to be detected, and if the extraction and verification are successful, determine that the image to be detected is a real image and end the process;

[0066] Step S100 generates an image with a digital watermark, and step S200 identifies the watermarked image. The system can attempt to read the expected watermark identifier from the metadata of the image to be detected and compare it with known legitimate watermarks. Alternatively, for watermarks embedded by pixel adjustment, the system can execute a reverse pixel analysis algorithm to attempt to recover the embedded watermark information and perform integrity verification.

[0067] Step S300: If extraction and verification are unsuccessful, input the images to be detected into the trained dual-vision encoder model to obtain fused features;

[0068] A dual-vision encoder model is a neural network architecture that includes two independent visual encoders. These two encoders are designed to extract features from images at different dimensions or levels to provide a more comprehensive image representation. This allows for the capture of potential forgery traces from different dimensions of the image, even if extraction and verification fail. For example, a dual-vision encoder model can consist of two independent convolutional neural networks, each trained to focus on different aspects of the image. It's important to note that "unsuccessful extraction and verification" means either extraction fails or verification fails; either condition is sufficient.

[0069] Specifically, the first visual encoder is used to extract high-level semantic features of the image to be detected, and the high-level semantic features are used to characterize whether the image content conforms to physical logic.

[0070] Specifically, the first visual encoder is a component of the dual visual encoder model, and its main function is to extract high-level semantic features from images. These features are typically related to macroscopic understanding of image content, object recognition, scene layout, and so on. For example, the first visual encoder can analyze the relative size and positional relationships between different objects in an image, or determine whether the lighting direction in an image is consistent, in order to identify whether there are logical contradictions that do not conform to common sense in the real world.

[0071] Advanced semantic features refer to the characteristics in an image that can characterize whether its content conforms to the physical laws or common sense of the real world. Examples include the consistency of light and shadow, the rationality of object perspective, and the normality of human body structure.

[0072] The second visual encoder is used to extract low-level artifact features of the image to be detected, which are used to characterize the generation traces that exist in the image in the pixel domain or frequency domain.

[0073] Specifically, the second visual encoder is another component of the dual-vision encoder model, and its main function is to extract low-level artifact features from the image. These features are typically related to the pixel-level details, texture, noise distribution, and traces that the generative model may have left behind. For example, the second visual encoder can detect whether there are anomalous noise distribution patterns in the image, or whether there are repetitive micro-textures left by a specific generative model, which are often imperceptible to the human eye.

[0074] For the recognition results of the first visual encoder and the second visual encoder, the high-level semantic features and the low-level artifact features are fused to generate fused features. Here, low-level artifact features refer to anomalous patterns or traces in the image that may be introduced by a generative artificial intelligence model and exist in the pixel domain or frequency domain. These features may include anomalous noise distribution, specific texture patterns, or frequency component anomalies. A simple concatenation operation is performed between the high-level semantic feature vector output by the first visual encoder and the low-level artifact feature vector output by the second visual encoder to form a longer feature vector.

[0075] Step S400: Input the fused features into the trained large language model decoder to generate and output the interpretability detection result for the image to be detected; the interpretability detection result includes the authenticity judgment label of the image to be detected and the corresponding natural language description information, the natural language description information is used to indicate the specific basis for judging the image as fake;

[0076] A large language model decoder is a decoder built upon a large-scale pre-trained language model. Its core function is to receive encoded feature representations and transform them into human-understandable natural language text, such as descriptions or analyses of image content. Interpretable detection results refer to results that include not only labels indicating the authenticity of an image (e.g., "real" or "fake"), but also natural language descriptive information supporting those judgments. This approach aims to improve the transparency and credibility of the detection process.

[0077] Natural language description information refers to text presented in the form of everyday human language, used to explain in detail the specific reasons or basis for determining that an image is forged. For example, it can point out physical inconsistencies or specific traces of image creation.

[0078] Regarding step S100, the steps of acquiring the original image, performing digital watermark embedding processing on the original image, and generating a watermarked image to be distributed include:

[0079] The original image is acquired, and a two-dimensional discrete cosine transform is performed on the original image to obtain the frequency domain coefficient matrix;

[0080] Select the mid-frequency region coefficients from the frequency domain coefficient matrix;

[0081] The intermediate frequency coefficients are embedded using amplitude modulation based on the watermark bit sequence;

[0082] Perform an inverse discrete cosine transform on the modified frequency domain coefficient matrix to generate an image to be distributed with a watermark.

[0083] For the digital watermark insertion process, the present invention provides a feasible technical solution. For the acquired original image, a two-dimensional discrete cosine transform is performed on the original image to obtain a frequency domain coefficient matrix, which is actually a frequency domain transformation. The mid-frequency region coefficients in the frequency domain coefficient matrix are selected, which belong to the main content of the image. Then, a digital watermark is generated and inserted into the frequency domain image. Specifically, the mid-frequency coefficients are amplitude-modulated and embedded according to the watermark bit sequence. After the operation is completed, an inverse discrete cosine transform is performed on the modified frequency domain coefficient matrix to generate an image to be distributed with a watermark.

[0084] Regarding step S300, the dual-vision encoder model includes a first vision encoder and a second vision encoder;

[0085] The first visual encoder is used to extract high-level semantic features of the image to be detected, and the high-level semantic features are used to characterize whether the image content conforms to physical logic;

[0086] The second visual encoder is used to extract low-level artifact features of the image to be detected, and the low-level artifact features are used to characterize the generation traces that exist in the image in the pixel domain or frequency domain.

[0087] After obtaining high-level semantic features and low-level artifact features, the high-level semantic features and low-level artifact features are fused to obtain fused features.

[0088] The above describes the working process of the first and second visual encoders. The first visual encoder is constructed based on the visual encoder part of the CLIP model; the second visual encoder is constructed based on a neural network used to extract high-frequency noise residuals and texture anomaly regions. In one example of the technical solution of this invention, the first visual encoder is constructed based on the visual encoder part of the CLIP model. The CLIP model is a multimodal model pre-trained on large-scale image-text pair data through contrastive learning. Its visual encoder part is specifically used to extract feature representations with rich semantic information from images. These features perform well in tasks such as image classification and object detection, and can effectively capture the semantic relationship between image content and text description. This visual encoder can be pre-trained on large-scale image-text pair datasets using contrastive learning based on the training idea of ​​CLIP, using other advanced visual backbone networks (such as ResNet, EfficientNet, etc.), thereby constructing a visual encoder with similar semantic understanding capabilities.

[0089] The second visual encoder is built upon a neural network designed to extract high-frequency noise residuals and texture anomalies. This network is specifically designed to capture subtle, imperceptible artifacts in images, typically manifesting as high-frequency noise residuals, unnatural texture patterns, or statistical anomalies. These artifacts are the "fingerprints" left by generative models when synthesizing images. This neural network can be constructed as a residual-learning-based convolutional neural network, which learns the residual between the original and denoised images to highlight high-frequency noise components. For example, an SRM filter bank can be used as the initial layer to enhance sensitivity to noise patterns, combined with a deep residual network structure for feature extraction. Alternatively, a neural network incorporating frequency domain analysis can be designed, such as converting the image to the frequency domain using discrete cosine transform or wavelet transform, and then using convolutional layers to operate on the frequency domain coefficients to identify anomalous frequency distributions or periodic texture patterns. Furthermore, an attention architecture can be introduced to make the network focus more on local regions in the image where artifacts may exist.

[0090] This invention, through the specific construction of a first and second visual encoder during the multi-dimensional feature extraction process, enables the fraudulent image identification method to more accurately and comprehensively capture clues to the authenticity of images. Specifically, the first visual encoder is constructed based on the visual encoder part of the CLIP model, the core of which lies in utilizing the powerful semantic understanding capabilities learned by the CLIP model in large-scale image-text alignment pre-training. When the image to be detected is input into this encoder, it can extract high-level semantic features. These features not only contain the category information of objects in the image, but more importantly, they can deeply understand the physical logic of the image content, such as the projection direction of light and shadow, the perspective relationship between objects, and the rationality of human body structure. This construction method ensures that the extracted high-level semantic features can reliably characterize whether the image content conforms to the physical laws of the real world, thereby effectively identifying forged images that are semantically contradictory or unnatural. At the same time, the second visual encoder is constructed based on a neural network for extracting high-frequency noise residuals and texture anomaly regions. Its design goal is to focus on capturing subtle, low-level artifact traces left by generative models when synthesizing images. When the image to be detected is processed by this encoder, it can sensitively detect and extract abnormal patterns in the pixel domain or frequency domain, such as noise distribution specific to the generation model, unnatural texture repetition, or inconsistencies in noise levels between different regions. This specialized neural network structure allows it to penetrate the visual content of the image surface and reach the "fingerprints" left by the generation process, thereby compensating for the generation artifact problem that general feature extraction models may ignore. Through this dual-encoder collaborative approach, the method of this invention can perform a comprehensive and in-depth analysis of the image to be detected that is without a watermark or has an ineffective watermark after the watermark verification process fails to identify the real image. The first visual encoder judges the logical rationality of the image from a macroscopic semantic level, while the second visual encoder captures generation traces from a microscopic pixel level. The features extracted by the two are fused during the feature fusion process to form a comprehensive feature representation that contains both high-level semantic information and low-level artifact information. This multi-dimensional and complementary feature extraction architecture significantly enhances the model's ability to identify various fraudulent images, enabling it to effectively cope with different generation techniques and forgery methods. It improves the accuracy and generalization ability of fraudulent image recognition, provides high-quality input for subsequent interpretable decoding and output processes, and thus generates more reliable and convincing detection results.

[0091] As a preferred embodiment of the present invention, the combination and fusion process of the first visual encoder and the second visual encoder can specifically be as follows: the high-level semantic features and the low-level artifact features are interacted and fused through a channel attention architecture and / or a cross attention architecture.

[0092] In one example of the technical solution of this invention, the channel attention architecture is an architecture used in deep learning models to enable the model to adaptively adjust the importance of different feature channels. It learns the weights of each channel, allowing the model to focus more on feature channels that are more important to the current task while suppressing less important channels. As one possible implementation, the channel attention architecture can compress the spatial information of each channel using global average pooling, then learn the dependencies between channels using a small neural network (e.g., composed of fully connected or convolutional layers) and generate the weights for each channel, finally multiplying these weights back into the original feature map. Alternatively, a combination of global max pooling and global average pooling can be used to capture different types of channel statistics, and then channel weights can be generated using a shared multilayer perceptron.

[0093] Cross-attention architecture is an architecture that allows a model to "attention" one feature to another when processing input features from two different types or sources, thereby capturing the interdependencies and correlations between them. It generates attention weights by calculating the similarity between queries, keys, and values, and applies these weights to the values ​​to achieve information interaction and fusion. For example, high-level semantic features can be used as queries, and low-level artifact features as keys and values, or vice versa, by calculating attention scores for weighted fusion. Specifically, high-level semantic features can be transformed into queries through linear transformations, and low-level artifact features into keys and values. Then, the dot product of the query and key is calculated, normalized by softmax, and multiplied by the value. As an alternative implementation, a multi-head attention architecture can be used, where multiple attention heads run in parallel, each learning different attention weights independently, and their outputs are then concatenated to capture richer interactive information.

[0094] This invention optimizes the fusion process of high-level semantic features and low-level artifact features by introducing a channel attention architecture and / or a cross-attention architecture. The dual-vision encoder model has already extracted high-level semantic features representing the physical logic of image content and low-level artifact features representing image generation traces, respectively. During feature fusion, the channel attention architecture adaptively assigns weights to each feature channel, allowing the model to focus on information channels that are more discriminative for fraudulent image recognition, thereby enhancing the representational power of the fused features. Simultaneously, the cross-attention architecture promotes deep interaction between high-level semantic features and low-level artifact features, enabling the model to effectively capture the potential correlation between these two different dimensional features. For example, when an image has semantic inconsistencies, its low-level artifact features may exhibit specific generation traces. This combination of architectures ensures that the fused features comprehensively and robustly reflect the authenticity clues of the image, providing high-quality input for subsequent large language model decoders to generate accurate and interpretable detection results, thus effectively solving the problem of insufficient feature interaction in traditional feature fusion.

[0095] As a preferred embodiment of the technical solution of the present invention, the working process of the first visual encoder includes:

[0096] Perform semantic segmentation on the input image to locate object regions and their corresponding shadow regions;

[0097] Extract the principal direction vector for each shaded region;

[0098] Calculate the angle difference between all principal direction vectors;

[0099] The mean and variance of the included angle difference are calculated. When the variance exceeds a preset threshold, the variance is used as one of the elements of the advanced semantic feature.

[0100] In one example of the technical solution of this invention, the elements in the high-level semantic features are limited, specifically to the direction of the shadow region. A principal direction vector is extracted for each shadow region, and the angle difference between all principal direction vectors is calculated. The mean and variance of the angle differences are statistically analyzed. When the variance exceeds a preset threshold, the variance is used as one element of the high-level semantic features. This essentially inserts a parameter generated by an algorithm into the high-level semantic features to ensure physical consistency. In the real physical world, if a scene has only one main light source, the shadow directions of all objects should be approximately the same. If the image conforms to the physical laws of a single light source, the angle difference between any two directions will be very close to 0. Therefore, the smaller the dispersion of the set of angle differences, the more consistent the shadow directions, and correspondingly, the higher the physical consistency. "Variance" is a standard indicator for measuring the dispersion of a set of data. If the variance is close to 0, it indicates a high concentration of directions; if the variance is large, it indicates that the shadow directions are divergent, with multiple inconsistent light source directions or geometric anomalies. When generative models synthesize images, shadows are often generated independently from different regions, making it difficult to strictly maintain a unified light source constraint, thus easily leading to inconsistent local shadow directions.

[0101] Specifically, the working process of the second visual encoder includes:

[0102] High-pass filtering is applied to the input image to obtain the residual image;

[0103] Perform a Fast Fourier Transform on the residual image to obtain its frequency domain representation;

[0104] The frequency domain representation is divided into a fixed number of frequency bands;

[0105] Calculate the energy value of each frequency band to form a frequency band energy vector;

[0106] Calculate the Euclidean distance between the frequency band energy vector and the frequency domain statistical model of the real image, and use the Euclidean distance as one element of the low-level artifact feature.

[0107] Similar to the above, a parameter, specifically Euclidean distance, is also inserted into the low-level artifact features. A high-pass filter is performed on the input image to obtain a residual image. A fast Fourier transform is then performed on the residual image to obtain a frequency domain representation. This frequency domain representation is divided into a fixed number of frequency bands, and the energy value of each frequency band is calculated to form a frequency band energy vector. The Euclidean distance between the frequency band energy vector and the frequency domain statistical model of the real image is calculated to obtain the required parameter. The frequency domain distribution of the real image has a stable statistical law. The spectral energy of a natural image usually follows an approximate 1 / f decay law, that is, low-frequency energy is high, and high-frequency energy decreases with frequency. If the frequency domain is divided into fixed frequency bands and the energy of each band is calculated, a spectral energy vector is obtained. If a statistical model (mean vector) is established in advance based on a large number of real images, then the spectral vector of the real image should fluctuate around the mean vector. However, generated model images often have problems such as over-smoothing at high frequencies, abnormal enhancement of specific frequency bands, or periodic texture residues, all of which will cause the frequency band energy vector to deviate from the real distribution. Based on this, the significance of Euclidean distance is to measure the overall offset between the current image spectral vector and the real statistical mean. A small distance indicates that it is close to the real frequency domain statistical distribution, while a large distance indicates that it deviates from the real statistical characteristics. At this time, it can be determined that there are generation traces. Therefore, Euclidean distance is the most basic, stable, and interpretable difference measure in multidimensional space. It is equivalent to whether the current sample falls near the center of the real image frequency domain distribution. Therefore, Euclidean distance actually represents the degree of deviation of the frequency domain distribution, and can also be understood as an indicator of the intensity of generation traces.

[0108] In summary, the shadow direction variance measures the consistency and stability of physical constraints; the spectral Euclidean distance measures the deviation of the statistical distribution.

[0109] As a preferred embodiment of the technical solution of the present invention, the dual-visual encoder model and the large language model decoder are obtained through joint training, and the specific process is as follows:

[0110] Obtain a fraudulent image dataset, which includes sample images and their corresponding authenticity labels;

[0111] A fraudulent image dataset is a collection of images used to train and evaluate model performance. It contains both real images and forged images processed using generative artificial intelligence techniques. A sample image refers to each image in the dataset, while a authenticity label is a binary label indicating whether each sample image is real or forged, or more specifically, annotation information indicating the type and region of forgery. This dataset can be obtained through various means. For example, it can be selected and organized from publicly available benchmark datasets, such as FaceForensics++ and Celeb-DF, which typically contain a large number of real video frames and video frames processed using different forgery techniques. Alternatively, a large number of forged images can be synthesized using generative models (such as GANs and DiffusionModels), and these images can be labeled with authenticity using manual annotation or semi-supervised learning methods. Furthermore, images can be manually identified and labeled by crowdsourcing platforms or professional teams to construct a high-quality fraudulent image dataset. This step provides the necessary data foundation for subsequent model training, ensuring that the model can learn the key features that distinguish real from forged images.

[0112] The parameters of the first and second visual encoders are frozen. The features extracted by the dual encoders from the sample image are fused and then input into the initial large language model decoder for supervised fine-tuning. The large language model decoder is trained to generate a natural language explanation that matches the realism label.

[0113] Freezing the parameters of the first and second visual encoders during training means keeping the weights and biases of the first and second visual encoders unchanged and not updating gradients during the training of the large language model decoder. This is done to preserve the powerful feature extraction capabilities already learned by the visual encoders and avoid interfering with or damaging these converged feature extraction capabilities during decoder fine-tuning. The "initial large language model decoder" refers to a pre-trained model with certain language understanding and generation capabilities, such as the GPT, BERT, or Llama series models based on the Transformer architecture. "Supervised fine-tuning" is a transfer learning technique that further trains the pre-trained model to adapt it to a specific task using labeled data with explicit input-output pairs (i.e., natural language explanations corresponding to fused features and realism labels). In this step, the goal of supervised fine-tuning is to enable the large language model decoder to generate logical and readable natural language explanations that match the image's realism labels based on the input image's fused features. For example, if an image is labeled as fake, the decoder should be able to generate an explanation such as "This image has inconsistent lighting in region XX, which may indicate it is fake."

[0114] The direct preference optimization algorithm is used to optimize the decoder of the large language model after supervised fine-tuning, so as to improve the accuracy and naturalness of the model output.

[0115] Direct preference optimization (RLHF) is a reinforcement learning method based on human feedback, aiming to optimize the generation quality of a model by directly learning human preferences for its output. Unlike traditional supervised learning, it does not rely on precise labels but instead uses ranking or rating data of different model outputs from humans to train a reward model, or directly optimizes the policy model through preference data. For example, RLHF simplifies the training process by transforming preference data into a simple classification problem, avoiding explicit training of the reward model. Another implementation involves using reinforcement learning algorithms such as proximal policy optimization, combined with a pre-trained reward model, to optimize the generation policy of a large language model decoder by maximizing the reward. The goal of this step is to further improve the accuracy (i.e., the degree to which the explanation matches the real-world image) and naturalness (i.e., the fluency, readability, and conformity to human cognitive habits) of the explanations generated by the large language model decoder, making its output explanations more persuasive and practical.

[0116] As a preferred embodiment of the present invention, the digital watermark embedding process specifically includes: embedding the generated invisible digital watermark into the frequency domain coefficients or spatial domain pixels of the original image.

[0117] In this embodiment, digital watermarking embedding refers to the technical process of embedding specific digital information (i.e., digital watermark) into digital media (such as images) in a covert manner. Its purpose is to achieve functions such as copyright protection, content authentication, and tamper detection. This process typically needs to balance the watermark's imperceptibility (not affecting the visual quality of the original image), robustness (resisting various image processing operations without being destroyed), and capacity (the amount of information that can be embedded). Digital watermarking embedding can employ various algorithms and techniques, such as transform domain-based methods (e.g., Discrete Cosine Transform (DCT), Discrete Wavelet Transform (DWT)) or spatial domain-based methods (e.g., Least Significant Bit (LSB) replacement). The generated invisible digital watermark is one that, after being embedded in the original image, is difficult for the human eye to detect. This watermark visually blends with the original image without causing significant visual distortion, thus maintaining the original perceptual quality of the image. Invisibility is a crucial characteristic of digital watermarking technology, ensuring that the presence of the watermark does not interfere with the normal use and dissemination of the image.

[0118] Embedding a digital watermark into the frequency domain coefficients of the original image refers to modifying specific frequency coefficients in the transform domain after performing a frequency transformation (such as discrete cosine transform, discrete wavelet transform, or Fourier transform) on the original image, thereby embedding the digital watermark information. Frequency domain coefficients represent the energy distribution of an image at different frequencies. Modifying these coefficients can disperse the watermark information throughout the image, thus enhancing the watermark's robustness against spatial domain attacks such as cropping and compression. For example, the intermediate frequency coefficients of the image can be modified because these coefficients contain enough energy to carry the watermark information without excessively affecting the image's visual quality.

[0119] Embedding a digital watermark into the spatial pixels of the original image refers to directly modifying the pixel values ​​of the original image to embed the digital watermark information. This method is generally simple to operate, but requires fine-grained control of the modification magnitude to ensure the watermark's invisibility. For example, the watermark can be embedded by modifying the least significant bit (LSB) of the pixel, as changes to the LSB are usually imperceptible to the human eye. Spatial domain embedding has the advantage of simplicity, but its robustness is generally lower than that of frequency domain embedding, and it is more sensitive to geometric transformations and lossy compression of the image.

[0120] As a preferred embodiment of the present invention, the physical logic includes one or more of the following: consistency of light and shadow direction, geometric rationality of object projection, normality of human anatomical structure, or correctness of perspective relationship between multiple objects.

[0121] In this embodiment, consistent lighting and shadow direction means that the lighting and shadow directions of different objects or different parts of the same object in an image should conform to a unified light source assumption. This can be achieved by analyzing the relationship between the brightness gradient of the object's surface and the position of the light source. For example, a physically based inverse lighting estimation model can be used to infer the direction of the light source in the image and check whether all shadows are consistent with that light source direction.

[0122] The geometric rationality of object projection refers to the fact that the shape, size, and position of objects in an image should conform to geometric principles under perspective projection. Specifically, computer vision technology can be used to detect whether the vanishing point of objects in an image conforms to the principles of perspective. For example, parallel lines should converge at the same vanishing point under perspective projection. Alternatively, 3D reconstruction technology can be used to geometrically model objects in an image, and then verify whether their projection on a 2D image is reasonable and whether there are any geometric distortions or anomalies.

[0123] Normality of human anatomy refers to the fact that human body parts appearing in an image, such as limbs and facial features, should conform to normal human anatomy. This can be achieved by extracting human skeletal key points using human pose estimation models (e.g., OpenPose or AlphaPose models based on key point detection), and then analyzing whether bone length ratios, joint angles, and limb connection relationships conform to normal human anatomy.

[0124] The correctness of perspective relationships between multiple objects refers to the fact that the relative positions and sizes of multiple objects in an image should conform to a unified perspective principle. This can be determined by analyzing whether the converging lines of the edges of different objects in the image intersect at the same vanishing point to ensure that they are in the same perspective space; alternatively, a depth estimation network can be used to determine whether the relative depth relationships between different objects conform to the perspective principle. For example, objects that are farther away should appear smaller, and their positions in the image should match the depth information.

[0125] As a preferred embodiment of the present invention, the low-level artifact features include one or more of the following: abnormal coefficient distribution in the image frequency domain, fixed texture patterns left over from the generation model, or inconsistencies in noise levels between different regions of the image.

[0126] In this embodiment, the anomalous coefficient distribution in the image frequency domain refers to a pattern in which the frequency components of an image exhibit significantly different statistical characteristics from those of the real image after frequency domain transformations such as Fourier transform. This anomalous distribution may manifest as an excessive or insufficient number of high-frequency components, abnormal concentration or dispersion of energy within a specific frequency range, or altered correlation between coefficients. For example, generative models may not be able to fully simulate the complex statistical patterns of real images in the frequency domain when synthesizing images, resulting in unnatural artificial traces in high-frequency details. This anomalous distribution can be detected by analyzing the statistical histogram of frequency domain coefficients, energy spectral density, or by using a specially designed filter bank.

[0127] Residual texture patterns from generative models refer to repetitive or regular microstructures or textures unintentionally introduced by a specific generative AI model during image generation. These patterns are often byproducts of model architecture, training data bias, or optimization algorithm defects, and are statistically detectable. For example, some generative models may produce periodic grid-like artifacts in smooth regions of generated images. Identifying these residual texture patterns can be achieved through texture analysis algorithms, convolutional kernel response analysis in deep learning, or by using Fourier transforms to detect periodic signals.

[0128] Inconsistent noise levels across different regions of an image refer to significant differences in noise characteristics between different local areas within the same image. Real-world images typically have a relatively uniform noise distribution, while generative models, during image synthesis, may exhibit mismatched noise levels or noise fingerprints in different parts of the image due to their generation architecture, stitching operations, or post-processing steps. For example, one region might exhibit Gaussian noise characteristics, while another region might exhibit Poisson noise. Detecting this inconsistency can be achieved by estimating local noise in the image, analyzing noise components at different scales using wavelet decomposition, or extracting noise residuals through high-pass filtering and calculating their local statistics.

[0129] Figure 2 The diagram shows the structural composition of a fraud image recognition system based on generative artificial intelligence ethics. As a preferred embodiment of the technical solution of this invention, the technical solution also provides a fraud image recognition system based on generative artificial intelligence ethics, wherein the system 10 includes:

[0130] The watermark image generation module 11 is used to acquire the original image, perform digital watermark embedding processing on the original image, and generate a watermarked image to be distributed.

[0131] The watermark image detection module 12 is used to acquire the image to be detected, extract the digital watermark information embedded in the image to be detected, and if the extraction and verification are successful, the image to be detected is determined to be a real image and the process ends.

[0132] Feature extraction module 13 is used to input the image to be detected into the trained dual-vision encoder model to obtain fused features if the feature extraction and verification are unsuccessful.

[0133] The feature interpretation module 14 is used to input the fused features into the trained large language model decoder to generate and output the interpretability detection result for the image to be detected; the interpretability detection result includes the authenticity judgment label of the image to be detected and the corresponding natural language description information, the natural language description information is used to indicate the specific basis for judging the image as fake.

[0134] Furthermore, the watermark image generation module 11 includes:

[0135] The frequency domain transformation unit is used to acquire the original image and perform a two-dimensional discrete cosine transform on the original image to obtain the frequency domain coefficient matrix.

[0136] Regional positioning unit, used to select mid-frequency regional coefficients in the frequency domain coefficient matrix;

[0137] A watermark embedding unit is used to perform amplitude modulation embedding on the intermediate frequency coefficients according to the watermark bit sequence;

[0138] The inverse transform unit is used to perform an inverse discrete cosine transform on the modified frequency domain coefficient matrix to generate an image to be distributed with a watermark.

[0139] Specifically, the dual-vision encoder model includes a first visual encoder and a second visual encoder;

[0140] The first visual encoder is used to extract high-level semantic features of the image to be detected, and the high-level semantic features are used to characterize whether the image content conforms to physical logic;

[0141] The second visual encoder is used to extract low-level artifact features of the image to be detected, and the low-level artifact features are used to characterize the generation traces that exist in the image in the pixel domain or frequency domain.

[0142] After obtaining high-level semantic features and low-level artifact features, the high-level semantic features and low-level artifact features are fused to obtain fused features.

[0143] Furthermore, the working process of the first visual encoder includes:

[0144] Perform semantic segmentation on the input image to locate object regions and their corresponding shadow regions;

[0145] Extract the principal direction vector for each shaded region;

[0146] Calculate the angle difference between all principal direction vectors;

[0147] The mean and variance of the included angle difference are calculated. When the variance exceeds a preset threshold, the variance is used as one of the elements of the advanced semantic feature.

[0148] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying fraudulent images based on generative artificial intelligence ethics, characterized in that, The method includes: Obtain the original image, perform digital watermark embedding processing on the original image, and generate an image to be distributed with watermark marking; Obtain the image to be detected, extract the digital watermark information embedded in the image to be detected, and if the extraction and verification are successful, determine that the image to be detected is a real image and end the process; If extraction and verification fail, the images to be detected are input into the trained dual-vision encoder model to obtain fused features. The fused features are input into the trained large language model decoder to generate and output interpretability detection results for the image to be detected. The interpretability detection results include the authenticity judgment label of the image to be detected and the corresponding natural language description information, which is used to indicate the specific basis for determining that the image is fake.

2. The method for identifying fraudulent images based on generative artificial intelligence ethics according to claim 1, characterized in that, The steps of acquiring the original image, performing digital watermark embedding processing on the original image, and generating a watermarked image to be distributed include: The original image is acquired, and a two-dimensional discrete cosine transform is performed on the original image to obtain the frequency domain coefficient matrix; Select the mid-frequency region coefficients from the frequency domain coefficient matrix; The intermediate frequency coefficients are embedded using amplitude modulation based on the watermark bit sequence; Perform an inverse discrete cosine transform on the modified frequency domain coefficient matrix to generate an image to be distributed with a watermark.

3. The method for identifying fraudulent images based on generative artificial intelligence ethics according to claim 1, characterized in that, The dual-vision encoder model includes a first vision encoder and a second vision encoder; The first visual encoder is used to extract high-level semantic features of the image to be detected, and the high-level semantic features are used to characterize whether the image content conforms to physical logic; The second visual encoder is used to extract low-level artifact features of the image to be detected, and the low-level artifact features are used to characterize the generation traces that exist in the image in the pixel domain or frequency domain. After obtaining high-level semantic features and low-level artifact features, the high-level semantic features and low-level artifact features are fused to obtain fused features.

4. The method for identifying fraudulent images based on generative artificial intelligence ethics according to claim 3, characterized in that, The working process of the first visual encoder includes: Perform semantic segmentation on the input image to locate object regions and their corresponding shadow regions; Extract the principal direction vector for each shaded region; Calculate the angle difference between all principal direction vectors; The mean and variance of the included angle difference are calculated. When the variance exceeds a preset threshold, the variance is used as one of the elements of the advanced semantic feature.

5. The method for identifying fraudulent images based on generative artificial intelligence ethics according to claim 1, characterized in that, The working process of the second visual encoder includes: High-pass filtering is applied to the input image to obtain the residual image; Perform a Fast Fourier Transform on the residual image to obtain its frequency domain representation; The frequency domain representation is divided into a fixed number of frequency bands; Calculate the energy value of each frequency band to form a frequency band energy vector; Calculate the Euclidean distance between the frequency band energy vector and the frequency domain statistical model of the real image, and use the Euclidean distance as one element of the low-level artifact feature.

6. The method for identifying fraudulent images based on generative artificial intelligence ethics according to claim 1, characterized in that, The dual-vision encoder model and the large language model decoder are obtained through joint training, specifically as follows: Obtain a fraudulent image dataset, which includes sample images and their corresponding authenticity labels; The parameters of the first and second visual encoders are frozen. The features extracted by the dual encoders from the sample image are fused and then input into the initial large language model decoder for supervised fine-tuning. The large language model decoder is trained to generate a natural language explanation that matches the realism label. The direct preference optimization algorithm is used to optimize the decoder of the large language model after supervised fine-tuning, so as to improve the accuracy and naturalness of the model output.

7. A fraud image recognition system based on generative artificial intelligence ethics, characterized in that, The system includes: The watermark image generation module is used to acquire the original image, perform digital watermark embedding processing on the original image, and generate a watermarked image to be distributed. The watermark image detection module is used to acquire the image to be detected, extract the digital watermark information embedded in the image to be detected, and if the extraction and verification are successful, the image to be detected is determined to be a real image and the process ends. The feature extraction module is used to input the image to be detected into the trained dual-vision encoder model to obtain fused features if the feature extraction and verification are unsuccessful. The feature interpretation module is used to input the fused features into the trained large language model decoder, generate and output the interpretability detection result for the image to be detected; the interpretability detection result includes the authenticity judgment label of the image to be detected and the corresponding natural language description information, the natural language description information is used to indicate the specific basis for judging the image as fake.

8. The fraud image recognition system based on generative artificial intelligence ethics according to claim 7, characterized in that, The watermark image generation module includes: The frequency domain transformation unit is used to acquire the original image and perform a two-dimensional discrete cosine transform on the original image to obtain the frequency domain coefficient matrix. Regional positioning unit, used to select mid-frequency regional coefficients in the frequency domain coefficient matrix; A watermark embedding unit is used to perform amplitude modulation embedding on the intermediate frequency coefficients according to the watermark bit sequence; The inverse transform unit is used to perform an inverse discrete cosine transform on the modified frequency domain coefficient matrix to generate an image to be distributed with a watermark.

9. The fraud image recognition system based on generative artificial intelligence ethics according to claim 7, characterized in that, The dual-vision encoder model includes a first vision encoder and a second vision encoder; The first visual encoder is used to extract high-level semantic features of the image to be detected, and the high-level semantic features are used to characterize whether the image content conforms to physical logic; The second visual encoder is used to extract low-level artifact features of the image to be detected, and the low-level artifact features are used to characterize the generation traces that exist in the image in the pixel domain or frequency domain. After obtaining high-level semantic features and low-level artifact features, the high-level semantic features and low-level artifact features are fused to obtain fused features.

10. The fraud image recognition system based on generative artificial intelligence ethics according to claim 9, characterized in that, The working process of the first visual encoder includes: Perform semantic segmentation on the input image to locate object regions and their corresponding shadow regions; Extract the principal direction vector for each shaded region; Calculate the angle difference between all principal direction vectors; The mean and variance of the included angle difference are calculated. When the variance exceeds a preset threshold, the variance is used as one of the elements of the advanced semantic feature.

Images