An AIGC image detection method and system based on multi-level feature fusion

Abstract

The application provides an AIGC image detection method and system based on multi-level feature fusion in the field of artificial intelligence security technology, which comprises the following steps: S1, constructing a training data set; S2, creating an AIGC image detection model based on a frequency score calculation module, a bottom layer feature extraction module, a high layer semantic feature extraction module, a feature fusion module and an image classification and discrimination module; S3, training the AIGC image detection model through the training data set; S4, deploying the AIGC image detection model after training; S5, receiving a detection request carrying an RGB image sent by a client; S6, performing image block division processing on the RGB image to obtain an image set containing multiple image blocks, and inputting the image set into the AIGC image detection model to obtain a discrimination result. The application has the advantage that the generalization ability and robustness of the detection model when facing images synthesized by multiple unknown generation models outside the training data distribution are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security technology, and in particular to an AIGC image detection method and system based on multi-level feature fusion. Background Technology

[0002] In recent years, generative artificial intelligence (AIGC) technology, especially image generation models, has made groundbreaking progress. From early generative adversarial networks (GANs) to current large-scale diffusion models and multimodal pre-trained architectures, the image synthesis capabilities of these models have approached or even surpassed the level of some human creators. Technologies such as OpenAI's GPT-Image-1, Alibaba Cloud's Qwen-Image, and the next-generation multimodal generation system Seedance2.0 can generate highly realistic and detailed images based on text prompts, marking a new stage in AIGC image generation technology characterized by large-scale, low-barrier, and high-quality production.

[0003] Technological advancements have significantly lowered the barriers to visual content creation, revolutionizing efficiency in industries such as e-commerce, advertising, and entertainment. However, they have also triggered severe information security and trust crises. Artificial intelligence-generated fake images, with their high degree of visual deception, can easily be combined with trending events and spread rapidly on social media platforms, potentially misleading the public, impacting financial markets, and even affecting social stability. The widespread impact of the AI-generated "Pentagon on fire" image is a case in point. Research shows that when faced with high-quality AI-generated images, the accuracy rate of human visual identification is approaching random levels, highlighting the urgent need to develop automated detection tools.

[0004] Against this backdrop, AIGC image detection technology emerged, with its core objective being to automatically determine the authenticity of images by analyzing the statistical traces and structural features left behind during the image generation process. Traditional image forensics methods typically rely on analyzing image noise patterns, compression traces, or frequency domain statistical features. However, with the development of next-generation generation technologies such as diffusion models, synthetic images are continuously improving in terms of texture detail, noise distribution, and structural consistency, gradually limiting the performance of traditional detection methods that rely on single feature analysis.

[0005] Currently, most advanced AIGC image detection methods typically rely on a large amount of data generated by a specific type of generative model (such as a single diffusion model or generative adversarial network) during the training phase. While this approach achieves high accuracy on test sets with similar training data distributions, it faces significant challenges in real-world open environments. Real-world AIGC image sources are extremely diverse, and the generative models themselves are rapidly iterating and updating. When the detection model encounters images synthesized by novel models with different generation mechanisms that were not seen during training, its detection performance often drops significantly. This insufficient generalization ability due to over-reliance on a single data distribution has become a core bottleneck restricting the practical deployment and application of AIGC image detection technology.

[0006] Therefore, how to provide an AIGC image detection method and system based on multi-level feature fusion to improve the generalization ability and robustness of the detection model when facing images synthesized by multiple unknown generation models outside the training data distribution has become an urgent technical problem to be solved. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide an AIGC image detection method and system based on multi-level feature fusion, so as to improve the generalization ability and robustness of the detection model when facing images synthesized by multiple unknown generation models outside the training data distribution.

[0008] In a first aspect, the present invention provides an AIGC image detection method based on multi-level feature fusion, comprising the following steps: Step S1: Obtain AI-generated images and real-shot images from various different generation models, and construct a training dataset based on the AI-generated images and real-shot images. Step S2: Create an AIGC image detection model based on the frequency score calculation module, the low-level feature extraction module, the high-level semantic feature extraction module, the feature fusion module, and the image classification and discrimination module; Step S3: Train the AIGC image detection model using the training dataset to optimize the model parameters; during the training process, fine-tune the feature extractor of the low-level feature extraction module, freeze the parameters of the SAM3 visual encoder of the high-level semantic feature extraction module, and train only the linear projection layer of the high-level semantic feature extraction module. Step S4: Containerize and deploy the trained AIGC image detection model as a service; Step S5: Receive a detection request sent by the client, wherein the detection request carries an encoded RGB image to be detected; Step S6: Perform image block segmentation processing on the RGB image to obtain an image set containing multiple image blocks. Input the image set into the deployed AIGC image detection model to obtain the discrimination result of whether the RGB image is an artificial intelligence generated image. Return the discrimination result to the client.

[0009] Furthermore, in step S2, the frequency score calculation module is used to calculate the frequency score of each image block based on discrete cosine transform and a plurality of preset bandpass filters, and to select a preset number of high-frequency image blocks and low-frequency image blocks from the image set according to the frequency score. The low-level feature extraction module is used to extract high-frequency low-level features and low-frequency low-level features from the high-frequency image block and the low-frequency image block respectively, and to fuse the high-frequency low-level features and low-frequency low-level features to obtain the fused low-level feature representation. The high-level semantic feature extraction module is used to perform overall analysis of the RGB image and extract high-level semantic features; The feature fusion module is used to concatenate the low-level feature representation with the high-level semantic features to obtain multi-level fused features; The image classification and discrimination module is used to input the multi-level fusion features into the classifier to obtain a discrimination result on whether the RGB image is an artificial intelligence generated image.

[0010] Furthermore, in the frequency score calculation module, the frequency score calculation process is specifically as follows: Take the absolute value of the DCT coefficients of the m-th image block, add 1, and then perform logarithmic transformation to obtain the logarithmic result. ; The k-th bandpass filter The intermediate result of each bandpass filter is obtained by multiplying the logarithmic result at all positions and channels and summing the results. Multiply the intermediate results of each of the bandpass filters by The frequencies of the m-th image patch are then summed to obtain the frequency score. .

[0011] Furthermore, the underlying feature extraction module is specifically used for: The selected high-frequency and low-frequency image blocks are scaled to a preset resolution, and the scaled high-frequency and low-frequency image blocks are input into the spatial enrichment model filter to extract the corresponding noise mode features. The noise pattern features are input into a feature extractor built on a ConvNeXt network to obtain the corresponding feature maps. Average pooling is performed on each of the feature maps to obtain high-frequency low-level features. and low-frequency underlying features ; For the high-frequency low-level features and low-frequency underlying features Average pooling is performed to fuse the features, resulting in the fused low-level feature representation. : .

[0012] Furthermore, the high-level semantic feature extraction module is specifically used for: The RGB image is input into the SAM3 visual encoder to obtain a high-dimensional feature representation; the high-dimensional feature representation is input into a linear projection layer for dimensional mapping to obtain projected features; the projected features are then subjected to spatial average pooling to obtain high-level semantic features. In the image classification and discrimination module, the classifier is a multilayer perceptron.

[0013] Secondly, the present invention provides an AIGC image detection system based on multi-level feature fusion, comprising the following modules: The training dataset construction module is used to acquire AI-generated images and real-world images from various different generative models, and to construct a training dataset based on the AI-generated images and real-world images. The AIGC image detection model creation module is used to create an AIGC image detection model based on the frequency score calculation module, the low-level feature extraction module, the high-level semantic feature extraction module, the feature fusion module, and the image classification and discrimination module. The AIGC image detection model training module is used to train the AIGC image detection model using the training dataset to optimize model parameters. During the training process, the feature extractor of the low-level feature extraction module is trained and fine-tuned, the parameters of the SAM3 visual encoder of the high-level semantic feature extraction module are frozen, and only the linear projection layer of the high-level semantic feature extraction module is trained. The AIGC image detection model deployment module is used to containerize and deploy the trained AIGC image detection model as a service. The detection request receiving module is used to receive a detection request sent by the client, wherein the detection request carries an encoded RGB image to be detected; The discrimination result output module is used to perform image block division processing on the RGB image to obtain an image set containing multiple image blocks, input the image set into the deployed AIGC image detection model, obtain the discrimination result of whether the RGB image is an artificial intelligence generated image, and return the discrimination result to the client.

[0014] Furthermore, in the AIGC image detection model creation module, the frequency score calculation module is used to calculate the frequency score of each image block based on discrete cosine transform and multiple preset bandpass filters, and to select a preset number of high-frequency image blocks and low-frequency image blocks from the image set according to the frequency score. The low-level feature extraction module is used to extract high-frequency low-level features and low-frequency low-level features from the high-frequency image block and the low-frequency image block respectively, and to fuse the high-frequency low-level features and low-frequency low-level features to obtain the fused low-level feature representation. The high-level semantic feature extraction module is used to perform overall analysis of the RGB image and extract high-level semantic features; The feature fusion module is used to concatenate the low-level feature representation with the high-level semantic features to obtain multi-level fused features; The image classification and discrimination module is used to input the multi-level fusion features into the classifier to obtain a discrimination result on whether the RGB image is an artificial intelligence generated image.

[0015] Furthermore, in the frequency score calculation module, the frequency score calculation process is specifically as follows: Take the absolute value of the DCT coefficients of the m-th image block, add 1, and then perform logarithmic transformation to obtain the logarithmic result. ; The k-th bandpass filter The intermediate result of each bandpass filter is obtained by multiplying the logarithmic result at all positions and channels and summing the results. Multiply the intermediate results of each of the bandpass filters by The frequencies of the m-th image patch are then summed to obtain the frequency score. .

[0016] Furthermore, the underlying feature extraction module is specifically used for: The selected high-frequency and low-frequency image blocks are scaled to a preset resolution, and the scaled high-frequency and low-frequency image blocks are input into the spatial enrichment model filter to extract the corresponding noise mode features. The noise pattern features are input into a feature extractor built on a ConvNeXt network to obtain the corresponding feature maps. Average pooling is performed on each of the feature maps to obtain high-frequency low-level features. and low-frequency underlying features ; For the high-frequency low-level features and low-frequency underlying features Average pooling is performed to fuse the features, resulting in the fused low-level feature representation. : .

[0017] Furthermore, the high-level semantic feature extraction module is specifically used for: The RGB image is input into the SAM3 visual encoder to obtain a high-dimensional feature representation; the high-dimensional feature representation is input into a linear projection layer for dimensional mapping to obtain projected features; the projected features are then subjected to spatial average pooling to obtain high-level semantic features. In the image classification and discrimination module, the classifier is a multilayer perceptron. The advantages of this invention are: 1. By training with mixed data from multiple generative models, the AIGC image detection model is forced to learn common forgery patterns across models rather than fingerprints from a single model. At the same time, a multi-level analysis architecture is constructed that integrates frequency domain scores, low-level noise patterns, and high-level semantic features extracted by SAM3. This ensures that even if an unknown model is close to reality in one aspect, traces left behind at other levels can still be captured. Furthermore, by freezing the parameters of the SAM3 visual encoder, the general visual representation learned on large-scale data is preserved, and only the projection layer is fine-tuned for task adaptation. This systematically enhances the model's generalization ability and robustness when facing unknown generative models.

[0018] 2. By acquiring AIGC images from various different generative models to construct a training dataset, the diversity of the model is enhanced from the data source. This allows the trained detection model to learn more general and essential AIGC image features rather than targeting a single generator (such as only StableDiffusion or DALL-E). As a result, it can still maintain a high detection accuracy when facing unknown or newly emerging generative models, effectively improving the model's generalization ability and robustness in practical applications.

[0019] 3. Through a frequency scoring screening mechanism, low-level features reflecting subtle differences in noise, texture, etc., are extracted from high-frequency and low-frequency image blocks respectively. At the same time, the powerful visual foundation model (SAM3) is used to extract the overall high-level semantic features of the image. Finally, the two are fused to form a multi-level fused feature. This design combines bottom-up detail analysis and top-down global understanding, enabling the model to capture the small artifacts left by AIGC in local pixel statistical regularities, and also to perceive its defects in global semantic coherence and object structure rationality, thus achieving a more comprehensive and accurate discrimination.

[0020] 4. During model training, a differentiated parameter update strategy was adopted: the low-level feature extractor was fine-tuned to adapt to the specific detection task, while the backbone parameters of the SAM3 visual encoder were frozen and only its linear projection layer was trained. This strategy makes full use of the powerful general feature representation capability of large-scale pre-trained visual models, while avoiding overfitting or catastrophic forgetting problems that may be caused by completely fine-tuning such large models. This not only ensures the quality of high-level semantic features, but also significantly reduces the number of parameters that need to be trained, improves training efficiency and stability, and enables the model to quickly converge to a high-performance solution.

[0021] 5. During the inference phase, the input image is first divided into blocks, and key high-frequency and low-frequency blocks are selected based on frequency scores for subsequent deep feature extraction, rather than performing equally complex calculations on all regions of the entire image. This "attention" mechanism allows computational resources to be focused on the contrasting regions that are most likely to contain generation traces (high-frequency detail anomalies) and least likely to contain them (low-frequency smooth regions). Without losing detection accuracy or even improving it (due to the removal of irrelevant information interference), the computational cost required for a single detection is significantly reduced, making the scheme more feasible for application in real-time or high-concurrency scenarios.

[0022] 6. By systematically analyzing the feature changes produced by AI-generated images at the visual level, the rationality of AI-generated image detection tasks was re-verified, and unreasonable assumptions in the training and testing settings of existing studies were re-examined and improved. Traditional methods often assume that training and testing data come from the same or similar generative model distributions. This assumption is difficult to hold in practical applications. Therefore, this invention introduces cross-model data sources to improve the generalization ability of the detection model on unknown generative models. In terms of model architecture, a simple and efficient detector is designed based on the AIDE method. By fusing low-level image patch statistical features and high-level semantic representations, multi-level information collaborative modeling is achieved. The low-level features mainly capture statistical anomalies in texture distribution, noise patterns, and local structure of the generative model, while the high-level semantic features are used to identify inconsistencies in object structure, spatial relationships, and semantic logic. Through the dual-layer feature fusion mechanism, the model can simultaneously utilize pixel-level detail information and semantic-level contextual information, thereby significantly improving the accuracy and robustness of AI-generated image detection. Finally, while maintaining the simplicity of the model structure, effective identification of complex AIGC forged images is achieved. Attached Figure Description

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0024] Figure 1This is a flowchart of an AIGC image detection method based on multi-level feature fusion according to the present invention.

[0025] Figure 2 This is a schematic diagram of the structure of an AIGC image detection system based on multi-level feature fusion according to the present invention. Detailed Implementation

[0026] The overall approach of the technical solution in this application is as follows: Robustness is systematically improved by training with mixed data from multiple generative models and designing a multi-level feature fusion AIGC image detection model. First, the input image is divided into blocks, and key high-frequency and low-frequency image blocks are selected using frequency domain (DCT) analysis. Then, the ConvNeXt network of the AIGC image detection model extracts low-level features reflecting subtle differences in noise, texture, etc. Simultaneously, a SAM3 visual encoder with frozen parameters is used to extract high-level semantic features of the image. Then, the low-level and high-level features are fused to form complementary multi-level representations, which are ultimately judged by a classifier. By combining a data strategy oriented towards diversity, a model architecture that considers both local details and global semantics, and an efficient training method that freezes large model parameters and only fine-tunes lightweight components, more effective detection of images generated by unknown models outside the training data distribution is achieved.

[0027] Please refer to Figures 1 to 2 As shown, a preferred embodiment of the AIGC image detection method based on multi-level feature fusion of the present invention includes the following steps: Step S1: Obtain AI-generated images and real-shot images from various different generation models, and construct a training dataset based on the AI-generated images and real-shot images. To simulate the real-world application environment of AIGC image detection, three core principles were followed during the training dataset construction phase: ① Images possess high realism, effectively deceiving human vision; ② Image categories are broadly covered, including various semantic types such as people, landscapes, architecture, indoor scenes, animals, and products; ③ Overall image quality is excellent, with resolution and detail reaching mainstream commercial generation standards. Based on these principles, a training dataset that balances realism and diversity was constructed.

[0028] In terms of image generation, to obtain high-quality and diverse AI-generated images, works publicly shared by users were collected from mainstream AI painting community platforms such as ArtStation, Civitai, and Liblib. Most of the images on these platforms originate from commercial interfaces such as Midjourney and DALL-E 3, or are generated based on Stable Diffusion and various LoRA modules with fine-tuning, representing the true generation level under the current mainstream diffusion model system. Ultimately, over 15,000 high-quality AI-generated images were collected.

[0029] In terms of collecting real images, to ensure consistency between real and generated images in semantic category, composition style, and resolution distribution, a completely identical keyword system was used for reverse retrieval, obtaining high-quality real photographic works from global photographer sharing platforms such as Unsplash. Through unified screening criteria and resolution constraints, the real images were ensured to be comparable to AIGC images in terms of visual quality and scene complexity, ultimately collecting over 10,000 real-scene images.

[0030] Step S2: Create an AIGC image detection model based on the frequency score calculation module, the low-level feature extraction module, the high-level semantic feature extraction module, the feature fusion module, and the image classification and discrimination module; Step S3: Train the AIGC image detection model using the training dataset to optimize model parameters; during training, fine-tune the feature extractor of the low-level feature extraction module, freeze the parameters of the SAM3 visual encoder of the high-level semantic feature extraction module, and train only the linear projection layer of the high-level semantic feature extraction module; AdamW is used as the optimizer during model training, with 20 iterations, a batch size of 32, and a learning rate set to... ; Step S4: Containerize and deploy the trained AIGC image detection model as a service; Step S5: Receive a detection request sent by the client, wherein the detection request carries an encoded RGB image to be detected; Step S6: Perform image block segmentation processing on the RGB image to obtain an image set containing multiple image blocks. Input the image set into the deployed AIGC image detection model to obtain the discrimination result of whether the RGB image is an artificial intelligence generated image. Return the discrimination result to the client.

[0031] In practice, for an RGB image, it is first divided into multiple 32*32 window-sized image blocks using a sliding window with a step size of 16. ,Right now ,in R is used to define a tensor space, and N defines the spatial dimension of the tensor; the first two N represent the height and width of the image, respectively, and the third dimension 3 is fixed to represent the number of channels of the image. For a standard RGB color image, this means three color channels: red, green, and blue.

[0032] The trained AIGC image detection model is containerized for deployment to improve system portability, scalability, and service stability in practical applications. During system operation, the RGB image to be detected is first converted to Base64 encoding format by the client and encapsulated as a JSON data request transmitted via HTTP protocol before being sent to the server. Upon receiving the detection request, the server parses the JSON data, extracts the Base64 encoded image data, and decodes it back to a standard RGB image.

[0033] Subsequently, the decoded RGB image is input into the deployed AIGC image detection model for inference analysis. The AIGC image detection model comprehensively determines whether the image belongs to artificial intelligence-generated images based on the low-level statistical features and high-level semantic features in the RGB image, and outputs the corresponding detection score. Finally, the judgment result (including the detection score and related task-related additional information) is repackaged into JSON format and returned to the backend system via HTTP interface for visualization and result management, thereby realizing a complete and efficient online AIGC image detection service process.

[0034] In step S2, the frequency score calculation module is used to calculate the frequency score of each image block based on discrete cosine transform and multiple preset bandpass filters, and to select a preset number of high-frequency image blocks and low-frequency image blocks from the image set according to the frequency score. Images generated by generative models such as Generative Adversarial Networks (GANs) and diffusion models often exhibit artifacts such as oversmoothing and anti-aliasing effects. To capture this difference, a frequency scoring module is introduced to identify high-frequency and low-frequency image patches. By focusing on these extreme frequency patches, the unique features of AI-generated images are highlighted, thereby improving the model's discriminative ability.

[0035] The low-level feature extraction module is used to extract high-frequency low-level features and low-frequency low-level features from the high-frequency image block and the low-frequency image block respectively, and to fuse the high-frequency low-level features and low-frequency low-level features to obtain the fused low-level feature representation. The high-level semantic feature extraction module is used to perform overall analysis of the RGB image and extract high-level semantic features; The feature fusion module is used to concatenate the low-level feature representation with the high-level semantic features to obtain multi-level fused features. This fusion method can simultaneously utilize the statistical differences of images at the pixel level and the structural information at the semantic level, thereby improving the model's overall ability to characterize the features of AI-generated images. The image classification and discrimination module is used to input the multi-level fusion features into the classifier to obtain a discrimination result on whether the RGB image is an artificial intelligence generated image.

[0036] In the frequency score calculation module, the frequency score calculation process is as follows: Take the absolute value of the DCT coefficients of the m-th image block, add 1, and then perform logarithmic transformation to obtain the logarithmic result. ; The k-th bandpass filter The intermediate result of each bandpass filter is obtained by multiplying the logarithmic result at all positions and channels and summing the results. Multiply the intermediate results of each of the bandpass filters by The frequencies of the m-th image patch are then summed to obtain the frequency score. .

[0037] The mathematical definition of a bandpass filter is: ; in, This indicates the total number of bandpass filters; The index represents the bandpass filter; i and j represent the two-dimensional spatial coordinate indices of pixels within the image patch (i is the row index, j is the column index); N represents the size (width or height) of the image patch. This represents the weight of the k-th bandpass filter at position (i,j) in the image block.

[0038] The formula for calculating the frequency score is: ; Where c represents the number of channels in the image block.

[0039] The frequency scores of all image patches are obtained using the above formula. Select k high-frequency image blocks with the highest frequency scores. and the k low-frequency image patches with lower frequency scores .

[0040] The underlying feature extraction module is specifically used for: The selected high-frequency and low-frequency image blocks are scaled to a preset resolution (256×256 pixels), and the scaled high-frequency and low-frequency image blocks are input into the spatial enrichment model filter to extract the corresponding noise mode features. The noise pattern features are input into a feature extractor built on a ConvNeXt network to obtain the corresponding feature maps. Average pooling is performed on each of the feature maps to obtain high-frequency low-level features. and low-frequency underlying features ; For the high-frequency low-level features and low-frequency underlying features Average pooling is performed to fuse the features, resulting in the fused low-level feature representation. : .

[0041] Average pooling can reduce feature dimensionality while preserving key information, resulting in a more stable and representative low-level feature representation.

[0042] Compared to the traditional ResNet50 structure, this invention employs the ConvNext network as the backbone feature extractor in the feature extraction stage, constructing independent network branches. The ConvNext network used has significant advantages in this task: First, through large convolutional kernels (such as 7*7 depthwise separable convolutions) and modern network design, the ConvNext network maintains the efficiency of the convolutional structure while possessing a larger receptive field, which is beneficial for capturing subtle texture artifacts in the generated images. Second, the ConvNext network introduces structures such as LayerNorm, depthwise separable convolutions, and inverted bottlenecks, making its feature representation ability and training stability significantly superior to the traditional ResNet network, enabling it to better characterize the fine-grained features generated by the generative model. In addition, the ConvNext network achieves a good balance between parameters and computational efficiency, exhibiting stronger modeling capabilities for features such as hidden noise patterns, texture smoothing, and anti-aliasing artifacts in high-frequency and low-frequency image patches, thereby further improving the feature discrimination ability in the AIGC image detection task.

[0043] The high-level semantic feature extraction module is specifically used for: The RGB image is input into the SAM3 visual encoder to obtain a high-dimensional feature representation; the high-dimensional feature representation is input into a linear projection layer for dimensional mapping to obtain projected features; the projected features are then subjected to spatial average pooling to obtain high-level semantic features. To capture rich semantic features in images (such as object co-occurrence relationships, scene context associations, and spatial structure consistency), this invention optimizes the traditional method using OpenClip's visual encoder to use a Segment AnythingModel3 (SAM3) visual encoder to compute the visual embedding (high-dimensional feature representation) of the image. To further enhance the global semantic representation capability, a linear projection layer g(·) is cascaded after the SAM3 visual encoder to map and compress the feature dimensions of the high-dimensional feature representation, and the final high-level semantic features are obtained through spatial average pooling. The formula is: .

[0044] Compared to traditional visual encoders, the SAM3 visual encoder has significant advantages in this invention. First, as a large-scale visual foundation model, SAM3 is pre-trained on massive image data, possessing stronger general visual representation capabilities. It can effectively capture semantic structural information in complex scenes, such as spatial relationships between objects, contextual dependencies, and the overall layout of the scene. This is of great significance for identifying potential semantic inconsistencies or structural anomalies in AIGC-generated images. Second, the SAM3 visual encoder adopts a Transformer-based global modeling mechanism, which can establish long-distance dependencies within a larger receptive field, thereby more accurately modeling global contextual features in images. Traditional convolutional networks are often limited by local receptive fields in this respect. Third, the SAM3 visual encoder is designed with greater emphasis on the ability to express image structure and boundary information. It can perform more refined modeling of object contours, regional relationships, and structural continuity in images. This helps to discover potential anomalies in the object structure, perspective relationships, or semantic logic of AI-generated images.

[0045] Therefore, the SAM3 visual encoder is introduced in this invention for semantic feature extraction. This not only obtains a more discriminative global semantic embedding, but also complements the statistical features of the underlying image patches. This enables joint modeling of texture-level artifacts and semantic-level anomalies in the AIGC image detection task, further improving the accuracy and generalization ability of the detection model.

[0046] In the image classification and discrimination module, the classifier is a multilayer perceptron (MLP). The calculation formula for the classifier is: ; in, The symbol ";" represents the structure of a multilayer perceptron, which consists of a linear mapping layer, a GELU activation function, and a final classification layer; the symbol ";" represents the feature concatenation operation in the channel dimension.

[0047] Through the image classification and discrimination module, the AIGC image detection model can comprehensively utilize low-level statistical features and high-level semantic information to achieve efficient and accurate identification of AI-generated images and real images.

[0048] A preferred embodiment of the AIGC image detection system based on multi-level feature fusion of the present invention includes the following modules: The training dataset construction module is used to acquire AI-generated images and real-world images from various different generative models, and to construct a training dataset based on the AI-generated images and real-world images. To simulate the real-world application environment of AIGC image detection, three core principles were followed during the training dataset construction phase: ① Images possess high realism, effectively deceiving human vision; ② Image categories are broadly covered, including various semantic types such as people, landscapes, architecture, indoor scenes, animals, and products; ③ Overall image quality is excellent, with resolution and detail reaching mainstream commercial generation standards. Based on these principles, a training dataset that balances realism and diversity was constructed.

[0049] In terms of image generation, to obtain high-quality and diverse AI-generated images, works publicly shared by users were collected from mainstream AI painting community platforms such as ArtStation, Civitai, and Liblib. Most of the images on these platforms originate from commercial interfaces such as Midjourney and DALL-E 3, or are generated based on Stable Diffusion and various LoRA modules with fine-tuning, representing the true generation level under the current mainstream diffusion model system. Ultimately, over 15,000 high-quality AI-generated images were collected.

[0050] In terms of collecting real images, to ensure consistency between real and generated images in semantic category, composition style, and resolution distribution, a completely identical keyword system was used for reverse retrieval, obtaining high-quality real photographic works from global photographer sharing platforms such as Unsplash. Through unified screening criteria and resolution constraints, the real images were ensured to be comparable to AIGC images in terms of visual quality and scene complexity, ultimately collecting over 10,000 real-scene images.

[0051] The AIGC image detection model creation module is used to create an AIGC image detection model based on the frequency score calculation module, the low-level feature extraction module, the high-level semantic feature extraction module, the feature fusion module, and the image classification and discrimination module. The AIGC image detection model training module is used to train the AIGC image detection model using the training dataset to optimize model parameters. During training, the feature extractor of the low-level feature extraction module is fine-tuned, the parameters of the SAM3 visual encoder of the high-level semantic feature extraction module are frozen, and only the linear projection layer of the high-level semantic feature extraction module is trained. AdamW is used as the optimizer during model training, with 20 iterations, a batch size of 32, and a learning rate set to... ; The AIGC image detection model deployment module is used to containerize and deploy the trained AIGC image detection model as a service. The detection request receiving module is used to receive a detection request sent by the client, wherein the detection request carries an encoded RGB image to be detected; The discrimination result output module is used to perform image block division processing on the RGB image to obtain an image set containing multiple image blocks, input the image set into the deployed AIGC image detection model, obtain the discrimination result of whether the RGB image is an artificial intelligence generated image, and return the discrimination result to the client.

[0052] In practice, for an RGB image, it is first divided into multiple 32*32 window-sized image blocks using a sliding window with a step size of 16. ,Right now ,in R is used to define a tensor space, and N defines the spatial dimension of the tensor; the first two N represent the height and width of the image, respectively, and the third dimension 3 is fixed to represent the number of channels of the image. For a standard RGB color image, this means three color channels: red, green, and blue.

[0053] The trained AIGC image detection model is containerized for deployment to improve system portability, scalability, and service stability in practical applications. During system operation, the RGB image to be detected is first converted to Base64 encoding format by the client and encapsulated as a JSON data request transmitted via HTTP protocol before being sent to the server. Upon receiving the detection request, the server parses the JSON data, extracts the Base64 encoded image data, and decodes it back to a standard RGB image.

[0054] Subsequently, the decoded RGB image is input into the deployed AIGC image detection model for inference analysis. The AIGC image detection model comprehensively determines whether the image belongs to artificial intelligence-generated images based on the low-level statistical features and high-level semantic features in the RGB image, and outputs the corresponding detection score. Finally, the judgment result (including the detection score and related task-related additional information) is repackaged into JSON format and returned to the backend system via HTTP interface for visualization and result management, thereby realizing a complete and efficient online AIGC image detection service process.

[0055] In the AIGC image detection model creation module, the frequency score calculation module is used to calculate the frequency score of each image block based on discrete cosine transform and multiple preset bandpass filters, and to select a preset number of high-frequency image blocks and low-frequency image blocks from the image set according to the frequency score. Images generated by generative models such as Generative Adversarial Networks (GANs) and diffusion models often exhibit artifacts such as oversmoothing and anti-aliasing effects. To capture this difference, a frequency scoring module is introduced to identify high-frequency and low-frequency image patches. By focusing on these extreme frequency patches, the unique features of AI-generated images are highlighted, thereby improving the model's discriminative ability.

[0056] The low-level feature extraction module is used to extract high-frequency low-level features and low-frequency low-level features from the high-frequency image block and the low-frequency image block respectively, and to fuse the high-frequency low-level features and low-frequency low-level features to obtain the fused low-level feature representation. The high-level semantic feature extraction module is used to perform overall analysis of the RGB image and extract high-level semantic features; The feature fusion module is used to concatenate the low-level feature representation with the high-level semantic features to obtain multi-level fused features. This fusion method can simultaneously utilize the statistical differences of images at the pixel level and the structural information at the semantic level, thereby improving the model's overall ability to characterize the features of AI-generated images. The image classification and discrimination module is used to input the multi-level fusion features into the classifier to obtain a discrimination result on whether the RGB image is an artificial intelligence generated image.

[0057] In the frequency score calculation module, the frequency score calculation process is as follows: Take the absolute value of the DCT coefficients of the m-th image block, add 1, and then perform logarithmic transformation to obtain the logarithmic result. ; The k-th bandpass filter The intermediate result of each bandpass filter is obtained by multiplying the logarithmic result at all positions and channels and summing the results. Multiply the intermediate results of each of the bandpass filters by The frequencies of the m-th image patch are then summed to obtain the frequency score. .

[0058] The mathematical definition of a bandpass filter is: ; in, This indicates the total number of bandpass filters; The index represents the bandpass filter; i and j represent the two-dimensional spatial coordinate indices of pixels within the image patch (i is the row index, j is the column index); N represents the size (width or height) of the image patch. This represents the weight of the k-th bandpass filter at position (i,j) in the image block.

[0059] The formula for calculating the frequency score is: ; Where c represents the number of channels in the image block.

[0060] The frequency scores of all image patches are obtained using the above formula. Select k high-frequency image blocks with the highest frequency scores. and the k low-frequency image patches with lower frequency scores .

[0061] The underlying feature extraction module is specifically used for: The selected high-frequency and low-frequency image blocks are scaled to a preset resolution (256×256 pixels), and the scaled high-frequency and low-frequency image blocks are input into the spatial enrichment model filter to extract the corresponding noise mode features. The noise pattern features are input into a feature extractor built on a ConvNeXt network to obtain the corresponding feature maps. Average pooling is performed on each of the feature maps to obtain high-frequency low-level features. and low-frequency underlying features ; For the high-frequency low-level features and low-frequency underlying features Average pooling is performed to fuse the features, resulting in the fused low-level feature representation. : .

[0062] Average pooling can reduce feature dimensionality while preserving key information, resulting in a more stable and representative low-level feature representation.

[0063] Compared to the traditional ResNet50 structure, this invention employs the ConvNext network as the backbone feature extractor in the feature extraction stage, constructing independent network branches. The ConvNext network used has significant advantages in this task: First, through large convolutional kernels (such as 7*7 depthwise separable convolutions) and modern network design, the ConvNext network maintains the efficiency of the convolutional structure while possessing a larger receptive field, which is beneficial for capturing subtle texture artifacts in the generated images. Second, the ConvNext network introduces structures such as LayerNorm, depthwise separable convolutions, and inverted bottlenecks, making its feature representation ability and training stability significantly superior to the traditional ResNet network, enabling it to better characterize the fine-grained features generated by the generative model. In addition, the ConvNext network achieves a good balance between parameters and computational efficiency, exhibiting stronger modeling capabilities for features such as hidden noise patterns, texture smoothing, and anti-aliasing artifacts in high-frequency and low-frequency image patches, thereby further improving the feature discrimination ability in the AIGC image detection task.

[0064] The high-level semantic feature extraction module is specifically used for: The RGB image is input into the SAM3 visual encoder to obtain a high-dimensional feature representation; the high-dimensional feature representation is input into a linear projection layer for dimensional mapping to obtain projected features; the projected features are then subjected to spatial average pooling to obtain high-level semantic features. To capture rich semantic features in images (such as object co-occurrence relationships, scene context associations, and spatial structure consistency), this invention optimizes the traditional method using OpenClip's visual encoder to use a Segment AnythingModel3 (SAM3) visual encoder to compute the visual embedding (high-dimensional feature representation) of the image. To further enhance the global semantic representation capability, a linear projection layer g(·) is cascaded after the SAM3 visual encoder to map and compress the feature dimensions of the high-dimensional feature representation, and the final high-level semantic features are obtained through spatial average pooling. The formula is: .

[0065] Compared to traditional visual encoders, the SAM3 visual encoder has significant advantages in this invention. First, as a large-scale visual foundation model, SAM3 is pre-trained on massive image data, possessing stronger general visual representation capabilities. It can effectively capture semantic structural information in complex scenes, such as spatial relationships between objects, contextual dependencies, and the overall layout of the scene. This is of great significance for identifying potential semantic inconsistencies or structural anomalies in AIGC-generated images. Second, the SAM3 visual encoder adopts a Transformer-based global modeling mechanism, which can establish long-distance dependencies within a larger receptive field, thereby more accurately modeling global contextual features in images. Traditional convolutional networks are often limited by local receptive fields in this respect. Third, the SAM3 visual encoder is designed with greater emphasis on the ability to express image structure and boundary information. It can perform more refined modeling of object contours, regional relationships, and structural continuity in images. This helps to discover potential anomalies in the object structure, perspective relationships, or semantic logic of AI-generated images.

[0066] Therefore, the SAM3 visual encoder is introduced in this invention for semantic feature extraction. This not only obtains a more discriminative global semantic embedding, but also complements the statistical features of the underlying image patches. This enables joint modeling of texture-level artifacts and semantic-level anomalies in the AIGC image detection task, further improving the accuracy and generalization ability of the detection model.

[0067] In the image classification and discrimination module, the classifier is a multilayer perceptron (MLP). The calculation formula for the classifier is: ; in, The symbol ";" represents the structure of a multilayer perceptron, which consists of a linear mapping layer, a GELU activation function, and a final classification layer; the symbol ";" represents the feature concatenation operation in the channel dimension.

[0068] Through the image classification and discrimination module, the AIGC image detection model can comprehensively utilize low-level statistical features and high-level semantic information to achieve efficient and accurate identification of AI-generated images and real images.

[0069] In summary, the advantages of this invention are: 1. By training with mixed data from multiple generative models, the AIGC image detection model is forced to learn common forgery patterns across models rather than fingerprints from a single model. At the same time, a multi-level analysis architecture is constructed that integrates frequency domain scores, low-level noise patterns, and high-level semantic features extracted by SAM3. This ensures that even if an unknown model is close to reality in one aspect, traces left behind at other levels can still be captured. Furthermore, by freezing the parameters of the SAM3 visual encoder, the general visual representation learned on large-scale data is preserved, and only the projection layer is fine-tuned for task adaptation. This systematically enhances the model's generalization ability and robustness when facing unknown generative models.

[0070] 2. By acquiring AIGC images from various different generative models to construct a training dataset, the diversity of the model is enhanced from the data source. This allows the trained detection model to learn more general and essential AIGC image features rather than targeting a single generator (such as only StableDiffusion or DALL-E). As a result, it can still maintain a high detection accuracy when facing unknown or newly emerging generative models, effectively improving the model's generalization ability and robustness in practical applications.

[0071] 3. Through a frequency scoring screening mechanism, low-level features reflecting subtle differences in noise, texture, etc., are extracted from high-frequency and low-frequency image blocks respectively. At the same time, the powerful visual foundation model (SAM3) is used to extract the overall high-level semantic features of the image. Finally, the two are fused to form a multi-level fused feature. This design combines bottom-up detail analysis and top-down global understanding, enabling the model to capture the small artifacts left by AIGC in local pixel statistical regularities, and also to perceive its defects in global semantic coherence and object structure rationality, thus achieving a more comprehensive and accurate discrimination.

[0072] 4. During model training, a differentiated parameter update strategy was adopted: the low-level feature extractor was fine-tuned to adapt to the specific detection task, while the backbone parameters of the SAM3 visual encoder were frozen and only its linear projection layer was trained. This strategy makes full use of the powerful general feature representation capability of large-scale pre-trained visual models, while avoiding overfitting or catastrophic forgetting problems that may be caused by completely fine-tuning such large models. This not only ensures the quality of high-level semantic features, but also significantly reduces the number of parameters that need to be trained, improves training efficiency and stability, and enables the model to quickly converge to a high-performance solution.

[0073] 5. During the inference phase, the input image is first divided into blocks, and key high-frequency and low-frequency blocks are selected based on frequency scores for subsequent deep feature extraction, rather than performing equally complex calculations on all regions of the entire image. This "attention" mechanism allows computational resources to be focused on the contrasting regions that are most likely to contain generation traces (high-frequency detail anomalies) and least likely to contain them (low-frequency smooth regions). Without losing detection accuracy or even improving it (due to the removal of irrelevant information interference), the computational cost required for a single detection is significantly reduced, making the scheme more feasible for application in real-time or high-concurrency scenarios.

[0074] 6. By systematically analyzing the feature changes produced by AI-generated images at the visual level, the rationality of AI-generated image detection tasks was re-verified, and unreasonable assumptions in the training and testing settings of existing studies were re-examined and improved. Traditional methods often assume that training and testing data come from the same or similar generative model distributions. This assumption is difficult to hold in practical applications. Therefore, this invention introduces cross-model data sources to improve the generalization ability of the detection model on unknown generative models. In terms of model architecture, a simple and efficient detector is designed based on the AIDE method. By fusing low-level image patch statistical features and high-level semantic representations, multi-level information collaborative modeling is achieved. The low-level features mainly capture statistical anomalies in texture distribution, noise patterns, and local structure of the generative model, while the high-level semantic features are used to identify inconsistencies in object structure, spatial relationships, and semantic logic. Through the dual-layer feature fusion mechanism, the model can simultaneously utilize pixel-level detail information and semantic-level contextual information, thereby significantly improving the accuracy and robustness of AI-generated image detection. Finally, while maintaining the simplicity of the model structure, effective identification of complex AIGC forged images is achieved.

[0075] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An AIGC image detection method based on multi-level feature fusion, characterized in that: Includes the following steps: Step S1: Obtain AI-generated images and real-shot images from various different generation models, and construct a training dataset based on the AI-generated images and real-shot images. Step S2: Create an AIGC image detection model based on the frequency score calculation module, the low-level feature extraction module, the high-level semantic feature extraction module, the feature fusion module, and the image classification and discrimination module; Step S3: Train the AIGC image detection model using the training dataset to optimize the model parameters; during the training process, fine-tune the feature extractor of the low-level feature extraction module, freeze the parameters of the SAM3 visual encoder of the high-level semantic feature extraction module, and train only the linear projection layer of the high-level semantic feature extraction module. Step S4: Containerize and deploy the trained AIGC image detection model as a service; Step S5: Receive a detection request sent by the client, wherein the detection request carries an encoded RGB image to be detected; Step S6: Perform image block segmentation processing on the RGB image to obtain an image set containing multiple image blocks. Input the image set into the deployed AIGC image detection model to obtain the discrimination result of whether the RGB image is an artificial intelligence generated image. Return the discrimination result to the client.

2. The AIGC image detection method based on multi-level feature fusion as described in claim 1, characterized in that: In step S2, the frequency score calculation module is used to calculate the frequency score of each image block based on discrete cosine transform and multiple preset bandpass filters, and to select a preset number of high-frequency image blocks and low-frequency image blocks from the image set according to the frequency score. The low-level feature extraction module is used to extract high-frequency low-level features and low-frequency low-level features from the high-frequency image block and the low-frequency image block respectively, and to fuse the high-frequency low-level features and low-frequency low-level features to obtain the fused low-level feature representation. The high-level semantic feature extraction module is used to perform overall analysis of the RGB image and extract high-level semantic features; The feature fusion module is used to concatenate the low-level feature representation with the high-level semantic features to obtain multi-level fused features; The image classification and discrimination module is used to input the multi-level fusion features into the classifier to obtain a discrimination result on whether the RGB image is an artificial intelligence generated image.

3. The AIGC image detection method based on multi-level feature fusion as described in claim 2, characterized in that: In the frequency score calculation module, the frequency score calculation process is as follows: Take the absolute value of the DCT coefficients of the m-th image block, add 1, and then perform logarithmic transformation to obtain the logarithmic result. ; The k-th bandpass filter The intermediate result of each bandpass filter is obtained by multiplying the logarithmic result at all positions and channels and summing the results. Multiply the intermediate results of each of the bandpass filters by The frequencies of the m-th image patch are then summed to obtain the frequency score. .

4. The AIGC image detection method based on multi-level feature fusion as described in claim 2, characterized in that: The underlying feature extraction module is specifically used for: The selected high-frequency and low-frequency image blocks are scaled to a preset resolution, and the scaled high-frequency and low-frequency image blocks are input into the spatial enrichment model filter to extract the corresponding noise mode features. The noise pattern features are input into a feature extractor built on a ConvNeXt network to obtain the corresponding feature maps. Average pooling is performed on each of the feature maps to obtain high-frequency low-level features. and low-frequency underlying features ; For the high-frequency low-level features and low-frequency underlying features Average pooling is performed to fuse the features, resulting in the fused low-level feature representation. : .

5. The AIGC image detection method based on multi-level feature fusion as described in claim 2, characterized in that: The high-level semantic feature extraction module is specifically used for: The RGB image is input into the SAM3 visual encoder to obtain a high-dimensional feature representation; the high-dimensional feature representation is then input into a linear projection layer for dimensional mapping to obtain projected features. Spatial average pooling is performed on the projected features to obtain high-level semantic features; In the image classification and discrimination module, the classifier is a multilayer perceptron.

6. An AIGC image detection system based on multi-level feature fusion, characterized in that: Includes the following modules: The training dataset construction module is used to acquire AI-generated images and real-world images from various different generative models, and to construct a training dataset based on the AI-generated images and real-world images. The AIGC image detection model creation module is used to create an AIGC image detection model based on the frequency score calculation module, the low-level feature extraction module, the high-level semantic feature extraction module, the feature fusion module, and the image classification and discrimination module. The AIGC image detection model training module is used to train the AIGC image detection model using the training dataset to optimize model parameters. During the training process, the feature extractor of the low-level feature extraction module is trained and fine-tuned, the parameters of the SAM3 visual encoder of the high-level semantic feature extraction module are frozen, and only the linear projection layer of the high-level semantic feature extraction module is trained. The AIGC image detection model deployment module is used to containerize and deploy the trained AIGC image detection model as a service. The detection request receiving module is used to receive a detection request sent by the client, wherein the detection request carries an encoded RGB image to be detected; The discrimination result output module is used to perform image block division processing on the RGB image to obtain an image set containing multiple image blocks, input the image set into the deployed AIGC image detection model, obtain the discrimination result of whether the RGB image is an artificial intelligence generated image, and return the discrimination result to the client.

7. The AIGC image detection system based on multi-level feature fusion as described in claim 6, characterized in that: In the AIGC image detection model creation module, the frequency score calculation module is used to calculate the frequency score of each image block based on discrete cosine transform and multiple preset bandpass filters, and to select a preset number of high-frequency image blocks and low-frequency image blocks from the image set according to the frequency score. The low-level feature extraction module is used to extract high-frequency low-level features and low-frequency low-level features from the high-frequency image block and the low-frequency image block respectively, and to fuse the high-frequency low-level features and low-frequency low-level features to obtain the fused low-level feature representation. The high-level semantic feature extraction module is used to perform overall analysis of the RGB image and extract high-level semantic features; The feature fusion module is used to concatenate the low-level feature representation with the high-level semantic features to obtain multi-level fused features; The image classification and discrimination module is used to input the multi-level fusion features into the classifier to obtain a discrimination result on whether the RGB image is an artificial intelligence generated image.

8. The AIGC image detection system based on multi-level feature fusion as described in claim 7, characterized in that: In the frequency score calculation module, the frequency score calculation process is as follows: Take the absolute value of the DCT coefficients of the m-th image block, add 1, and then perform logarithmic transformation to obtain the logarithmic result. ; The k-th bandpass filter The intermediate result of each bandpass filter is obtained by multiplying the logarithmic result at all positions and channels and summing the results. Multiply the intermediate results of each of the bandpass filters by The frequencies of the m-th image patch are then summed to obtain the frequency score. .

9. The AIGC image detection system based on multi-level feature fusion as described in claim 7, characterized in that: The underlying feature extraction module is specifically used for: The selected high-frequency and low-frequency image blocks are scaled to a preset resolution, and the scaled high-frequency and low-frequency image blocks are input into the spatial enrichment model filter to extract the corresponding noise mode features. The noise pattern features are input into a feature extractor built on a ConvNeXt network to obtain the corresponding feature maps. Average pooling is performed on each of the feature maps to obtain high-frequency low-level features. and low-frequency underlying features ; For the high-frequency low-level features and low-frequency underlying features Average pooling is performed to fuse the features, resulting in the fused low-level feature representation. : .

10. The AIGC image detection system based on multi-level feature fusion as described in claim 7, characterized in that: The high-level semantic feature extraction module is specifically used for: The RGB image is input into the SAM3 visual encoder to obtain a high-dimensional feature representation; the high-dimensional feature representation is then input into a linear projection layer for dimensional mapping to obtain projected features. Spatial average pooling is performed on the projected features to obtain high-level semantic features; In the image classification and discrimination module, the classifier is a multilayer perceptron.

Images