Image generation source tracing method based on spatial frequency cross-domain second-order statistics

Abstract

This invention discloses a method for tracing the source of generated images based on spatial-frequency cross-domain second-order statistics. This method extracts spatial domain residual anomalous features and frequency domain anomalous features from the generated image, and constructs a cross-domain second-order statistical coupling relationship between them, forming a cross-domain statistical fingerprint for source tracing. The cross-domain statistical fingerprint matrix is ​​obtained by calculating the cross-domain cross-correlation matrix, and then normalized and compressed to obtain a cross-domain second-order statistical feature representation. Based on the similarity matching or distance measurement results between this feature representation and the statistical fingerprint templates of each source category, the source category or confidence score of the generated image is output. This invention can explicitly characterize the coupling relationship between spatial anomalies and frequency anomalies, and still has strong source discrimination ability and good interpretability and generalization performance even when the generation models are highly similar.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision and image forensics, specifically relating to a method for tracing the source of generated images based on spatial frequency cross-domain second-order statistics. Background Technology

[0002] With the rapid development of generative model technology, AI-generated images have been widely used in content creation and media dissemination. Due to differences in network structure, training mechanisms, and sampling strategies, different generative models may introduce potential anomalous features with source relevance during the generation process. These features constitute the model fingerprint of the generated image, which is an important basis for tracing the source of the generated image.

[0003] Existing methods for tracing the source of generated images mainly rely on spatial or frequency domain features. However, as generative models evolve, the visual quality and statistical distribution of their generated images gradually converge, and spatial domain residual anomalies and frequency domain energy anomalies both show a weakening trend. This makes it difficult for source tracing methods based on single-domain features to extract stable and discriminative source fingerprints, especially between highly similar generative models, where the source discrimination performance drops significantly.

[0004] Furthermore, while some methods simultaneously incorporate spatial and frequency domain features, they often employ simple splicing, independent modeling, or linear fusion, making it difficult to explicitly characterize the statistical dependency between spatial domain residual anomalies and frequency domain anomalies. This results in the inadequate extraction of cross-domain coupling discriminative information, thus affecting the accuracy and stability of source tracing for generated images. Moreover, under post-processing conditions such as compression and scaling, cross-domain statistical features are more prone to degradation, further impacting the stability of source tracing feature representations. Summary of the Invention

[0005] To address the aforementioned issues, this invention discloses a generative image source tracing method based on spatial frequency cross-domain second-order statistics, which can effectively improve the source differentiation capability and source tracing stability in highly similar generative model scenarios.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A generative image source tracing method based on spatial frequency cross-domain second-order statistics includes the following steps:

[0008] S1: For the generated image Frequency domain anomaly feature extraction and spatial domain residual anomaly feature extraction are performed separately to obtain frequency domain anomaly features. and spatial domain residual characteristics ;

[0009] S2: Based on the aforementioned frequency domain anomaly features Distinct features of spatial domain residuals Constructing a cross-domain second-order statistical matrix To characterize the statistical coupling relationship between frequency domain anomalies and spatial domain residual anomalies;

[0010] S3: For the aforementioned cross-domain second-order statistical matrix Normalization and feature compression are performed to obtain cross-domain second-order statistical feature representations. ;

[0011] S4: Based on cross-domain second-order statistical feature representation It performs similarity matching or distance measurement with pre-built statistical fingerprint templates for each source category, and outputs the source category based on the matching results.

[0012] Furthermore, S1 specifically includes the following steps:

[0013] S1.1: For the generated image A frequency domain transformation is performed to obtain a frequency domain representation, and the frequency domain representation is divided into frequency bands according to the frequency distribution to obtain frequency domain anomaly features that reflect the spectral energy distribution shift characteristics of the generated model. ;

[0014] S1.2: For the generated image Residual modeling is performed to suppress semantic content in the image, and region-aware feature modulation is applied to the residual features to obtain spatial domain residual abnormal features. ;

[0015] Furthermore, in step S2, the frequency domain anomaly features are analyzed. Distinct features of spatial domain residuals Spatial resolution alignment is performed, and the sequences are flattened into two-dimensional feature sequences according to their spatial location, followed by mean centering. Based on the centered two-dimensional feature sequences, a cross-domain second-order statistical matrix is ​​calculated. .

[0016] Furthermore, in step S3, the cross-domain second-order statistical matrix obtained in step S2 is... The matrix is ​​then normalized by sign square root, and the normalized matrix is ​​expanded row-wise into a one-dimensional vector. This one-dimensional vector is then L2 normalized to obtain a cross-domain second-order statistical feature representation. .

[0017] Furthermore, in step S4, for each source category, based on the cross-domain second-order statistical feature representation corresponding to the training samples... Mean aggregation is performed to construct a statistical fingerprint template corresponding to the source category. This yields a cross-domain second-order statistical feature representation of the generated image to be tested. Then, calculate The cosine similarity between the fingerprint templates and the data from each source category is calculated, and the source category with the highest cosine similarity is determined as the source category of the generated image to be tested.

[0018] The beneficial effects of this invention are:

[0019] This invention discloses a method for tracing the source of generated images based on cross-domain second-order statistics of spatial frequency. By extracting frequency domain anomaly features and spatial domain residual anomaly features from the generated image respectively, and constructing a cross-domain second-order statistical matrix between them, it can explicitly characterize the statistical coupling fingerprint between spectral anomalies and spatial residual anomalies, thereby avoiding the problem of insufficient discriminative information caused by relying solely on single-domain features or simple splicing and fusion. This invention further normalizes and compresses the cross-domain second-order statistical matrix to obtain a stable cross-domain second-order statistical feature representation. Source discrimination is achieved based on the similarity matching between the feature representation and the source category statistical fingerprint template, enabling the method to maintain strong source discrimination ability even when the generated models are highly similar and explicit artifacts gradually weaken. This invention's method has good interpretability and generalization performance, and is suitable for applications such as forensic source tracing of generated images, providing a reliable technical means for identifying the source of generated images. Attached Figure Description

[0020] Figure 1 This is a network structure diagram of the present invention.

[0021] Figure 2 This is a network structure diagram of the frequency domain anomaly feature extraction module in a specific implementation plan.

[0022] Figure 3 This is a network structure diagram of the spatial domain residual abnormal feature extraction module in a specific embodiment of the present invention.

[0023] Figure 4 This is a cross-domain two-stage statistical matrix network structure diagram in a specific embodiment of the present invention. Detailed Implementation

[0024] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0025] As shown in the figure, the image source tracing method based on spatial frequency cross-domain second-order statistics of the present invention includes the following steps:

[0026] S1: For the generated image Frequency domain anomaly feature extraction and spatial domain residual anomaly feature extraction are performed separately to obtain frequency domain anomaly features. and spatial domain residual characteristics ;

[0027] S2: Based on the aforementioned frequency domain anomaly features Distinct features of spatial domain residuals Constructing a cross-domain second-order statistical matrix To characterize the statistical coupling relationship between frequency domain anomalies and spatial domain residual anomalies;

[0028] S3: For the aforementioned cross-domain second-order statistical matrix Normalization and feature compression are performed to obtain cross-domain second-order statistical feature representations. ;

[0029] S4: Based on cross-domain second-order statistical feature representation It performs similarity matching or distance measurement with pre-built statistical fingerprint templates for each source category, and outputs the source category based on the matching results.

[0030] Furthermore, S1 specifically includes the following steps:

[0031] S1.1: For the generated image A frequency domain transformation is performed to obtain a frequency domain representation, and the frequency domain representation is divided into frequency bands according to the frequency distribution to obtain frequency domain anomaly features that reflect the spectral energy distribution shift characteristics of the generated model. ;

[0032] Specifically, for the generated image A two-dimensional fast Fourier transform is performed to obtain the complex representation in the frequency domain, and its amplitude spectrum is calculated as the frequency domain representation. Subsequently, the amplitude spectrum is divided into frequency bands according to its frequency distribution. The low-frequency component is concatenated with the mid-to-high-frequency components and then input into a frequency domain anomaly feature extraction network for feature encoding to obtain frequency domain anomaly features. Frequency domain anomaly characteristics Used to characterize anomalous information in the spectral energy distribution of the generated image.

[0033] S1.2: For the generated image Residual modeling is performed to suppress semantic content in the image, and region-aware feature modulation is applied to the residual features to obtain spatial domain residual abnormal features. ;

[0034] Specifically, for the generated image Residual modeling is performed to suppress the semantic content of the image and highlight the spatial residual abnormalities introduced during the generation process, resulting in a residual response map. The processing procedure is shown in equation (1):

[0035] (1);

[0036] In the formula, This represents the generated image as input. This invention represents the residual extraction operator. This is a 3×3 Laplace high-pass filter operator. Subsequently, the residual response plot is shown. The input spatial domain anomaly feature extraction network is used for feature encoding, and region-aware feature modulation processing is introduced during the feature encoding process to highlight the spatial domain residual anomaly information introduced during the generation process, thereby obtaining spatial domain residual anomaly features. Spatial domain residual characteristics Used to characterize residual anomalies in the generated image at the spatial structure level.

[0037] Furthermore, in step S2, the frequency domain anomaly features obtained in step S1.1 are... Spatial domain residual abnormal features obtained in step S1.2 Perform spatial resolution alignment to make them have the same spatial size. .in Indicates the feature map height. The feature map width is represented by the matrix. The aligned features are flattened into a two-dimensional feature sequence according to their spatial location, and then mean-centered. Based on the centered two-dimensional feature sequence, a cross-domain second-order statistical matrix is ​​calculated through matrix multiplication. The processing procedure is shown in equation (2):

[0038] (2);

[0039] In the formula, This represents the number of spatial locations after flattening. Cross-domain second-order statistical matrix. A statistical fingerprint used to characterize the coupling between spatial domain residual anomalies and frequency domain anomalies.

[0040] Furthermore, in step S3, the cross-domain second-order statistical matrix obtained in step S2 is... Perform sign-square root normalization to obtain the normalized matrix. The processing procedure is shown in equation (3):

[0041] (3);

[0042] Then, the normalized matrix Expanding by rows yields a one-dimensional vector, which is then L2 normalized to obtain a cross-domain second-order statistical feature representation. .

[0043] Furthermore, in step S4, during the training phase, for each source category... The cross-domain second-order statistical features calculated from training samples belonging to this source category. This indicates that mean aggregation is performed to obtain a statistical fingerprint template for that source category. The processing procedure is shown in equation (4):

[0044] (4);

[0045] In the formula, Indicates source category The number of training samples. Indicates source category The Middle Cross-domain second-order statistical feature representations corresponding to each training sample. During the testing phase, cross-domain second-order statistical feature representations are obtained for the generated image to be tested. ,calculate Statistical fingerprint templates for each source category cosine similarity The processing procedure is shown in equation (5):

[0046] (5);

[0047] The category with the highest cosine similarity is taken as the source category of the generated image to be tested. The processing procedure is shown in equation (6):

[0048] (6);

[0049] Therefore, the source category of the generated image to be tested is output.

[0050] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A generative image source tracing method based on spatial frequency cross-domain second-order statistics, characterized in that, Includes the following steps: S1: For the generated image Frequency domain anomaly feature extraction and spatial domain residual anomaly feature extraction are performed separately to obtain frequency domain anomaly features. and spatial domain residual characteristics ; S2: Based on the aforementioned frequency domain anomaly features Distinct features of spatial domain residuals Constructing a cross-domain second-order statistical matrix To characterize the statistical coupling relationship between frequency domain anomalies and spatial domain residual anomalies; S3: For the aforementioned cross-domain second-order statistical matrix Normalization and feature compression are performed to obtain cross-domain second-order statistical feature representations. ; S4: Based on cross-domain second-order statistical feature representation It performs similarity matching or distance measurement with pre-built statistical fingerprint templates for each source category, and outputs the source category based on the matching results.

2. The image source tracing method based on spatial frequency cross-domain second-order statistics according to claim 1, characterized in that, S1 specifically includes the following steps. S1.1: For the generated image A frequency domain transformation is performed to obtain a frequency domain representation, and the frequency domain representation is divided into frequency bands according to the frequency distribution to obtain frequency domain anomaly features that reflect the spectral energy distribution shift characteristics of the generated model. ; Specifically, for the generated image A two-dimensional fast Fourier transform is performed to obtain the complex representation in the frequency domain, and its amplitude spectrum is calculated as the frequency domain representation. Subsequently, the amplitude spectrum is divided into frequency bands according to the frequency distribution. The low-frequency component is concatenated with the mid-to-high-frequency components and then input into the frequency domain anomaly feature extraction network for feature encoding to obtain the frequency domain anomaly features. Frequency domain anomaly characteristics Used to characterize anomalous information in the spectral energy distribution of generated images; S1.2: For the generated image Residual modeling is performed to suppress semantic content in the image, and region-aware feature modulation is applied to the residual features to obtain spatial domain residual abnormal features. ; Specifically, for the generated image Residual modeling is performed to suppress the semantic content of the image and highlight the spatial residual abnormalities introduced during the generation process, resulting in a residual response map. The processing procedure is shown in equation (1): （1）； In the formula, This represents the generated image as input. This represents the residual extraction operator. A 3×3 Laplace high-pass filter operator is used; subsequently, the residual response plot is shown. The input spatial domain anomaly feature extraction network is used for feature encoding, and region-aware feature modulation processing is introduced during the feature encoding process to highlight the spatial domain residual anomaly information introduced during the generation process, thereby obtaining spatial domain residual anomaly features. Spatial domain residual abnormal characteristics Used to characterize residual anomalies in the generated image at the spatial structure level.

3. The image source tracing method based on spatial frequency cross-domain second-order statistics according to claim 2, characterized in that, In step S2, the frequency domain anomaly features obtained in step S1.1 are... Spatial domain residual abnormal features obtained in step S1.2 Perform spatial resolution alignment to make them have the same spatial size. ;in Indicates the feature map height. The feature map width is represented; the aligned features are flattened into a two-dimensional feature sequence according to their spatial location, and mean centering is performed on each sequence; based on the centered two-dimensional feature sequence, a cross-domain second-order statistical matrix is ​​calculated through matrix multiplication. The processing procedure is shown in equation (2): （2）； In the formula, Represents the number of spatial locations after flattening; cross-domain second-order statistical matrix A statistical fingerprint used to characterize the coupling between spatial domain residual anomalies and frequency domain anomalies.

4. The image source tracing method based on spatial frequency cross-domain second-order statistics according to claim 1, characterized in that, In step S3, the cross-domain second-order statistical matrix obtained in step S2 is... Perform sign-square root normalization to obtain the normalized matrix. The processing procedure is shown in equation (3): （3）； Then, the normalized matrix Expanding by rows yields a one-dimensional vector, which is then L2 normalized to obtain a cross-domain second-order statistical feature representation. .

5. The image source tracing method based on spatial frequency cross-domain second-order statistics according to claim 1, characterized in that, In step S4, during the training phase, for each source category... The cross-domain second-order statistical features calculated from training samples belonging to this source category. This indicates that mean aggregation is performed to obtain a statistical fingerprint template for that source category. The processing procedure is shown in equation (4): （4）； In the formula, Indicates source category The number of training samples; Indicates source category The Middle Cross-domain second-order statistical feature representations corresponding to each training sample; During the testing phase, cross-domain second-order statistical feature representations were obtained for the generated image to be tested. ,calculate Statistical fingerprint templates for each source category cosine similarity The processing procedure is shown in equation (5): （5）； The category with the highest cosine similarity is taken as the source category of the generated image to be tested. The processing procedure is shown in equation (6): （6）； Therefore, the source category of the generated image to be tested is output.

Images