An image forgery tracing method based on graph convolutional modeling and fingerprint library matching

Abstract

The present invention discloses a method for tracing the source of image forgery based on graph convolution modeling and fingerprint library matching, which belongs to the technical field of image forgery tracing. The present invention extracts the explicit texture and implicit frequency features of the forged image by fusing the spatial domain and frequency domain features, and combines the graph convolution network to model the feature topological relationship to improve the accuracy of forged fingerprint recognition. The double marginal loss is used to optimize the feature distribution, distinguish the known and identify new generation technologies, and combine the fingerprint library matching with the dynamic threshold to achieve efficient classification and real-time detection in open environments. The lightweight structure reduces the computational overhead; DBSCAN clustering automatically expands the fingerprint library without manual intervention or retraining, reducing maintenance costs. The dynamic update module generates new categories and updates the fingerprint library to adapt to technology iterations and is suitable for digital content verification and copyright protection.

Description

Technical Field

[0001] The present invention belongs to the technical field of image forgery tracing, and in particular relates to an image forgery tracing method based on graph convolution modeling and fingerprint library matching. Background Art

[0002] Image forgery traceability technology is crucial for identifying image authenticity and tracing the source of forgeries. Existing technologies primarily rely on supervised learning methods, training classifiers to distinguish between real images and those generated using specific forgery methods. For example, methods based on convolutional neural networks extract spatial image features and utilize multi-class classifiers to identify forgery methods. While these methods are effective against known forgery methods, they still have significant drawbacks:

[0003] Insufficient feature relationship modeling: Although existing methods integrate multi-scale features, they do not fully model the topological relationship between features, making it difficult to capture the deep patterns of forged fingerprints and affecting the traceability accuracy of complex forgery scenarios.

[0004] Lack of generalization ability: Relying on fixed training data and model structure, it can only identify known forgery methods and cannot adapt to new generation technologies in open environments (such as iterative GANs or diffusion models). It is difficult to cope with the rapidly evolving forgery methods in practical applications.

[0005] Inefficient computation: Multi-branch networks and complex structures (such as capsule networks) introduce excessive computational overhead, making them difficult to deploy in large-scale, real-time scenarios (such as social media content moderation or online infringement tracking). This problem stems from existing technologies' neglect of feature correlation modeling and the limitations of static architectures, resulting in systems that cannot balance accuracy, adaptability, and real-time performance. Therefore, a traceability solution is urgently needed that can efficiently model feature relationships, dynamically expand recognition capabilities, and optimize computational efficiency. Summary of the Invention

[0006] To solve the above technical problems, the present invention proposes an image forgery tracing method and system based on graph convolutional modeling and fingerprint library matching to solve the problems existing in the above-mentioned prior art.

[0007] In a first aspect, to achieve the above-mentioned objectives, the present invention provides an image forgery tracing method based on graph convolutional modeling and fingerprint library matching, comprising the following steps:

[0008] Extracting spatial domain features and frequency domain features from an input image, and fusing the spatial domain features and frequency domain features to generate comprehensive features;

[0009] Inputting the comprehensive features into a graph convolutional network, generating enhanced forged fingerprint features by constructing a feature relationship graph and aggregating node information;

[0010] Optimizing the distribution of the forged fingerprint features based on a dual marginal contrast loss function so that the feature distance of the same forgery method is reduced and the feature distance of different forgery methods is increased;

[0011] Matching the forged fingerprint features with reference fingerprints in a fingerprint database, and determining the type of forgery method based on the matching distance;

[0012] When an unknown forgery method is detected, the unknown forgery fingerprint corresponding to the unknown forgery method is stored in the fingerprint pool. When the capacity of the fingerprint pool reaches a threshold, a new fingerprint category is generated by clustering and the fingerprint library is updated.

[0013] Optionally, the process of extracting spatial domain features and frequency domain features includes:

[0014] Extracting spatial domain features of the input image through a pre-trained residual network;

[0015] Convert the input image into a grayscale image, extract frequency domain features through discrete cosine transform, and adjust the dimension of the frequency domain features through a feature enhancement module;

[0016] The spatial domain features and the frequency domain features are spliced ​​together to form the comprehensive features.

[0017] Optionally, the process of constructing the feature relationship graph includes:

[0018] Calculating the cosine similarity between nodes based on the comprehensive features to generate an adjacency matrix;

[0019] Adding self-connection to the adjacency matrix and performing symmetric normalization to obtain a normalized adjacency matrix;

[0020] The normalized adjacency matrix and the two-layer graph convolutional network are used to aggregate node information to generate the enhanced forged fingerprint features.

[0021] Optionally, the optimization process of the dual marginal contrast loss function includes:

[0022] According to the forgery method category label and method similarity label of the sample, the intra-class distance constraint of the same forgery method category and the inter-class distance constraint of different method categories are calculated;

[0023] The distribution of the forged fingerprint features is controlled by preset intra-class margin parameters and inter-class margin parameters.

[0024] Optionally, the fingerprint database construction and matching process includes:

[0025] Selecting a reference sample set for each known counterfeiting method, extracting the fingerprint of the reference sample set and calculating the class center;

[0026] The minimum Euclidean distance between the test fingerprint and the center of the category is calculated. If the minimum Euclidean distance exceeds a preset threshold, it is determined to be an unknown forgery method.

[0027] Optionally, the process of updating the fingerprint database includes:

[0028] Storing the unknown forged fingerprints in a fingerprint pool according to categories, and performing density clustering on the fingerprint pool when the capacity of the fingerprint pool reaches a preset capacity threshold;

[0029] A new fingerprint category is generated according to the clustering result, and the category center in the fingerprint database is updated.

[0030] In a second aspect, the present invention further provides an image forgery tracing system based on graph convolutional modeling and fingerprint library matching, which is used to implement an image forgery tracing method based on graph convolutional modeling and fingerprint library matching. The system includes:

[0031] A multi-domain feature extraction module is used to extract spatial domain features and frequency domain features from the input image, and fuse the spatial domain features and frequency domain features to generate comprehensive features;

[0032] A graph convolution modeling module is configured to receive the comprehensive features, construct a feature relationship graph, and aggregate node information to generate enhanced forged fingerprint features;

[0033] a loss optimization module for optimizing the distribution of the forged fingerprint features based on a double marginal contrast loss function, so as to reduce the feature distance of the same forgery method and increase the feature distance of different forgery methods;

[0034] A fingerprint matching module, configured to match the forged fingerprint features with reference fingerprints in a fingerprint library and determine the type of forgery method based on the matching distance;

[0035] The dynamic update module is used to store the unknown forged fingerprint corresponding to the unknown forgery method into the fingerprint pool when an unknown forgery method is detected, and to generate a new fingerprint category by clustering and update the fingerprint library when the fingerprint pool capacity reaches a threshold.

[0036] Optionally, the multi-domain feature extraction module includes:

[0037] A spatial domain feature extraction unit, used to extract spatial domain features of an input image through a pre-trained residual network;

[0038] A frequency domain feature extraction unit is used to convert the input image into a grayscale image, extract frequency domain features through discrete cosine transform, and adjust the dimension of the frequency domain features through a feature enhancement module;

[0039] The feature fusion unit is used to combine the spatial domain features and the frequency domain features to form the comprehensive features.

[0040] In a third aspect, the present invention further provides a computer terminal device, comprising:

[0041] one or more processors;

[0042] a memory, coupled to the processor, for storing one or more programs;

[0043] When the one or more programs are executed by the one or more processors, the one or more processors implement, for example, an image forgery tracing method based on graph convolution modeling and fingerprint library matching.

[0044] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements, for example, an image forgery tracing method based on graph convolution modeling and fingerprint library matching.

[0045] Compared with the prior art, the present invention has the following advantages and technical effects:

[0046] The present invention provides an image forgery tracing method and system based on graph convolutional modeling and fingerprint library matching. The present invention improves feature representation capabilities by comprehensively utilizing spatial domain, frequency domain, and feature relationship information through multi-domain feature fusion and graph convolutional modeling. It also enhances generalization capabilities by combining DMC loss and fingerprint library matching to support the detection of unprecedented forgery methods. The dynamic update mechanism based on DBSCAN enables the automatic expansion of the fingerprint library to adapt to the rapid iteration of forgery technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which constitute part of the present invention, are provided to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0048] Figure 1 This is a flow chart of a traceability method according to an embodiment of the present invention;

[0049] Figure 2 This is a model structure diagram of an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of an FE enhancement module according to an embodiment of the present invention;

[0051] Figure 4 This is a dynamic update flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0052] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0053] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0054] Example 1

[0055] like Figure 1 As shown, this embodiment provides an image forgery tracing method based on graph convolutional modeling and fingerprint library matching, including:

[0056] Extracting spatial domain features and frequency domain features from an input image, and fusing the spatial domain features and frequency domain features to generate comprehensive features;

[0057] Inputting the comprehensive features into a graph convolutional network, generating enhanced forged fingerprint features by constructing a feature relationship graph and aggregating node information;

[0058] Optimizing the distribution of the forged fingerprint features based on a dual marginal contrast loss function so that the feature distance of the same forgery method is reduced and the feature distance of different forgery methods is increased;

[0059] Matching the forged fingerprint features with reference fingerprints in a fingerprint database, and determining the type of forgery method based on the matching distance;

[0060] When an unknown forgery method is detected, the unknown forgery fingerprint corresponding to the unknown forgery method is stored in the fingerprint pool. When the capacity of the fingerprint pool reaches a threshold, a new fingerprint category is generated by clustering and the fingerprint library is updated.

[0061] Specifically, such as Figure 1 As shown, Figure 1 The process of image forgery tracing based on graph convolutional modeling and fingerprint library matching is shown. Training phase (left): spatial domain features and frequency domain features are extracted from real and forged images, and then fused and input into GCN to generate features. , optimize the model parameters through DMC loss. Testing phase (right): Use the trained model to extract test image features, match the category fingerprints in the fingerprint library, and identify known forgery methods. If it is an unseen model, DBSCAN clustering is triggered by the fingerprint pool capacity threshold to generate new category fingerprints, dynamically update the fingerprint library, and support the adaptation of new forgery methods. The entire process of the testing phase specifically includes: obtaining photos from N categories in the dataset, extracting features through the trained network to build a fingerprint library;

[0062] Get the test set image, input the network to obtain the feature, calculate the distance between the feature and the feature in the fingerprint library. Determine its category as the type with the closest feature distance. If it exceeds the threshold , compare the center distance of GAN / DM, classified as unknown_gan / unknown_dm;

[0063] Dynamic update mechanism, feature cache, storage of unseen model features, detection times > T, trigger DBSCAN clustering, generate M fingerprints, update the fingerprint library, and add new categories unknown_gan_1 / unknown_dm_1.

[0064] More specifically, the present invention proposes an image forgery tracing method based on graph convolutional modeling and fingerprint library matching, aiming to solve the problems of insufficient generalization ability, incomplete feature representation, and poor scalability of existing forged image tracing technologies. This method captures the deep patterns of forged fingerprints through multi-domain feature extraction, graph convolutional modeling, fingerprint library matching and dynamic updating, combining spatial and frequency domain features, and achieves efficient tracing of known and unseen forgery methods. Figure 2 Model structure diagram, the method shown is divided into five core modules:

[0065] Multi-domain feature extraction module: extracts spatial and frequency domain features from the input image, generates comprehensive features, and provides multimodal information for subsequent feature modeling.

[0066] Graph convolutional modeling module: Capture the topological relationship between features through the graph convolutional network (GCN), enhance feature distinguishability, and generate deep representations of forged fingerprints.

[0067] Loss function module: Use dual marginal contrast loss (DMC) to optimize feature distribution, shorten the feature distance of the same forgery method, and increase the feature distance of different methods.

[0068] Fingerprint library construction and matching module: Build a reference fingerprint library, achieve forgery traceability through feature matching, and support open set classification (i.e., identify unseen models).

[0069] Dynamic update module: Dynamically updates the fingerprint library based on DBSCAN clustering to adapt to new forgery methods and improve the scalability and practicality of the system.

[0070] As an implementation method of this embodiment, the process of extracting spatial domain features and frequency domain features includes:

[0071] Extracting spatial domain features of the input image through a pre-trained residual network;

[0072] Convert the input image into a grayscale image, extract frequency domain features through discrete cosine transform, and adjust the dimension of the frequency domain features through a feature enhancement module;

[0073] The spatial domain features and the frequency domain features are spliced ​​together to form the comprehensive features.

[0074] Specifically, refer to Figure 2 The image forgery tracing method based on graph convolutional modeling and fingerprint library matching includes a multi-domain feature extraction module, and the steps include:

[0075] a) Feature extraction in spatial domain:

[0076] Input: Input image , resize to a uniform size of 224*224 and perform normalization;

[0077] Processing: Use the pre-trained ResNet50 as the encoder to extract spatial domain features, remove the last fully connected layer of ResNet50, and retain the feature output after global average pooling.

[0078] Output: Spatial domain features in is the batch size, is the feature dimension.

[0079] b) Feature extraction in frequency domain:

[0080] Input: Input image , resize to a uniform size of 224*224 and convert to grayscale;

[0081] Processing: The image is passed through the DCT module to obtain the frequency domain features, and then passed through the feature enhancement module FE to adjust the frequency domain feature dimension and spatial domain feature matching. The FE module is as follows Figure 3 As shown;

[0082] Output: Spatial domain features in is the batch size, is the feature dimension.

[0083] c) Feature fusion:

[0084] The spatial domain features and frequency domain The features are fused and the fused features are obtained by simple direct splicing and fusion. .

[0085] As an implementation method of this embodiment, the process of constructing the feature relationship graph includes:

[0086] Calculating the cosine similarity between nodes based on the comprehensive features to generate an adjacency matrix;

[0087] Adding self-connection to the adjacency matrix and performing symmetric normalization to obtain a normalized adjacency matrix;

[0088] The normalized adjacency matrix and the two-layer graph convolutional network are used to aggregate node information to generate the enhanced forged fingerprint features.

[0089] Specifically, refer to Figure 2 The image forgery tracing method based on graph convolutional modeling and fingerprint library matching includes a graph convolutional modeling module, including:

[0090] Input: Input fusion features , the shape is , where B is the batch size, =4096 is the feature dimension. Fusion feature It is generated by the multi-domain feature extraction module, specifically the spatial domain feature and frequency domain Features are spliced ​​together, that is ;

[0091] Processing: Use graph convolutional network GCN to fuse features Modeling is performed to capture the topological relationship between features, enhance the distinguishability of features, and generate a deep representation of the forged fingerprint. The specific steps are as follows:

[0092] a) Constructing feature relationship graph:

[0093] The fusion features It is regarded as a set of nodes in a graph, where each sample corresponds to a node and the feature vector of the node is , calculate the similarity between nodes and build the adjacency matrix The similarity measure uses cosine similarity, and the formula is:

[0094] ;

[0095] in They are Hedi The feature vector of the sample, Representation node and nodes The similarity between . Perform normalization and add self-connection to enhance the feature contribution of the node itself to obtain a normalized adjacency matrix :

[0096] ;

[0097] in is the identity matrix, indicating self-connection, and further calculation of the degree matrix ,in , and Perform symmetric normalization:

[0098] ;

[0099] Normalized Used in subsequent graph convolution operations to ensure the stability of information propagation.

[0100] b) Graph convolution calculation:

[0101] Use graph convolutional network GCN to feature Update the topological structure representation of the feature by aggregating the information of neighboring nodes. The calculation formula of GCN is:

[0102] ;

[0103] : Normalized adjacency matrix.

[0104] : Input feature matrix, .

[0105] The learnable weight matrix, is the output feature dimension.

[0106] : Activation function, using ReLU activation function, introducing nonlinear transformation.

[0107] In order to enhance the expressive power of the model, a two-layer GCN structure is adopted. The two-layer GCN makes the features of the same forgery method more compact and the features of different methods more dispersed through information propagation and topological structure learning, providing initial differentiation for subsequent fingerprint matching and classification. The formula is expressed as:

[0108] ;

[0109] ;

[0110] in , , output , , Dropout is added after each GCN layer (the dropout rate is set to 0.5) to prevent overfitting and improve the generalization ability of the model.

[0111] Output: Graph convolution enhanced features , Enhanced features capture fusion features The topological relationship of the fingerprints is enhanced, which enhances the distinguishability of forged fingerprints and provides a more robust feature representation for subsequent loss function optimization and fingerprint matching.

[0112] As an implementation method in this embodiment, the optimization process of the dual marginal contrast loss function includes:

[0113] According to the forgery method category label and method similarity label of the sample, the intra-class distance constraint of the same forgery method category and the inter-class distance constraint of different method categories are calculated;

[0114] The distribution of the forged fingerprint features is controlled by preset intra-class margin parameters and inter-class margin parameters.

[0115] Specifically, refer to Figure 2 The image forgery tracing method based on graph convolutional modeling and fingerprint library matching includes a loss function module:

[0116] Input: The input is the enhanced features output by the graph convolutional modeling module , and enter the category label and method similarity labels , which is used to guide the calculation of DMC loss.

[0117] DMC loss calculation: DMC loss optimizes feature distribution through a dual margin comparison mechanism, and the formula is:

[0118] ;

[0119] in, , No. and The feature vector of the sample, :Ruodi and samples belong to the same forgery method, for example, they are all images generated by ProGAN, then ,otherwise . :Ruodi and samples belong to similar method categories, for example, all are GAN-generated images, but not DM-generated images, then ,otherwise . : Feature distance, using Euclidean distance, is calculated as follows:

[0120] ;

[0121] : Double marginal parameters, satisfying , which is used to control the degree of separation of features of different categories:

[0122] a) : Similar method categories, such as the margins of different GAN models, such as ProGAN and StyleGAN, are set to smaller values ,Because the fake fingerprints generated by these methods may have a certain similarity, the ,feature distance does not need to be pushed too far.

[0123] b) : Different method categories, such as GAN and DM margins, are set to larger values ,Because the fake fingerprints generated by GAN and DM are quite different, the ,feature distance needs to be pushed further.

[0124] As an implementation method in this embodiment, the fingerprint library construction and matching process includes:

[0125] Selecting a reference sample set for each known counterfeiting method, extracting the fingerprint of the reference sample set and calculating the class center;

[0126] The minimum Euclidean distance between the test fingerprint and the center of the category is calculated. If the minimum Euclidean distance exceeds a preset threshold, it is determined to be an unknown forgery method.

[0127] Specifically, refer to Figure 2 The image forgery tracing method based on graph convolutional modeling and fingerprint library matching includes a fingerprint library construction and matching module, and the steps include:

[0128] a) Fingerprint library construction:

[0129] A reference subset is selected from the training data for each known forgery method (including real image categories). The training data contains N known categories, each with K images. M images are randomly selected for each category to form the reference subset. The fingerprint of each image in the reference subset is extracted using the trained model. The fingerprint of the mth image of the nth category is recorded as ,in , fingerprint library construction collection ,in is the label of the category. In addition, to support the detection of unseen methods, the fingerprint center of each category is calculated. , further calculate the fingerprint centers of GAN and DM categories for classification of unseen methods:

[0130] ;

[0131] ;

[0132] in is a set of known GAN categories, is the number of known GAN categories.

[0133] b) Fingerprint matching:

[0134] Given a test image , extract its fingerprint through the trained model , calculate the average distance between the test fingerprint and each category:

[0135] ;

[0136] in is the Euclidean distance:

[0137] ;

[0138] Get the distance vector , represents the average distance between the test fingerprint and each known category, and finds the minimum distance , the corresponding categories are:

[0139] ;

[0140] Setting the threshold ,like , the test image is judged as an unseen method and further classified as unknown_GAN or unknown_DM, and the test fingerprint is calculated. Distance to known GAN and DM fingerprint centers:

[0141] ;

[0142] , it is classified as unseen GAN; otherwise it is classified as unseen DM.

[0143] As an implementation method of this embodiment, the process of updating the fingerprint database includes:

[0144] Storing the unknown forged fingerprints in a fingerprint pool according to categories, and performing density clustering on the fingerprint pool when the capacity of the fingerprint pool reaches a preset capacity threshold;

[0145] A new fingerprint category is generated according to the clustering result, and the category center in the fingerprint database is updated.

[0146] Specifically, refer to Figure 4 The image forgery tracing method based on graph convolution modeling and fingerprint library matching includes a dynamic update module, and the steps include:

[0147] Calculate the minimum distance ,when Less than or equal to the threshold When the update ends, if Greater than threshold When the unknown fingerprint is classified, the unknown fingerprint According to the classification results, store them in the corresponding fingerprint pool and , set the threshold If the capacity of a fingerprint pool is greater than the threshold T, the update is terminated directly. The dynamic update strategy is triggered and DBSCAN clustering is used for the fingerprint pool that exceeds the threshold to obtain new unknown categories. For example, for Fingerprint pool clustering, obtain , , , add new categories such as fingerprint library, clear , update the fingerprint centers of GAN and DM categories at the same time, and end the update. The specific update is as follows Figure 4 shown.

[0148] Based on this, embodiments of the present invention provide a method for tracing image forgeries based on graph convolutional modeling and fingerprint library matching. This method comprehensively captures the explicit and implicit features of forged images by fusing spatial and frequency domain features. Combined with a graph convolutional network to model the topological relationships between features, it significantly improves the ability to identify deep patterns in forged fingerprints, addressing the incomplete representation problem of traditional methods that rely on local features. By optimizing feature distribution using a dual-margin contrast loss function, the system can distinguish known forgery methods and identify unknown generation techniques (such as novel GANs or diffusion models). Combining fingerprint library matching with a dynamic threshold determination mechanism, it achieves efficient classification in open environments. Furthermore, a lightweight graph convolutional architecture and a dynamic update module reduce computational complexity while maintaining detection accuracy, supporting large-scale real-time applications. DBSCAN clustering automatically expands the fingerprint library without manual intervention or model retraining, significantly reducing maintenance costs. The system can generate new categories and update the fingerprint library based on clustering results of unknown forged fingerprints, rapidly adapting to evolving forgery techniques and possessing long-term practicality and scalability in scenarios such as digital content authenticity verification and copyright protection.

[0149] Example 2

[0150] In this embodiment, a computer terminal device is provided, including:

[0151] one or more processors;

[0152] a memory, coupled to the processor, for storing one or more programs;

[0153] When the one or more programs are executed by the one or more processors, the one or more processors implement the methods in the above embodiments.

[0154] In this embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the method in the above embodiment is implemented.

[0155] In this embodiment, an electronic device is further provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to execute the method in the above embodiment.

[0156] The above program can be executed in a processor or stored in a memory (or computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and can be implemented using any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0157] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more blocks can be implemented by different modules corresponding to different steps.

[0158] This embodiment provides such a device or system. The system is called an image forgery tracing system based on graph convolutional modeling and fingerprint library matching, and includes:

[0159] A multi-domain feature extraction module is used to extract spatial domain features and frequency domain features from the input image, and fuse the spatial domain features and frequency domain features to generate comprehensive features;

[0160] A graph convolution modeling module is configured to receive the comprehensive features, construct a feature relationship graph, and aggregate node information to generate enhanced forged fingerprint features;

[0161] a loss optimization module for optimizing the distribution of the forged fingerprint features based on a double marginal contrast loss function, so as to reduce the feature distance of the same forgery method and increase the feature distance of different forgery methods;

[0162] A fingerprint matching module, configured to match the forged fingerprint features with reference fingerprints in a fingerprint library and determine the type of forgery method based on the matching distance;

[0163] The dynamic update module is used to store the unknown forged fingerprint corresponding to the unknown forgery method into the fingerprint pool when an unknown forgery method is detected, and to generate a new fingerprint category by clustering and update the fingerprint library when the fingerprint pool capacity reaches a threshold.

[0164] As an implementation method of this embodiment, the multi-domain feature extraction module includes:

[0165] A spatial domain feature extraction unit, used to extract spatial domain features of an input image through a pre-trained residual network;

[0166] A frequency domain feature extraction unit is used to convert the input image into a grayscale image, extract frequency domain features through discrete cosine transform, and adjust the dimension of the frequency domain features through a feature enhancement module;

[0167] The feature fusion unit is used to combine the spatial domain features and the frequency domain features to form the comprehensive features.

[0168] As an implementation method in this embodiment, the graph convolution modeling module includes:

[0169] an adjacency matrix generating unit, configured to calculate the cosine similarity between nodes based on the comprehensive features and generate an adjacency matrix;

[0170] a normalization processing unit, configured to add self-connection to the adjacency matrix and perform symmetric normalization to obtain a normalized adjacency matrix;

[0171] A graph convolution calculation unit is used to aggregate node information using the normalized adjacency matrix and a two-layer graph convolution network to generate the enhanced forged fingerprint feature.

[0172] As an implementation method in this embodiment, the loss optimization module includes:

[0173] A constraint calculation unit, configured to calculate, based on the forgery method category label and the method similarity label of the sample, the intra-class distance constraint of the same forgery method category and the inter-class distance constraint of different method categories;

[0174] A parameter control unit is used to control the distribution of the forged fingerprint features through preset intra-class marginal parameters and inter-class marginal parameters.

[0175] As an implementation method of this embodiment, the fingerprint matching module includes:

[0176] A fingerprint library construction unit, configured to select a reference sample set for each known counterfeiting method, extract the fingerprint of the reference sample set, and calculate the category center;

[0177] The distance determination unit is used to calculate the minimum Euclidean distance between the test fingerprint and the center of the category, and if the minimum Euclidean distance exceeds a preset threshold, it is determined to be an unknown forgery method.

[0178] As an implementation method of this embodiment, the dynamic update module includes:

[0179] A fingerprint pool management unit, configured to store the unknown forged fingerprints into a fingerprint pool according to categories and monitor the capacity of the fingerprint pool;

[0180] The clustering updating unit is configured to perform density clustering on the fingerprint pool when the capacity of the fingerprint pool reaches a preset capacity threshold, generate a new fingerprint category according to the clustering result, and update the category center in the fingerprint library.

[0181] The system or device is used to implement the functions of the method in the above-mentioned embodiment. Each module in the system or device corresponds to each step in the method, which has been explained in the method and will not be repeated here.

[0182] Through the above implementation, the problem of image forgery tracing based on graph convolution modeling and fingerprint library matching in the related art is solved, thereby ensuring that the problems existing in the existing technology are solved.

[0183] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for tracing the source of image forgery based on graph convolutional modeling and fingerprint library matching, characterized by: The following steps are involved: Extracting spatial domain features and frequency domain features from an input image, and fusing the spatial domain features and frequency domain features to generate a comprehensive feature; Inputting the comprehensive features into a graph convolutional network, generating enhanced forged fingerprint features by constructing a feature relationship graph and aggregating node information; Optimizing the distribution of the forged fingerprint features based on a dual marginal contrast loss function so that the feature distance of the same forgery method is reduced and the feature distance of different forgery methods is increased; Matching the forged fingerprint features with reference fingerprints in a fingerprint database, and determining the type of forgery method based on the matching distance; When an unknown forgery method is detected, the unknown forgery fingerprint corresponding to the unknown forgery method is stored in the fingerprint pool. When the capacity of the fingerprint pool reaches a threshold, a new fingerprint category is generated by clustering and the fingerprint library is updated.

2. The method according to claim 1, characterized in that The process of extracting spatial domain features and frequency domain features includes: Extracting spatial domain features of the input image through a pre-trained residual network; Convert the input image into a grayscale image, extract frequency domain features through discrete cosine transform, and adjust the dimension of the frequency domain features through a feature enhancement module; The spatial domain features and the frequency domain features are spliced ​​together to form the comprehensive features.

3. The method according to claim 1, characterized in that The process of constructing the feature relationship graph includes: Calculating the cosine similarity between nodes based on the comprehensive features to generate an adjacency matrix; Adding self-connection to the adjacency matrix and performing symmetric normalization to obtain a normalized adjacency matrix; The normalized adjacency matrix and the two-layer graph convolutional network are used to aggregate node information to generate the enhanced forged fingerprint features.

4. The method according to claim 1, wherein The optimization process of the dual marginal contrast loss function includes: According to the forgery method category label and method similarity label of the sample, the intra-class distance constraint of the same forgery method category and the inter-class distance constraint of different method categories are calculated; The distribution of the forged fingerprint features is controlled by preset intra-class margin parameters and inter-class margin parameters.

5. The method according to claim 1, wherein The construction and matching process of the fingerprint library includes: Selecting a reference sample set for each known counterfeiting method, extracting the fingerprint of the reference sample set and calculating the class center; The minimum Euclidean distance between the test fingerprint and the center of the category is calculated. If the minimum Euclidean distance exceeds a preset threshold, it is determined to be an unknown forgery method.

6. The method according to claim 1, characterized in that The process of updating the fingerprint database includes: Storing the unknown forged fingerprints in a fingerprint pool according to categories, and performing density clustering on the fingerprint pool when the capacity of the fingerprint pool reaches a preset capacity threshold; A new fingerprint category is generated according to the clustering result, and the category center in the fingerprint database is updated.

7. An image forgery tracing system based on graph convolutional modeling and fingerprint library matching, characterized by: The system comprises: A multi-domain feature extraction module is used to extract spatial domain features and frequency domain features from the input image, and fuse the spatial domain features and frequency domain features to generate comprehensive features; A graph convolution modeling module is configured to receive the comprehensive features, construct a feature relationship graph, and aggregate node information to generate enhanced forged fingerprint features; a loss optimization module for optimizing the distribution of the forged fingerprint features based on a double marginal contrast loss function, so as to reduce the feature distance of the same forgery method and increase the feature distance of different forgery methods; A fingerprint matching module, configured to match the forged fingerprint features with reference fingerprints in a fingerprint library and determine the type of forgery method based on the matching distance; The dynamic update module is used to store the unknown forged fingerprint corresponding to the unknown forgery method into the fingerprint pool when an unknown forgery method is detected, and to generate a new fingerprint category by clustering and update the fingerprint library when the fingerprint pool capacity reaches a threshold.

8. The system according to claim 7, characterized in that The multi-domain feature extraction module includes: A spatial domain feature extraction unit, used to extract spatial domain features of an input image through a pre-trained residual network; A frequency domain feature extraction unit is used to convert the input image into a grayscale image, extract frequency domain features through discrete cosine transform, and adjust the dimension of the frequency domain features through a feature enhancement module; The feature fusion unit is used to combine the spatial domain features and the frequency domain features to form the comprehensive features.

9. A computer terminal device, characterized in that: include: one or more processors; a memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image forgery tracing method based on graph convolution modeling and fingerprint library matching according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image forgery tracing method based on graph convolutional modeling and fingerprint library matching according to any one of claims 1 to 6 is implemented.

Images