A method and device for realizing image tracing

Abstract

The present invention discloses a method and apparatus for image provenance tracing, comprising: training a provenance graph construction model based on information about modification relationship identification and modification direction judgment of feature graphs of input image pairs; initializing a known basic database to obtain a relational database containing modification relationships between data stored in a graph structure and basic database data; performing global feature extraction on target images for image provenance tracing to obtain a set of candidate images; and constructing a directed provenance graph for the candidate image set based on the provenance graph construction model, performing image modification relationship backtracing on the candidate image set. The present invention can accurately analyze modification relationships between images to construct a provenance graph, providing a precise and efficient image provenance implementation solution suitable for large-scale scenarios.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for realizing image tracing. Background Art

[0002] The increasing popularity of social media platforms and image editing tools has enabled digital images to be not only rapidly and widely disseminated, but also to be extensively and arbitrarily modified. While this phenomenon brings convenience, it also raises a series of serious issues, such as the spread of false information, copyright infringement, and the proliferation of forged images. To address this, researchers have begun exploring ways to use technology to track and verify the origin and modification history of images, a process known as image provenance analysis.

[0003] Currently, the process of image traceability analysis usually includes the following two steps:

[0004] (1) Source Filtering: Filter out all images that have modification relationships with the target image from a large-scale image;

[0005] (2) Provenance graph construction: Generate a provenance graph that can represent the relationship between images for all images with modification relationships to analyze the image's derivative history.

[0006] The corresponding image traceability analysis process not only helps to identify forged images, but also provides strong support for copyright protection and responsibility tracing.

[0007] However, existing traceability analysis technologies have many flaws, including:

[0008] (1) Insufficient traceability accuracy:

[0009] The first existing technology uses only similarity to filter images, which will miss images that are similar but have low similarity, which will inevitably lead to insufficient filtering accuracy.

[0010] The second existing technology uses a minimum spanning tree to connect images using similarity as a metric to construct an image traceability graph, but it cannot exclude irrelevant images, resulting in inaccurate traceability graph construction.

[0011] The third existing technology is mostly unable to generate directed traceability graphs, and the only directed graph construction methods have low accuracy;

[0012] 2) High computational complexity: Existing technologies use local features to calculate the similarity between images, and when constructing the traceability graph, it is necessary to calculate the similarity between images pairwise, resulting in The computational complexity makes it difficult to apply to large-scale scenarios.

[0013] In view of this, the present invention is proposed. Summary of the Invention

[0014] The purpose of the present invention is to provide a method and device for realizing image tracing, which can effectively improve the tracing accuracy and reduce the computational complexity, thereby solving the above-mentioned technical problems existing in the prior art.

[0015] The purpose of the present invention is achieved through the following technical solutions:

[0016] A method for realizing image provenance tracing includes:

[0017] Based on the information of modification relationship identification and modification direction judgment of the feature graph of the input image pair, the traceability graph construction model is trained; and the known basic database is initialized to obtain a relational database containing modification relationships between data stored in the form of a graph structure and basic database data;

[0018] The target image to be traced is filtered to obtain a set of candidate images, and a model is constructed based on the traceability graph. A directed traceability graph is constructed for the candidate image set and image modification relationship backtracing is performed to complete the image traceability operation for the target image.

[0019] The process of training the traceability graph construction model includes:

[0020] After extracting features from the input image pair, the feature map is obtained, and the feature map is subjected to modification relationship identification to determine whether there is a modification relationship between the image pairs; and the identification result is input into a cross entropy loss function to calculate the modification relationship identification loss; and

[0021] Perform modification direction judgment on the feature map, and input the judgment result into the cross entropy loss function to calculate the modification direction judgment loss;

[0022] The sum of the modification relationship identification loss and the modification direction judgment loss is taken as the total loss, and the parameters of the traceability graph construction model are updated through back propagation until the total loss meets the predetermined requirements, thereby completing the training of the traceability graph construction model.

[0023] The relational database is obtained to include a global feature image node set and an edge set, wherein:

[0024] Performing global feature extraction on images contained in the image dataset in the basic database to obtain a global feature image node set;

[0025] Traverse all image nodes, establish directed edges between image nodes with modification relationships, and obtain an edge set.

[0026] The process of constructing the directed traceability graph includes:

[0027] Performing global feature extraction on the target image, and obtaining a candidate image set based on the relational database, wherein the candidate image set is composed of a predetermined number of images that are most similar in the results of the global feature extraction;

[0028] Building a model based on the traceability graph, determining the modification relationship between the target image and all candidate images in the candidate image set, and obtaining a corresponding traceability graph based on the modification relationship to implement directed traceability graph construction, wherein the traceability graph includes a traceability graph node set and a traceability graph edge set;

[0029] The image modification relationship backtracking process includes:

[0030] Traverse all images except the target image in the source graph node set and input them into the relational database as query nodes for relational query, so as to query all images and corresponding modification relationships that have modification relationships with the image corresponding to the query node;

[0031] Add all images obtained by the query to the traceability graph node, and add all modification relationships obtained by the query to the relationship node;

[0032] The traceability graph is updated based on the obtained traceability graph nodes and the relationship nodes, completing the image modification relationship backtracing process.

[0033] The process of constructing the directed traceability graph based on the modification relationship includes:

[0034] Traverse the candidate image set, and input the obtained candidate images and target images into the traceability graph construction model for processing, wherein:

[0035] When there is a modification relationship between the image pairs, the candidate image is added to the source graph node set, and the modification direction between the images is determined. Based on the result of the modification direction determination, a directed edge between the target image and the candidate image is determined; then, the directed edge is added to the source graph edge set;

[0036] After completing the traversal operation of the candidate image set, the provenance graph is obtained based on the obtained provenance graph node set and provenance graph edge set.

[0037] A device for realizing image tracing, comprising:

[0038] A model training unit, configured to train a traceability graph construction model based on information on modification relationship identification and modification direction judgment of a feature graph of an input image pair;

[0039] A relational database construction unit is used to initialize a known basic database to obtain a relational database containing a modification relationship between stored data in a graph structure and basic database data;

[0040] The image tracing unit is used to extract global features of the target image to be traced, and query the relational database based on the features to obtain a set of candidate images; and build a model based on the tracing graph, construct a directed tracing graph for the candidate image set and trace back the image modification relationship to complete the image tracing operation for the target image.

[0041] The model training unit includes:

[0042] A modification relationship identification module is configured to extract features from the input image pair to obtain the feature map, perform modification relationship identification on the feature map to determine whether a modification relationship exists between the image pairs; and input the identification result into a cross entropy loss function to calculate the modification relationship identification loss; and

[0043] A modification direction judgment module is used to perform modification direction judgment on the feature map and input the judgment result into a cross entropy loss function to calculate the modification direction judgment loss;

[0044] The provenance graph construction model training module is used to take the sum of the modification relationship identification loss and the modification direction judgment loss as the total loss, update the parameters of the provenance graph construction model through back propagation until the total loss meets the predetermined requirements, and complete the training of the provenance graph construction model.

[0045] The relational database obtained by the relational database construction unit includes a global feature image node set and an edge set, wherein:

[0046] Performing global feature extraction on images contained in the image dataset in the basic database to obtain a global feature image node set;

[0047] Traverse all image nodes, establish directed edges between image nodes with modification relationships, and obtain an edge set.

[0048] The image tracing unit includes: a tracing filtering module, a directed tracing graph construction module, and a modification relationship backtracking module, wherein:

[0049] The source tracing filtering module is used to extract global features of the target image to be traced, and query the relational database based on the features to obtain a set of candidate images, wherein the set of candidate images is composed of a predetermined number of images that are most similar to the results of the global feature extraction;

[0050] The directed traceability graph construction module includes:

[0051] Obtaining the candidate image set obtained by the source tracing filtering module;

[0052] Building a model based on the traceability graph, determining the modification relationship between the target image and all candidate images in the candidate image set, and obtaining a corresponding traceability graph based on the modification relationship to implement directed traceability graph construction, wherein the traceability graph includes a traceability graph node set and a traceability graph edge set;

[0053] The modification relationship backtracking module includes:

[0054] Traverse all images except the target image in the source graph node set and input them into the relational database as query nodes for relational query, so as to query all images and corresponding modification relationships that have modification relationships with the image corresponding to the query node;

[0055] Add all images obtained by the query to the traceability graph node, and add all modification relationships obtained by the query to the relationship node;

[0056] The traceability graph is updated based on the obtained traceability graph nodes and the relationship nodes, completing the image modification relationship backtracing process.

[0057] The process of constructing the directed traceability graph based on the modification relationship includes:

[0058] Traverse the candidate image set, and input the obtained candidate images and target images into the traceability graph construction model for processing, wherein:

[0059] When there is a modification relationship between the image pairs, the candidate image is added to the source graph node set, and the modification direction between the images is determined. Based on the result of the modification direction determination, a directed edge between the target image and the candidate image is determined; then, the directed edge is added to the source graph edge set;

[0060] After completing the traversal operation of the candidate image set, the provenance graph is obtained based on the obtained provenance graph node set and provenance graph edge set.

[0061] Compared with the existing technology, the present invention provides an image tracing implementation method and device, which can accurately filter out all images that have a modification relationship with the target image from a large-scale image, and can accurately analyze the modification relationship between images to construct a tracing map; further, it can also optimize the computational complexity of the existing technology in the filtering stage and the tracing map construction stage, thereby providing an accurate and efficient image tracing technology solution suitable for large-scale scenarios, solving the problems of insufficient image tracing accuracy and high computational complexity in the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0063] Figure 1 A flowchart of a method provided by an embodiment of the present invention;

[0064] Figure 2 A schematic structural diagram of a device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the specific content of the present invention. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments, and do not constitute a limitation of the present invention. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0066] First, the following terms may be used in this article:

[0067] The term “and / or” means that either or both of them can be realized at the same time. For example, X and / or Y includes both “X” or “Y” and “X and Y”.

[0068] The terms "include," "comprises," "contains," "has," or other similar expressions should be interpreted as non-exclusive. For example, "including certain technical features (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products, or manufactured articles)" should be interpreted as including not only the technical features explicitly listed, but also other technical features known in the art that are not explicitly listed.

[0069] The term "consisting of" excludes any technical features not explicitly listed. If used in a claim, this term renders the claim closed, excluding any technical features other than those explicitly listed, except for conventional impurities associated with them. If this term appears only in a clause of a claim, it limits only the elements explicitly listed in that clause; elements listed in other clauses are not excluded from the claim as a whole.

[0070] Unless otherwise specified or limited, the terms "mounted," "connected," "connect," and "fixed" should be interpreted broadly. For example, they can refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediary; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in this document based on specific circumstances.

[0071] When concentration, temperature, pressure, size or other parameters are expressed in the form of a numerical range, the numerical range should be understood to specifically disclose all ranges formed by the pairing of any upper limit, lower limit, or preferred value within the numerical range, regardless of whether the range is explicitly stated. For example, if a numerical range of "2 to 8" is stated, the numerical range should be interpreted as including ranges of "2 to 7," "2 to 6," "5 to 7," "3 to 4 and 6 to 7," "3 to 5 and 7," "2 and 5 to 7," etc. Unless otherwise specified, the numerical ranges stated herein include both their endpoints and all integers and fractions within the numerical range.

[0072] The terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings and are only for the convenience and simplification of description, and do not explicitly or implicitly indicate that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore should not be understood as a limitation to this document.

[0073] In order to solve the problems of insufficient tracing accuracy and excessive computational overhead in existing image tracing technologies, the present invention provides a method and device for implementing image tracing. To facilitate understanding of the present invention, its specific implementation methods will be described in detail below with reference to the accompanying drawings.

[0074] The implementation method of an image tracing provided by the present invention mainly includes the following during the implementation process: first, based on the modification relationship identification and modification direction judgment information of the feature graph of the input image pair, the tracing graph construction model is trained; and the known basic database is initialized to obtain a relational database containing modification relationships between data stored in the form of a graph structure and basic database data; thereafter, global feature extraction is performed on the target image to be traced, and based on the tracing graph construction model and the relational database, a directed tracing graph is constructed and image modification relationship backtracing is performed to complete the image tracing operation for the target image. The specific implementation process of the image tracing method provided by the present invention is as follows: Figure 1As shown, it may include but is not limited to the following processing steps:

[0075] Step 11: training and establishing a traceability graph construction model;

[0076] The provenance graph construction model is a multi-task classification model. The training and establishment of the provenance graph construction model can include the following processing steps:

[0077] (11-1) performing feature extraction on the input image pair to obtain a feature map of the input image;

[0078] The corresponding feature map acquisition process includes: inputting a pair of images , perform feature extraction. During the feature provision process, the size of the pair of images needs to be processed according to the input requirements of the traceability graph construction model, and converted into corresponding tensors and normalized. After that, the processed image pair (i.e., a pair of images) can be input into the feature extraction model to obtain the corresponding feature map: ;

[0079] (11-2) performing modification relationship identification processing on the feature graph, which may specifically include:

[0080] The corresponding feature maps of the image pairs are spliced ​​and sampled to obtain the processed feature maps: ;

[0081] After processing, the feature map Perform modification relationship identification processing to determine whether there is a modification relationship between the image pairs. The modification relationship identification result is used to input the cross entropy loss function to calculate the modification relationship identification loss. ;

[0082] (11-3) Determining the modification direction of the feature map, which may specifically include:

[0083] Map the feature maps separately to obtain one-dimensional representation: ;

[0084] Concatenate the one-dimensional representations to obtain the concatenated one-dimensional representation: ; and the one-dimensional representation after splicing Perform modification direction judgment to judge the modification direction between images, and the result of the modification direction judgment is used to input the cross entropy loss function to calculate the modification direction judgment loss ;

[0085] (11-4) Conduct iterative training of the traceability graph construction model;

[0086] The sum of the modification relationship identification loss and the modification direction judgment loss is taken as the total loss, that is, the total loss of the traceability graph construction model is: , update the parameters of the traceability graph construction model through back propagation, repeat the above steps until the loss tends to be stable, that is, until the total loss meets the predetermined requirements, and complete the training of the traceability graph construction model.

[0087] Step 12: Initialize the database to obtain a relational database. ;

[0088] Specifically, the process is to extract features from all images in the base image dataset (i.e., the basic database), model the modification relationships of all images, and then store them in a graph database to obtain the relational database D. The graph database is a relational database that can store data and relationships between data in a graph structure and perform queries, and may include but is not limited to Neo4j, ArangoDB, OrientDB, etc. The relational database may include a global feature image node set and an edge set. The process of obtaining the relational database may include:

[0089] (12-1) Preprocess the images contained in the basic database, that is, adjust the image size according to the input requirements of the feature extraction model, convert the image into tensor form and perform normalization;

[0090] (12-2) Extracting global features of the image; that is, inputting the processed image into a feature extraction model to obtain its global features. The feature extraction model can be any model or algorithm that can extract global features of the image and measure the similarity between images based on the extracted features, including but not limited to the following algorithms or models: image fingerprinting, image hashing, or image retrieval;

[0091] (12-3) Model the relationship between data and model the image data into a graph structure;

[0092] Store the global features of a single image as nodes , get the node set ; That is, extracting global features from the images contained in the image dataset in the basic database to obtain a set of global feature image nodes; and,

[0093] Traverse all image nodes, establish directed edges between image nodes with modification relationships and obtain edge sets, that is: traverse all image nodes, establish directed edges between image nodes with modification relationships, and form edge sets Specifically, for the image pair , if the image By image Modified, then Add from point to Directed edges The image relationship in the image base can be all known modification relationships between data, or can be obtained through analysis of the traceability graph construction model constructed above;

[0094] (12-4) Set the nodes together and edge sets Stored in the database as a graph structure .

[0095] After completing the above processing and obtaining the traceability graph construction model and relational database, the target image to be traced can be subjected to corresponding image traceability processing based on the model. The processing may specifically include:

[0096] Step 13: perform image tracing and filtering processing, that is, preprocess the target image to obtain a set of candidate images;

[0097] Perform global feature extraction on the target image to obtain the processed target image ,Adjust the image size according to the input requirements of the feature extraction model, and convert the image into tensor form and perform normalization;

[0098] The processed image Input feature extraction model to obtain global feature ;

[0099] Based on the relational database, a candidate image set is obtained, and the candidate image set is composed of the first predetermined number of images that are most similar in the results of the global feature extraction; Entering a relational database Before querying, get The most similar images constitute the candidate image set .

[0100] Step 14: construct a directed traceability graph;

[0101] The processing process mainly involves building a model based on the traceability graph, determining the modification relationship between the target image and all candidate images in the candidate image set, and obtaining a corresponding traceability graph based on the modification relationship to achieve directed traceability graph construction. Specifically, the relationship between the target image and all candidate images is analyzed. If a modification relationship exists, a directed edge is established; if not, the candidate image is discarded. The corresponding processing process may include:

[0102] (14-1) Process the target image according to the input requirements of the traceability graph model and candidate image set , adjust the image size, convert it into a tensor, and perform normalization;

[0103] (14-2) Add the processed target image to the traceability graph node set ;

[0104] (14-3) Traversing the candidate image set, inputting the obtained candidate images and target images into the traceability graph construction model for processing, wherein:

[0105] When there is a modification relationship between the image pairs, the candidate image is added to the source graph node set, and the modification direction between the images is determined. Based on the result of the modification direction determination, a directed edge between the target image and the candidate image is determined; then, the directed edge is added to the source graph edge set;

[0106] The processing described in (14-3) above may specifically include:

[0107] Traverse the candidate image set , get candidate images , the target image and candidate images Input the traceability graph to build the model. If there is a relationship between the image pairs, the candidate image is added to the traceability graph node set. , and further determine the modification direction between images. If the candidate image is modified from the target image, the node corresponding to the image will be Add a line from point to Directed edges , add the directed edge to the traceability graph edge set ; If there is no modification relationship, the candidate image is discarded;

[0108] That is, after completing the traversal operation of the candidate image set, the traceability graph is obtained based on the obtained traceability graph node set and traceability graph edge set, that is, the traceability graph obtained after completing the relationship analysis between the target image and all candidate images Contains the target image and the candidate image set The images in the image that have a relationship with the target image, and the modified relationship between these images and the target image.

[0109] Step 15, performing image modification relationship backtracking processing;

[0110] (15-1) Traverse the edge set of the traceability graph ,Initialize all images in the set (except the target image) as query nodes;

[0111] (15-2) Inputting the query node into the relational database Performing a relationship query to query all images that have a modification relationship with the image corresponding to the query node; and correspondingly obtaining all images that have a modification relationship with the query node and the corresponding modification relationship may include:

[0112] For query nodes (images) , there may be two relationships between this node in the traceability graph maintained in the database: 1) Image Modified from other images, even multiple times: ;2)Image Modify to generate other images, and the generated images can also be modified again to generate new images: ; You can use the following query statements including but not limited to: MATCH p=(startNode:v)- -> (endNode)RETURN p UNION MATCH p=(endNode) - ->(startNode:v) RETURN p;

[0113] (15-3) Add all the images obtained from the query to the traceability graph node and add all the modified relationships obtained from the query to the relationship node ; Based on the obtained traceability graph nodes and the relationship nodes, the traceability graph is updated to complete the image modification relationship backtracing process;

[0114] That is, after completing step 15, you can get the traceability diagram , which contains all images that have modification relationships with the target image, as well as all modification relationships between all images. The traceability graph can represent the complete derivative history of the target image.

[0115] It can be seen that the above-mentioned image tracing implementation method provides a corresponding large-scale image tracing analysis implementation scheme, which can accurately filter out all images that have a modification relationship with the target image from a large-scale image, and can accurately analyze the modification relationship between images to construct a tracing map. In addition, it can also optimize the computational complexity of the existing technology in the filtering stage and the tracing map construction stage, that is, it provides an accurate and efficient image tracing technology solution suitable for large-scale scenarios.

[0116] The following describes in detail an implementation method of an image tracing device provided by an embodiment of the present invention.

[0117] The embodiment of the present invention provides a device for tracing the source of a model image. Figure 2 As shown, it can mainly include the following processing units and processing modules:

[0118] (1) a model training unit, configured to train a traceability graph construction model based on information on modification relationship identification and modification direction judgment of a feature graph of an input image pair;

[0119] The model training unit is specifically used to train the traceability graph construction model: the traceability graph construction model is a multi-task classification model , which specifically includes a feature extraction module , modify the relationship identification module , modify the direction judgment module And the corresponding traceability graph to build a model training module; the input of the model training unit is a pair of images ;

[0120] Feature extraction module , used for feature extraction, including:

[0121] Build a model based on the traceability diagram The input requires processing the image size, converting it into a tensor, and normalizing it; and inputting the image into the feature extraction model Get the corresponding feature map: ;

[0122] Modify the relationship identification module The processing performed includes:

[0123] The image is spliced ​​and sampled to obtain the processed feature map: , and the processed feature map The modification relationship identification module determines whether there is a modification relationship between the image pairs. The modification relationship identification module is a classification task, and its output (i.e., the judgment result) is used to input the cross entropy loss function to calculate the modification relationship identification loss. ;

[0124] Modify the direction judgment module The processing performed includes:

[0125] Map the feature maps separately to obtain one-dimensional representation: , and then concatenate the one-dimensional representations to obtain the concatenated one-dimensional representation: ;

[0126] Next, the concatenated one-dimensional representation Input the modification direction judgment module to judge the modification direction between images. The modification direction judgment module is a classification task, and the output (i.e., the judgment result) is used to input the cross entropy loss function to calculate the modification direction judgment loss. ;

[0127] The traceability graph construction model training module is used to perform model iterative training, that is, the sum of the modification relationship identification loss and the modification direction judgment loss is taken as the total loss: , update the parameters of the traceability graph construction model through back propagation, repeat the above steps until the loss tends to be stable, that is, until the total loss meets the predetermined requirements, and complete the training of the traceability graph construction model.

[0128] (2) a relational database construction unit, used to initialize a known basic database to obtain a relational database containing modification relationships between data stored in a graph structure and data in the basic database; wherein:

[0129] The relational database construction unit is specifically used to initialize the relational database, that is, to extract features from all images in the base image dataset (i.e., the basic database set), and then to model all image modification relationships and store them in a graph database. The relational database can be any relational database that can store data and relationships between data in a graph structure and can be queried, including but not limited to Neo4j, ArangoDB, OrientDB, etc. The specific processing performed by the unit may include:

[0130] Preprocess the images contained in the basic database set, that is, adjust the image size according to the input requirements of the feature extraction model, convert the image into a tensor form and normalize it; then, global feature extraction of the image can be performed, including: inputting the processed image into the feature extraction model Get the global representation from the feature extraction model It can be any model or algorithm that can extract global features of an image and measure the similarity between images, including but not limited to the following algorithms or models: image fingerprinting, image hashing, and image retrieval;

[0131] After feature extraction is completed, the relationship between the data can be modeled and the image data can be modeled into a graph structure, including:

[0132] Perform global feature extraction on the images contained in the image dataset in the basic database to obtain a set of global feature image nodes, that is, store the global representation of a single image as a node , get the image node set ;as well as,

[0133] Traverse all image nodes, establish directed edges between image nodes with modification relationships and obtain edge sets, that is, traverse all image nodes, establish directed edges between image nodes with modification relationships, and form edge sets Specifically, for the image pair , if the image By image Modified, then Add from point to Directed edges ; Among them, the image relationship in the basic database can be all known modification relationships between data, or the corresponding modification relationship can be obtained by analyzing the model constructed by the above-mentioned traceability diagram;

[0134] The node collection and edge sets Stored in a relational database as a graph structure , obtain the corresponding relational database;

[0135] (3) An image provenance unit, configured to extract global features from a target image to be traced, query the relational database based on the features to obtain a set of candidate images, and construct a directed provenance graph and perform image modification relationship backtracing on the candidate image set based on the provenance graph construction model, thereby completing the image provenance operation for the target image. Specifically, the unit may include a provenance filtering module, a directed provenance graph construction module, and a modification relationship backtracing module, wherein:

[0136] The source tracing filtering module is used to implement image source tracing filtering, that is, to extract global features of the target image and query the relational database based on the features to obtain a set of candidate images. The set of candidate images is composed of a predetermined number of images that are most similar to the results of the global feature extraction. Specifically, the corresponding filtering process may include:

[0137] First, perform global feature extraction on the target image, that is, obtain the target image , according to the input requirements of the feature extraction model, the image size is adjusted, the image is converted into a tensor form and normalized; then, the processed image Input feature extraction model to obtain global features ;

[0138] After that, image retrieval is performed, that is, the global features Entering a relational database Perform a query to obtain the first predetermined number of images that are most similar to the results of global feature extraction, for example, The most similar images constitute the corresponding candidate image set .

[0139] The processing performed by the directed traceability graph construction module includes: obtaining the candidate image set obtained by the traceability filtering module; constructing a model based on the traceability graph, determining the modification relationship between the target image and all candidate images in the candidate image set, and obtaining a corresponding traceability graph based on the modification relationship to implement directed traceability graph construction;

[0140] Specifically, the directed traceability graph construction module is used to analyze the relationship between the target image and all candidate images. If a modification relationship exists, a directed edge is established; if not, the candidate image is discarded, that is:

[0141] Build a model based on the traceability diagram Input requirements to process the target image and candidate image set Adjust the image size, convert it into a tensor, and perform normalization;

[0142] Add the target image to the source graph node collection ;

[0143] Traverse the candidate image set , get candidate images , the target image and candidate images Input Model If there is a relationship between the image pairs, add the candidate image to the provenance graph node set , and further determine the modification direction between images. If the candidate image is modified from the target image, the node corresponding to the image will be Add a line from point to Directed edges , add the directed edge to the traceability graph edge set ; If there is no modification relationship, the candidate image is discarded;

[0144] The traceability graph obtained after completing the relationship analysis between the target image and all candidate images Contains: target image and candidate image set The images in the image that have a relationship with the target image, and the modified relationship between these images and the target image.

[0145] The modification relationship backtracking module performs the following processing: traversing all images except the target image in the source graph node set and inputting them into the relational database as query nodes for performing a relational query to query all images that have a modification relationship with the image corresponding to the query node; adding all images obtained from the query to the source graph nodes, and adding all modification relationships obtained from the query to the relationship nodes; updating the source graph based on the obtained source graph nodes and the relationship nodes, thereby completing the image modification relationship backtracking process;

[0146] The specific image modification relationship backtracking process may include:

[0147] Traverse the traceability graph node set , initialize the images in the collection (except the target image) as query nodes; input the query nodes into the relational database Perform relationship query to obtain all images that have modification relationships with the node and the corresponding modification relationships; specifically, for the query node (image) , there may be two relationships between this node in the traceability graph maintained in the database:

[0148] Each is an image Modified from other images, even multiple times: ;

[0149] The second is image Modify to generate other images, and the generated images can also be modified again to generate new images: ;

[0150] You can use the following query statements including but not limited to: MATCH p=(startNode:v)- -> (endNode) RETURN p UNION MATCH p=(endNode) - ->(startNode:v) RETURN p;

[0151] Afterwards, all the images obtained from the query are added to the traceability graph node and add all the modified relationships obtained from the query to the relationship node ; The traceability diagram obtained after completing this step The provenance graph includes all images that have modification relationships with the target image, as well as all modification relationships between all images. It can represent the complete derivative history of the target image.

[0152] In summary, the implementation of the technical solution provided by the embodiment of the present invention can produce the following significant beneficial effects compared with the prior art:

[0153] (1) The existing technology only calculates image similarity to perform source tracing filtering, which may miss images that have a modification relationship with the target image but have a low similarity. However, the embodiment of the present invention uses similarity and combines image modification relationships to perform backtracking, returning images that have a modification relationship with the target image but have been missed due to low similarity, thereby improving the accuracy of the present invention in the image filtering stage. It has been verified that compared with the baseline solution, the embodiment of the present invention can achieve an improvement of up to 42.9% in the accuracy of source tracing filtering;

[0154] (2) The existing technology constructs a traceability graph based on the minimum spanning tree, connecting all candidate images based on similarity, which will cause image errors, fail to exclude irrelevant images, and fail to accurately construct a directed traceability graph. However, the embodiment of the present invention designs a data-driven directed traceability graph construction method, which can accurately identify the modification relationship and derivation direction between images, thereby accurately constructing a directed traceability graph for images and providing a solution. It has been verified that the embodiment of the present invention can achieve an accuracy improvement of up to 27.4% in the construction of a directed traceability graph, and can cope with more than 20 types of image tampering attacks;

[0155] (3) The existing technology uses local features to calculate image similarity, and it is necessary to calculate image similarity between two images during composition. Computational complexity. To address this issue, the embodiment of the present invention uses a global representation to reduce the computational overhead of similarity between images. By maintaining the modification relationships between images within the database and using these relationships for backtracking, the repeated computation of image relationships within the database is avoided, thereby reducing the time complexity of constructing the provenance graph. Verification shows that compared to existing technologies, the embodiment of the present invention can achieve a 26.6% improvement in overall provenance accuracy. In terms of time overhead, existing technologies require 12 minutes for a single provenance analysis, while the embodiment of the present invention only requires 4.61 seconds.

[0156] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims. The information disclosed in the background technology section of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or any form of implication that the information constitutes prior art already known to those skilled in the art.

Claims

1. A method for image tracing, characterized in that: include: Based on the modification relationship identification and modification direction judgment information of the feature graph of the input image pair, the traceability graph construction model is trained; and initializing a known basic database to obtain a relational database including modification relationships between data stored in a graph structure and data in the basic database; The target image to be traced is filtered to obtain a set of candidate images, and a model is constructed based on the traceability graph. A directed traceability graph is constructed for the candidate image set, and image modification relationship backtracing is performed to complete the image traceability operation for the target image; wherein, The process of constructing the directed traceability graph includes: Performing global feature extraction on the target image, and obtaining a candidate image set based on the relational database, wherein the candidate image set is composed of a predetermined number of images that are most similar in the results of the global feature extraction; Building a model based on the traceability graph, determining the modification relationship between the target image and all candidate images in the candidate image set, and obtaining a corresponding traceability graph based on the modification relationship to implement directed traceability graph construction, wherein the traceability graph includes a traceability graph node set and a traceability graph edge set; The image modification relationship backtracking process includes: Traverse all images except the target image in the source graph node set and input them into the relational database as query nodes for relational query, so as to query all images and corresponding modification relationships that have modification relationships with the image corresponding to the query node; Add all images obtained by the query to the traceability graph node, and add all modification relationships obtained by the query to the relationship node; The traceability graph is updated based on the obtained traceability graph nodes and the relationship nodes, completing the image modification relationship backtracing process.

2. The method for realizing image tracing according to claim 1, characterized in that: The process of training the traceability graph construction model includes: After extracting features from the input image pair, the feature map is obtained, and the feature map is subjected to modification relationship identification to determine whether there is a modification relationship between the image pairs; and the identification result is input into a cross entropy loss function to calculate the modification relationship identification loss; and Perform modification direction judgment on the feature map, and input the judgment result into the cross entropy loss function to calculate the modification direction judgment loss; The sum of the modification relationship identification loss and the modification direction judgment loss is taken as the total loss, and the parameters of the traceability graph construction model are updated through back propagation until the total loss meets the predetermined requirements, thereby completing the training of the traceability graph construction model.

3. The method for realizing image tracing according to claim 1, characterized in that: The relational database is obtained to include a global feature image node set and an edge set, wherein: Performing global feature extraction on images contained in the image dataset in the basic database to obtain a global feature image node set; Traverse all image nodes, establish directed edges between image nodes with modification relationships, and obtain an edge set.

4. The method for image tracing according to claim 1, 2 or 3, characterized in that: The process of constructing the directed traceability graph based on the modification relationship includes: Traverse the candidate image set, and input the obtained candidate images and target images into the traceability graph construction model for processing, wherein: When there is a modification relationship between the image pairs, the candidate image is added to the source graph node set, and the modification direction between the images is determined. Based on the result of the modification direction determination, a directed edge between the target image and the candidate image is determined; then, the directed edge is added to the source graph edge set; After completing the traversal operation of the candidate image set, the provenance graph is obtained based on the obtained provenance graph node set and provenance graph edge set.

5. A device for realizing image tracing, characterized in that: include: A model training unit, configured to train a traceability graph construction model based on information on modification relationship identification and modification direction judgment of a feature graph of an input image pair; A relational database construction unit is used to initialize a known basic database to obtain a relational database containing a modification relationship between stored data in a graph structure and basic database data; The image tracing unit is used to complete the image tracing operation for the target image and includes: (1) a source tracing filtering module, configured to extract global features of a target image to be traced, and query the relational database based on the features to obtain a set of candidate images, wherein the set of candidate images is composed of a predetermined number of images that are most similar to the results of the global feature extraction; (2) A directed traceability graph construction module, configured to obtain the candidate image set obtained by the traceability filtering module; based on the traceability graph construction model, construct a directed traceability graph for the candidate image set, including: determining the modification relationship between the target image and all candidate images in the candidate image set, and obtaining a corresponding traceability graph based on the modification relationship, thereby realizing directed traceability graph construction; the traceability graph includes a traceability graph node set and a traceability graph edge set; (3) A modification relationship backtracking module is used to traverse all images except the target image in the source graph node set, and input them into the relational database as query nodes for relational query, so as to query all images and corresponding modification relationships that have modification relationships with the image corresponding to the query node; all images obtained by the query are added to the source graph node, and all modification relationships obtained by the query are added to the relationship node; the source graph is updated based on the obtained source graph nodes and the relationship nodes, and the image modification relationship backtracking process is completed.

6. The image tracing device according to claim 5, characterized in that: The model training unit includes: A modification relationship identification module is configured to extract features from the input image pair to obtain the feature map, perform modification relationship identification on the feature map to determine whether a modification relationship exists between the image pairs; and input the identification result into a cross entropy loss function to calculate the modification relationship identification loss; and A modification direction judgment module is used to perform modification direction judgment on the feature map and input the judgment result into a cross entropy loss function to calculate the modification direction judgment loss; The provenance graph construction model training module is used to take the sum of the modification relationship identification loss and the modification direction judgment loss as the total loss, update the parameters of the provenance graph construction model through back propagation until the total loss meets the predetermined requirements, and complete the training of the provenance graph construction model.

7. The image tracing device according to claim 5, characterized in that: The relational database obtained by the relational database construction unit includes a global feature image node set and an edge set, wherein: Performing global feature extraction on images contained in the image dataset in the basic database to obtain a global feature image node set; Traverse all image nodes, establish directed edges between image nodes with modification relationships, and obtain an edge set.

8. The image tracing device according to claim 5, 6 or 7, characterized in that: The process of constructing the directed traceability graph based on the modification relationship includes: Traverse the candidate image set, and input the obtained candidate images and target images into the traceability graph construction model for processing, wherein: When there is a modification relationship between the image pairs, the candidate image is added to the source graph node set, and the modification direction between the images is determined. Based on the result of the modification direction determination, a directed edge between the target image and the candidate image is determined; then, the directed edge is added to the source graph edge set; After completing the traversal operation of the candidate image set, the provenance graph is obtained based on the obtained provenance graph node set and provenance graph edge set.

Images