Editing tool tracing method and system for multi-tool editing image
By employing a dual-domain block feature enhancement and error feedback training data sampling strategy, the problem of trace coverage in multi-tool edited images is solved, enabling efficient source tracing of multi-tool edited images and improving the accuracy and robustness of source tracing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-21
AI Technical Summary
Existing technologies cannot effectively identify all the tools involved in the editing of images using multiple tools, and multi-step editing leads to trace coverage, resulting in decreased traceability performance. Feature extraction lacks specificity and cannot distinguish overlapping traces.
A dual-domain block feature enhancement approach based on the spatial and frequency domains is adopted. Frequency domain block features are extracted through sliding window discrete cosine transform and combined with spatial domain block features. A cross-attention mechanism is used to achieve dual-domain feature fusion. Furthermore, the model training is optimized through an error feedback training data sampling strategy, and the sampling weights are dynamically adjusted to achieve progressive learning.
It accurately captures region-specific traces from different tools, mitigating the decline in traceability caused by trace coverage in multi-step editing, and improving the traceability accuracy and robustness of multi-tool edited images.
Smart Images
Figure CN121904567A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and image forensics, specifically to a method for tracing the source of all editing tools involved in multi-tool image editing, particularly applicable to tool tracing tasks for face images after multi-step, multi-tool editing. Background Technology
[0002] With the development of generative artificial intelligence technologies (such as generative adversarial networks and diffusion models), the ease of use and functional strength of image editing tools have been greatly improved. Ordinary users can complete highly realistic image editing with simple prompts. Currently, 71% of social media images involve artificial intelligence generation or editing, and these editing operations are often completed through a combination of multiple tools (i.e., multi-tool editing).
[0003] Existing related technologies can be mainly divided into two categories:
[0004] Image forgery detection technology identifies forged images by extracting traces of tampering in the image (such as spatial domain inconsistencies and frequency domain feature anomalies). Typical methods include deep learning models based on residual networks (ResNet) and efficient networks (EfficientNet), as well as F³-Net which focuses on frequency features and FatFormer based on the Transformer architecture.
[0005] Single-tool source tracing technology: The core is to identify the single tool used in image editing. It relies on model fingerprint extraction or active watermarking technology. Typical methods include DNA-Det (a single-tool source tracing model based on contrastive learning) and RepMix (a robust source tracing method for synthetic images). It assumes that the image has only been edited by a single tool and completes source tracing by learning the unique traces of a single tool.
[0006] Furthermore, some existing technologies attempt to handle multi-step editing scenarios, but they have limitations. For example, existing Modelship Attribution tasks rely on comparing images before and after two consecutive edits, but in real-world scenarios, it is difficult to obtain the editing history of images, thus limiting the practicality of this approach.
[0007] The problems and shortcomings of existing technologies include:
[0008] (1) Existing single-tool assumptions are out of touch with real-world scenarios. Existing source tracing technologies are all based on the core assumption that "images are edited by only a single tool". However, in real-world scenarios, users often combine multiple tools for progressive editing (such as face swapping before beautification, skin adjustment before makeup), or multiple people use different tools to independently edit the same image, resulting in overlapping and covering of tool traces. Existing methods cannot identify all the tools involved.
[0009] (2) Multi-step editing leads to trace decay. Since subsequent editing operations will cover or dilute the traces of the previous tools, the detection performance of the single-tool tracing method will drop sharply as the number of editing steps increases.
[0010] (3) Feature extraction lacks specificity: Existing methods do not fully consider the characteristics of traces in multi-tool editing, that is, the traces of different tools in the spatial domain (pixel-level differences) and frequency domain (frequency domain coefficient distribution) are regionally specific (e.g., face-swapping tools mainly affect the facial identity area, while beauty tools focus on the skin area). Single-domain feature or global feature extraction cannot distinguish overlapping traces.
[0011] Therefore, there is an urgent need to study a tool tracing method for multi-tool image editing to solve the core problems existing in the above-mentioned technologies. Summary of the Invention
[0012] To address the following core issues in existing technologies: overcoming the limitations of single-tool tracing and achieving the identification of all participating tools in multi-tool image editing; mitigating the performance degradation caused by trace coverage in multi-step editing; and accurately extracting the specific traces of different tools in the spatial and frequency domains and distinguishing overlapping traces, this application proposes a tool tracing method and system for multi-tool image editing.
[0013] In a first aspect, embodiments of this application provide a method for tracing the source of editing tools for multi-tool image editing, the method comprising:
[0014] Feature extraction and fusion steps: For the image dataset to be traced, based on the dual-domain block feature enhancement method of spatial and frequency domains, frequency domain block features are extracted by sliding window discrete cosine transform, combined with spatial domain block features, and a cross-attention mechanism is used to achieve dual-domain feature fusion and output fused features;
[0015] Model training data optimization steps: Based on the error feedback-based training data sampling strategy, the training samples are divided into multiple subsets according to the number of editing steps, and the sampling weights are dynamically adjusted based on the error feedback of each subset to optimize the source model training.
[0016] Editing tool output steps: Input the fused features into the optimized tracing model to realize the tracing of each editing tool.
[0017] In this embodiment of the invention, the above feature extraction and fusion step includes:
[0018] Input image preprocessing steps: Standardize the image to be traced;
[0019] Sliding window discrete cosine transform steps: Using an appropriate sliding window size and sliding step size, perform frequency domain transformation on the source image to be traced to obtain a set of frequency domain patches;
[0020] Dual-domain feature extraction steps: A pre-trained image representation network is used as the backbone network to extract spatial domain features from the source image to be traced; the same backbone network is used to extract frequency domain features from the source image to be traced.
[0021] In this embodiment of the invention, the above feature extraction and fusion step includes:
[0022] Position encoding and cross-attention fusion steps: Flatten the spatial domain features and frequency domain features, and superimpose the learnable position encoding on the position of each element in the flattened vector to obtain spatial domain position-aware features and frequency domain position-aware features; perform bidirectional cross-attention computation on the spatial domain position-aware features and frequency domain position-aware features, and perform enhancement operations.
[0023] Feature fusion steps: The enhanced spatial features and frequency domain features are concatenated, statistical dimensions are obtained through convolutional layers, and then the attention mechanism is used to fuse the features to obtain the final dual-domain fused feature F, which is then fed into the pre-trained source tracing model composed of a multi-label classification network.
[0024] In this embodiment of the invention, the above-mentioned model training data optimization step includes:
[0025] Sample grouping steps: Divide the training set into multiple subsets from easy to difficult according to the length of the editing steps;
[0026] Initialization parameter steps: Set the initial values for subset sampling weights;
[0027] Iterative training and weight adjustment steps: Samples are drawn from each subset according to the sampling weights, and the multi-label classifier is trained iteratively; the average binary cross-entropy error of each subset is calculated; the sampling weight values are adjusted according to the error to achieve the optimal weights.
[0028] In this embodiment of the invention, the above-mentioned editing tool output step includes:
[0029] Classifier training steps: Input the fused features F into the pre-trained source tracing model's multi-label classifier based on a fully connected layer to obtain the prediction vector of the probability of each tool's use;
[0030] Threshold judgment steps: The judgment is based on the preset probability threshold. The i-th dimension of the prediction vector output represents the probability that the i-th tool participates in the sample editing. If the model output is higher than the probability threshold, it is determined that the i-th tool has participated in the editing.
[0031] Secondly, embodiments of this application provide an editing tool tracing system for multi-tool image editing, employing the editing tool tracing method described above for multi-tool image editing. The system includes:
[0032] Feature extraction and fusion module: For the image dataset to be traced, based on the dual-domain block feature enhancement method of spatial and frequency domains, frequency domain block features are extracted by sliding window discrete cosine transform, combined with spatial domain block features, and a cross-attention mechanism is used to achieve dual-domain feature fusion and output fused features;
[0033] Model training data optimization module: Based on the error feedback training data sampling strategy, the training samples are divided into multiple subsets according to the number of editing steps, and the sampling weights are dynamically adjusted based on the error feedback of each subset to optimize the training of the source model.
[0034] Editing tool output module: This module integrates the source tracing model optimized by fusing feature inputs to enable source tracing for each editing tool.
[0035] In this embodiment of the invention, the feature extraction and fusion module includes:
[0036] Input image preprocessing module: performs standardization processing on the image to be traced;
[0037] Sliding window discrete cosine transform module: Using an appropriate sliding window size and sliding step size, the source image to be traced is transformed in the frequency domain to obtain a set of frequency domain patches;
[0038] Dual-domain feature extraction module: It uses a pre-trained image representation network as the backbone network to extract spatial domain features from the source image to be traced; it also uses the same backbone network to extract frequency domain features from the source image to be traced.
[0039] The position encoding and cross-attention fusion module: flattens the spatial domain features and frequency domain features, and superimposes the learnable position encoding on the position of each element in the flattened vector to obtain spatial domain position-aware features and frequency domain position-aware features; bidirectional cross-attention computation is performed on the spatial domain position-aware features and frequency domain position-aware features, and enhancement operations are performed.
[0040] Feature fusion module: The enhanced spatial features and frequency domain features are concatenated, statistical dimension is obtained through convolutional layers, and then feature fusion is performed using an attention mechanism to obtain the final dual-domain fused features, which are then fed into the pre-trained source tracing model composed of a multi-label classification network.
[0041] In this embodiment of the invention, the above-mentioned model training data optimization module includes:
[0042] Sample grouping module: Divides the training set into multiple subsets from easy to difficult based on the length of the editing steps;
[0043] Initialization parameter module: Sets the initial values of subset sampling weights;
[0044] Iterative training and weight adjustment module: Samples are extracted from each subset according to the sampling weights, and the multi-label classifier is trained iteratively; the average binary cross-entropy error of each subset is calculated; and the sampling weight values are adjusted according to the error to achieve the optimal weights.
[0045] Thirdly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an editing tool tracing method for multi-tool image editing.
[0046] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the editing tool tracing method for multi-tool image editing as described above.
[0047] Compared with existing technologies, it has the following outstanding advantages:
[0048] 1) The method and system of this invention propose a dual-domain block feature enhancement method based on the spatial domain and frequency domain. Frequency domain block features are extracted by sliding window discrete cosine transform and combined with spatial domain block features. A cross-attention mechanism is used to achieve dual-domain feature fusion, thereby accurately capturing the region-specific traces of different tools (such as identity region traces of face-swapping tools and skin region traces of beauty tools).
[0049] 2) The method and system design of this invention are based on an error feedback training data sampling strategy. The training samples are divided into multiple subsets such as simple, medium and difficult according to the number of editing steps. The sampling weights are dynamically adjusted based on the error feedback of each subset to achieve progressive learning from easy to difficult. This enables the model to maintain high robustness in multi-step editing scenarios and alleviates the decline in traceability caused by trace coverage. Attached Figure Description
[0050] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0051] Figure 1 This is a schematic diagram of the editing tool tracing method for multi-tool image editing according to the present invention;
[0052] Figure 2 This is a flowchart illustrating the origins of the editing tool in this invention.
[0053] Figure 3 This is a schematic diagram of the image editing tool tracing system for multi-tool image editing according to the present invention;
[0054] Figure 4 This is a schematic diagram of the computer hardware of the present invention. Detailed Implementation
[0055] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.
[0056] It should also be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.
[0057] It should also be understood that, in various embodiments of the present invention, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0058] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0059] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0060] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0061] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0062] To make the above-mentioned features and effects of the present invention clearer and easier to understand, specific embodiments are described below in conjunction with the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are merely illustrative. The scope of protection of the present invention is not limited to the disclosed embodiments, but is defined by the appended claims.
[0063] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.
[0064] The inventors of this invention discovered during their research on image forensics technology that the core defects of existing technologies stem from two key limitations: first, they have not broken through the inherent assumption of "single-tool editing" and have ignored the universality of multi-tool combined editing; second, they have not designed feature extraction mechanisms for the "regional specificity" and "inter-domain differences" of multi-tool traces, and at the same time, they lack progressive learning strategies to deal with trace coverage problems.
[0065] Further research shows that the traces from multi-tool editing are not entirely indistinguishable: different tools exhibit varying editing intensity across different facial regions (eyes, nose, skin, lips) (e.g., face-swapping tools show significantly higher L2 distances in facial contour regions than beauty tools), and their feature distributions in the spatial and frequency domains are unique (e.g., OpenCV shows higher cosine similarity in the frequency domain than diffusion model-based tools). Furthermore, the trace coverage from multi-tool editing exhibits a "gradual" nature; simple samples (few-step editing) show low trace overlap, while complex samples (multi-step editing) show high overlap. Training the model in an order from "easy to difficult" can improve the ability to distinguish complex traces.
[0066] Based on the above findings, the inventors proposed a dual-domain image block-level feature extraction method based on the spatial and frequency domains to accurately capture the region-specific traces of different tools. Furthermore, they introduced an error-based learning strategy to enable the model to progressively master the ability to distinguish traces from simple to complex editing samples, ultimately achieving tool tracing for multi-tool image editing.
[0067] The system of this application embodiment will be described in detail below with reference to specific embodiments:
[0068] like Figure 1 As shown, this invention proposes a method for tracing the source of image editing tools using multiple tools. The core of this multi-tool image editing source tracing method and system comprises two parts: a dual-domain block feature enhancement module and a training data sampling strategy based on error feedback. The overall process is as follows: Figure 1 As shown. Training samples are provided through a dataset; a dual-domain block feature enhancement module extracts accurate features; a training data sampling strategy based on error feedback optimizes model training; and finally, a multi-label classifier outputs all tools involved in the editing process. The method includes:
[0069] Feature extraction and fusion step 101: For the image dataset to be traced, based on the dual-domain block feature enhancement method of spatial domain and frequency domain, frequency domain block features are extracted by sliding window discrete cosine transform, combined with spatial domain block features, and a cross-attention mechanism is used to achieve dual-domain feature fusion and output fused features;
[0070] Model training data optimization step 102: Based on the error feedback training data sampling strategy, the training samples are divided into multiple subsets according to the number of editing steps, and the sampling weights are dynamically adjusted based on the error feedback of each subset to optimize the source model training.
[0071] Editing tool output step 103: Input the fused features into the optimized tracing model to realize the tracing of each editing tool.
[0072] In this embodiment of the invention, the feature extraction and fusion step 101 includes:
[0073] Input image preprocessing steps: Standardize the image to be traced;
[0074] Sliding window discrete cosine transform steps: Using an appropriate sliding window size and sliding step size, perform frequency domain transformation on the source image to be traced to obtain a set of frequency domain patches;
[0075] Dual-domain feature extraction steps: A pre-trained image representation network is used as the backbone network to extract spatial domain features from the source image to be traced; the same backbone network is used to extract frequency domain features from the source image to be traced.
[0076] Position encoding and cross-attention fusion steps: Flatten the spatial domain features and frequency domain features, and superimpose the learnable position encoding on the position of each element in the flattened vector to obtain spatial domain position-aware features and frequency domain position-aware features; perform bidirectional cross-attention computation on the spatial domain position-aware features and frequency domain position-aware features, and perform enhancement operations.
[0077] Feature fusion steps: The enhanced spatial features and frequency domain features are concatenated, statistical dimensions are obtained through convolutional layers, and then the attention mechanism is used to fuse the features to obtain the final dual-domain fused feature F, which is then fed into the pre-trained model composed of a multi-label classification network.
[0078] In this embodiment of the invention, the above-mentioned model training data optimization step 102 includes:
[0079] Sample grouping steps: Divide the training set into multiple subsets from easy to difficult according to the length of the editing steps;
[0080] Initialization parameter steps: Set the initial values for subset sampling weights;
[0081] Iterative training and weight adjustment steps: Samples are drawn from each subset according to the sampling weights, and the multi-label classifier is trained iteratively; the average binary cross-entropy error of each subset is calculated; the sampling weight values are adjusted according to the error to achieve the optimal weights.
[0082] In this embodiment of the invention, the above-mentioned editing tool output step 103 includes:
[0083] Classifier training steps: Input the fused features F into the pre-trained model's multi-label classifier based on fully connected layers to obtain the prediction vector of the probability of using each tool;
[0084] Threshold judgment steps: The judgment is based on the preset probability threshold. The i-th dimension of the prediction vector output represents the probability that the i-th tool participates in the sample editing. If the model output is higher than the probability threshold, it is determined that the i-th tool has participated in the editing.
[0085] Specifically, such as Figure 2 As shown, the method for tracing the source of editing tools for multi-tool image editing according to the present invention specifically includes:
[0086] Step 1: Enhancement of image block features based on spatial and frequency domains
[0087] 2.1 Input Image Preprocessing: Image I to be traced s Standardize the image to a uniform size of H×W×C, where H and W are the height and width of the image, respectively, and C is the number of image channels.
[0088] 2.2 Sliding Window Discrete Cosine Transform: Using an appropriate sliding window size and sliding step size, for I...s Perform frequency domain transformation to obtain frequency domain patch set I f .
[0089] 2.3 Dual-domain feature extraction:
[0090] Spatial domain feature extraction: A pre-trained image representation network (e.g., Xception network) is used as the backbone network for I... s Extracting spatial features F s ;
[0091] Frequency domain feature extraction: Using the same backbone network, for I f Extracting frequency domain features F f ;
[0092] 2.4 Location Encoding and Cross-Attention Fusion:
[0093] For F s and F f Flattening is performed (flattening the elements of each row in the matrix and concatenating them into a one-dimensional vector). Learnable positional codes are then superimposed on the positions of each element in the flattened vector to obtain the spatial domain position-aware feature F. sp and frequency domain position-aware features F fp ;
[0094] For F sp and F fp Bidirectional cross-attention computation is employed, specifically, spatial features are made attenuated by frequency domain features (F... s ' = F s +Reshape (Attn (F sp , F fp , F fp Frequency domain features focus on spatial features (F) f ' = F fp + Reshape (Attn(F fp , F sp , F sp Attention is calculated using the formula Attn(Q,K,V)= (d) k (where F is the key vector dimension), Reshape is the flattening operator that makes the attention calculation result similar to F. sp or F fp Maintain consistency in dimensions;
[0095] 2.5 Feature Fusion: The enhanced spatial features F s 'and frequency domain features F fThe features are concatenated, and the statistical dimensions are obtained through convolutional layers. Then, the attention mechanism is used to fuse the features to obtain the final dual-domain fused feature F, which is then fed into the final multi-label classification network.
[0096] Step 2: Training data sampling strategy based on error feedback
[0097] 2.1 Sample grouping: The training set is divided into M subsets S1 to SM according to the length of the editing steps, from simple (few editing steps) to difficult (many editing steps) (assuming that all training image samples have undergone at most M editing steps).
[0098] 2.2 Initialization Parameters: Set the subset sampling weight P={ The initial weights are all set to 1 / M (i.e., uniform distribution).
[0099] 2.3 Iterative Training and Weight Adjustment:
[0100] Each training round: Samples are drawn from each subset according to the sampling weight P, and a multi-label classifier is trained (the output dimension is consistent with the number of tools).
[0101] Error calculation: Calculate the average binary cross-entropy error e for each subset Si. i ;
[0102] Weight update: based on error e i Adjust the sampling weights using the following formula: (when hour), (when (when); where t takes a suitable value, and the reasonable range of values is t>=2. This is the model's age parameter, with an initial value of 0.5.
[0103] Parameter iteration: after each training round By incrementing by 1, the sampling ratio of difficult samples is gradually increased to achieve progressive learning from easy to difficult.
[0104] Step 3: Multi-label classification and source tracing output
[0105] 3.1 Classifier Training: Input the fused features F output from the module in step 1 into a multi-label classifier based on a fully connected layer to obtain the prediction vector of the probability of using each tool (output dimension M, corresponding to M classes of image editing tools).
[0106] 3.2 Threshold Judgment: Judgment is made based on a preset probability threshold (generally greater than or equal to 0.5). The i-th dimension of the prediction vector output represents the probability that the i-th tool participates in the editing of the sample. If the model output is higher than the threshold, it is determined that the i-th tool has participated in the editing.
[0107] In specific embodiments of the present invention, the extended implementation scheme includes:
[0108] The method of the present invention can also be modified to be applicable to the following situations:
[0109] 1. Tool expansion and transformation: For example, if the original source tracing model only performs local tiling, and you want to extend it to whole-image filter tools, you can retrain the existing model by expanding the dataset based on the existing tools, thereby achieving source tracing for more tools;
[0110] 2. Domain Expansion and Variation: For example, if the original source tracing model is only applicable to face images, and we want to extend it to landscape images, we only need to supplement the training set with samples of landscape images before and after editing, and continue to train the original source tracing model according to the method described in this invention to achieve multi-tool editing source tracing in more image domains.
[0111] As described above, the system and method of the present invention can be implemented well.
[0112] Compared with the prior art, the present invention has the following outstanding advantages and beneficial effects:
[0113] The present invention has the following beneficial technical effects: credit allocation is achieved through reverse fine-tuning, and the achievement of the final task goal is passed to each subtask through the reverse fine-tuning process; it is not necessary to design a reward function for the entire complex long-term task, but only to design a reward function for each relatively simple subtask, which reduces the design difficulty of the reward function; subtask linking and joint training are achieved through state transfer, avoiding failure due to unknown states; no additional expert demonstration data is required.
[0114] Secondly, embodiments of this application provide an editing tool tracing system for multi-tool image editing, employing the editing tool tracing method described above for multi-tool image editing. Figure 3 As shown, the system includes:
[0115] Feature extraction and fusion module 201: For the image dataset to be traced, based on the dual-domain block feature enhancement method of spatial and frequency domains, frequency domain block features are extracted by sliding window discrete cosine transform, combined with spatial domain block features, and dual-domain feature fusion is achieved by cross-attention mechanism to output fused features;
[0116] Model training data optimization module 202: Training data sampling strategy based on error feedback, which divides the training samples into multiple subsets according to the number of editing steps, and dynamically adjusts the sampling weights based on the error feedback of each subset to optimize model training;
[0117] Editing tool output module 203: Input the fused features into the optimized model to enable the traceability of each editing tool.
[0118] In this embodiment of the invention, the feature extraction and fusion module 201 includes:
[0119] Input image preprocessing module: performs standardization processing on the image to be traced;
[0120] Sliding window discrete cosine transform module: Using an appropriate sliding window size and sliding step size, the source image to be traced is transformed in the frequency domain to obtain a set of frequency domain patches;
[0121] Dual-domain feature extraction module: It uses a pre-trained image representation network as the backbone network to extract spatial domain features from the source image to be traced; it also uses the same backbone network to extract frequency domain features from the source image to be traced.
[0122] The position encoding and cross-attention fusion module: flattens the spatial domain features and frequency domain features, and superimposes the learnable position encoding on the position of each element in the flattened vector to obtain spatial domain position-aware features and frequency domain position-aware features; bidirectional cross-attention computation is performed on the spatial domain position-aware features and frequency domain position-aware features, and enhancement operations are performed.
[0123] Feature fusion module: The enhanced spatial features and frequency domain features are concatenated, and the feature fusion is performed by statistical dimension through convolutional layers and attention mechanism to obtain the final dual-domain fused feature F, which is then fed into the pre-trained model composed of a multi-label classification network.
[0124] In this embodiment of the invention, the above-mentioned model training data optimization module 202 includes:
[0125] Sample grouping module: Divides the training set into multiple subsets from easy to difficult based on the length of the editing steps;
[0126] Initialization parameter module: Sets the initial values of subset sampling weights;
[0127] Iterative training and weight adjustment module: Samples are extracted from each subset according to the sampling weights, and the multi-label classifier is trained iteratively; the average binary cross-entropy error of each subset is calculated; and the sampling weight values are adjusted according to the error to achieve the optimal weights.
[0128] Thirdly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned method for tracing the source of editing tools for multi-tool image editing.
[0129] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the editing tool tracing method for multi-tool image editing as described above.
[0130] In addition, combined Figure 1 The editing tool tracing method for multi-tool image editing described in this application embodiment can be implemented by an electronic device, such as a computer device. Figure 4 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application.
[0131] In some embodiments, the computer device may further include a communication interface 83 and a bus 80. For example, Figure 4 As shown, the processor 81, memory 82, and communication interface 83 are connected through bus 80 and complete communication with each other.
[0132] Specifically, the processor 81 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.
[0133] The memory 82 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 81.
[0134] The processor 81 reads and executes computer program instructions stored in the memory 82 to implement any of the editing tool tracing methods for multi-tool image editing in the above embodiments.
[0135] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0136] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A method for tracing the source of editing tools for multi-tool image editing, characterized in that, The method includes: Feature extraction and fusion steps: For the image dataset to be traced, based on the dual-domain block feature enhancement method of spatial and frequency domains, frequency domain block features are extracted by sliding window discrete cosine transform, combined with spatial domain block features, and a cross-attention mechanism is used to achieve dual-domain feature fusion and output fused features; Model training data optimization steps: Based on the error feedback training data sampling strategy, the training samples are divided into multiple subsets according to the number of editing steps, and the sampling weights are dynamically adjusted based on the error feedback of each subset to optimize the source model training; Editing tool output steps: Input the fused features into the optimized tracing model to realize the tracing of each editing tool.
2. The method for tracing the source of editing tools for multi-tool image editing according to claim 1, characterized in that, The feature extraction and fusion step includes: Input image preprocessing steps: Standardize the image to be traced; Sliding window discrete cosine transform steps: Using an appropriate sliding window size and sliding step size, perform frequency domain transformation on the image to be traced to obtain a set of frequency domain patches; Dual-domain feature extraction steps: A pre-trained image representation network is used as the backbone network to extract spatial domain features from the image to be traced; the same backbone network is used to extract frequency domain features from the image to be traced.
3. The method for tracing the source of editing tools for multi-tool image editing according to claim 2, characterized in that, The feature extraction and fusion step includes: Position encoding and cross-attention fusion steps: Flatten the spatial domain features and frequency domain features, and superimpose learnable position codes on the position of each element in the flattened vector to obtain spatial domain position-aware features and frequency domain position-aware features; perform bidirectional cross-attention calculation on the spatial domain position-aware features and frequency domain position-aware features, and perform enhancement operations. Feature fusion steps: The enhanced spatial features and frequency domain features are concatenated, statistical dimensions are obtained through convolutional layers, and then the attention mechanism is used to fuse the features to obtain the final dual-domain fused features, which are then fed into the pre-trained source tracing model composed of a multi-label classification network.
4. The method for tracing the source of editing tools for multi-tool image editing according to claim 1, characterized in that, The model training data optimization step includes: Sample grouping steps: Divide the training set into multiple subsets from easy to difficult according to the length of the editing steps; Initialization parameter steps: Set the initial values for subset sampling weights; Iterative training and weight adjustment steps: Samples are drawn from each subset according to the sampling weights, and the multi-label classifier is trained iteratively; the average binary cross-entropy error of each subset is calculated; the sampling weight values are adjusted according to the error to achieve the optimal weights.
5. The method for tracing the source of editing tools for multi-tool image editing according to claim 1, characterized in that, The editing tool output steps include: Classifier training steps: Input the fused features into a pre-trained source tracing model based on a fully connected layer multi-label classifier to obtain the prediction vector of the probability of use of each tool; Threshold determination step: Based on a preset probability threshold, the model output in the i-th dimension of the prediction vector represents the probability that the i-th tool participated in the editing of the sample. If the model output is higher than the probability threshold, it is determined that the i-th tool participated in the editing.
6. A tool tracing system for multi-tool image editing, employing the tool tracing method for multi-tool image editing as described in any one of claims 1-5, characterized in that, The system includes: Feature extraction and fusion module: For the image dataset to be traced, based on the dual-domain block feature enhancement method of spatial and frequency domains, frequency domain block features are extracted by sliding window discrete cosine transform, combined with spatial domain block features, and a cross-attention mechanism is used to achieve dual-domain feature fusion and output fused features; Model training data optimization module: Based on the error feedback training data sampling strategy, the training samples are divided into multiple subsets according to the number of editing steps, and the sampling weights are dynamically adjusted based on the error feedback of each subset to optimize the source tracing model training; Editing tool output module: Inputs the fused features into the optimized tracing model to realize the tracing of each editing tool.
7. The editing tool tracing system for multi-tool image editing according to claim 6, characterized in that, The feature extraction and fusion module includes: Input image preprocessing module: performs standardization processing on the image to be traced; Sliding window discrete cosine transform module: Using an appropriate sliding window size and sliding step size, the image to be traced is transformed in the frequency domain to obtain a set of frequency domain patches; Dual-domain feature extraction module: A pre-trained image representation network is used as the backbone network to extract spatial domain features from the image to be traced; the same backbone network is used to extract frequency domain features from the image to be traced. The position encoding and cross-attention fusion module: flattens the spatial domain features and frequency domain features, and superimposes learnable position codes on the position of each element in the flattened vector to obtain spatial domain position-aware features and frequency domain position-aware features; bidirectional cross-attention calculation is performed on the spatial domain position-aware features and frequency domain position-aware features, and enhancement operations are performed. Feature fusion module: The enhanced spatial features and frequency domain features are concatenated, statistical dimension is obtained through convolutional layers, and then feature fusion is performed using an attention mechanism to obtain the final dual-domain fused feature F, which is then fed into the pre-trained source tracing model composed of a multi-label classification network.
8. The editing tool tracing system for multi-tool image editing according to claim 6, characterized in that, The model training data optimization module includes: Sample grouping module: Divides the training set into multiple subsets from easy to difficult based on the length of the editing steps; Initialization parameter module: Sets the initial values of subset sampling weights; Iterative training and weight adjustment module: Samples are extracted from each subset according to the sampling weights, and the multi-label classifier is trained iteratively; the average binary cross-entropy error of each subset is calculated; and the sampling weight values are adjusted according to the error to achieve the optimal weights.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the editing tool tracing method for multi-tool image editing as described in any one of claims 1-5.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the editing tool tracing method for multi-tool image editing as described in any one of claims 1-5.