A multi-round interactive text-guided image editing method based on directed acyclic graph

Abstract

The application discloses a multi-round interactive text-guided image editing method based on a directed acyclic graph, organizes an editing history by using a directed acyclic graph, abstracts each editing result as a state node, supports version rollback, branch exploration and multi-path comparison, designs a rollback truncation and confirmation pruning mechanism, truncates invalid branches during rollback, prunes and releases image data of unnecessary nodes after confirmation, only retains an abstract vector to optimize storage, locates a target area by using multi-modal feature fusion during editing, generates a soft and hard mask constraint, only predicts a local pixel change amount in a residual manner and superimposes the local pixel change amount on an original image to maintain structural consistency, and generates a result which is checked in five dimensions of semantics, structure, identity, local retention and boundary naturalness, combines adaptive threshold and editing force closed-loop adjustment to ensure multi-round iteration convergence and robust generation quality, and realizes traceability, rollback, branching and controllable storage of multi-round editing.

Description

Technical Field

[0001] This invention belongs to the field of image processing and multimodal interaction technology, specifically relating to a multi-round interactive text-guided image editing method based on directed acyclic graphs. Background Technology

[0002] With the development of deep learning and multimodal generation technologies, text-guided image editing methods have gradually become an important direction in the field of image processing. Existing text-guided image editing techniques can typically modify images based on user-inputted text descriptions. However, for images with stringent requirements and complex content, even in multi-round interactive editing scenarios, the following shortcomings remain: Existing methods often lack structured management of the editing history, making it difficult to explicitly record the results of each round of editing, and hindering rollback, switching, and branch exploration during the editing process; existing methods have weak compression and reuse capabilities for historical information, resulting in high storage overhead during long-chain editing processes, and are not conducive to quickly locating historical states through natural language queries; when generating new results, existing editing models tend to reconstruct the entire image over a large area, causing changes in non-target areas and reducing image consistency and stability; furthermore, existing technologies typically offer a relatively singular quality assessment of editing results, lacking a comprehensive verification mechanism for factors such as semantic consistency, structural preservation, subject fidelity, local preservation, and boundary naturalness, resulting in a lack of stable criteria for accepting and rejecting candidate results.

[0003] Therefore, there is an urgent need for an image editing method that can adapt to multi-round interactive editing scenarios and has the capabilities of state management, history retrieval, local constraint editing, and closed-loop quality control, so as to improve the controllability, continuity, and stability of the editing process. Summary of the Invention

[0004] To address the aforementioned issues, this invention discloses a multi-round interactive text-guided image editing method based on a directed acyclic graph (DAG). The method organizes the multi-round editing history using a DAG, supporting rollback, switching, and branch comparison. It locates the target region through multimodal fusion and combines soft and hard mask constraints, modifying only local pixels using residual generation to maintain the original image structure. Furthermore, it employs five-dimensional quality verification and closed-loop adjustment of editing intensity to ensure precise, controllable, and stable convergence of multi-round editing.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A multi-round interactive text-guided image editing method based on directed acyclic graphs includes the following steps:

[0007] S1. Directed Acyclic Graph Construction: Each editing result is abstracted into a state node, and a directed acyclic graph is constructed with the original image as the root node to record the editing order and branching relationship, supporting the switching and comparison of different editing versions;

[0008] S2, Structured Memory and Historical Retrieval: Saves editing instructions, editing constraints, quality scores, parent node references and historical summary vectors for each status node, and generates a fixed-dimensional summary representation through the memory compression module, supporting natural language retrieval of historical edit nodes;

[0009] S3. Rollback and Pruning: After receiving the rollback command, locate the target historical node and truncate the invalid branches after it; after confirming the current result, prune the historical link, retaining only the necessary confirmation nodes and summary information, and then continue to generate new branches from the current node;

[0010] S4. Text-guided local editing: After receiving a new text editing instruction, the current image state and the text instruction are fused and modeled to locate the target area and generate editing constraints. Then, the local change is predicted by the residual generation method and superimposed on the original image to obtain a candidate image.

[0011] S5. Verify the semantic consistency, structural preservation, identity fidelity, local preservation, and boundary naturalness of the candidate image, and determine whether to accept it based on the comprehensive score. If the candidate image does not meet the preset threshold, maintain the current state, adjust the editing intensity parameters, and regenerate the candidate result so that the subsequent editing process converges to the target image.

[0012] Furthermore, the specific method of S1 is as follows:

[0013] First, the directed acyclic graph adopts... To express, among which For a set of state nodes, Let be a set of directed edges, any edge Representing state From state Editing operation Obtain; the state node Corresponding image state Furthermore, directed acyclic graphs allow a single node to correspond to multiple successor nodes to form branch editing paths.

[0014] The root node of the original input image Each subsequent edit generates a new state node, connected to its parent node via directed edges. Since any node can correspond to multiple different edit results, multiple branches can be generated in parallel within the same state to compare the effects of different editing strategies. This structure ensures the traceability of the editing history and provides a foundation for subsequent rollback, pruning, and history reuse.

[0015] Furthermore, to improve subsequent retrieval efficiency, the system can extract global visual features from the original image during the initialization phase and generate an initial summary vector. This summary vector can serve as a structured memory representation of the root node, used for subsequent rapid matching of historical states and semantic backtracking.

[0016] Directed acyclic graph uses To express, among which, Represents a set of state nodes. This represents a set of directed edges. Each state node represents the image state after one edit, and each directed edge represents the editing transition relationship from one state to the next. If the current state is generated from the previous state through an editing operation, then a directed edge is established between the corresponding nodes. Thus, the entire editing history forms a tree-like or graph-like structure, which can represent both linear continuous editing and multi-branch exploratory editing.

[0017] Furthermore, the specific method of S2 is as follows:

[0018] For any state node Record its structured memory information The structured memory information includes editing instructions. Editing constraints Quality score vector Parent node index and compressed history summary vector Editing constraints At least include soft holding area Editable area and the upper limit of pixel variation .

[0019] The memory compression module takes the node sequence on the preceding historical path as input and passes it through an encoder network. Generate a fixed-dimensional summary vector ,satisfy:

[0020]

[0021] in, Represents the summary vector of the parent node. This represents the structured memory information corresponding to the current node. This represents a vector concatenation operation. Using this method, the system can retain key information that can be used for retrieval and recovery without needing to save the complete features of all historical images.

[0022] Step S2, Structured Memory and Historical Retrieval. The system maintains structured memory information for each state node, including editing instructions, editing constraints, quality score vector, parent node reference, and compressed historical summary vector. The compressed historical summary vector is generated by the memory compression module, taking the node sequence on the previous historical path as input, and generating a fixed-dimensional summary representation through the encoder network Emem. It should be noted that in normal continuous editing scenarios (i.e., the user gives new editing instructions round by round without explicitly triggering rollback), the system directly starts the subsequent editing process from the current state node without triggering historical semantic retrieval, so as to avoid unnecessary retrieval overhead and potential error rollback. The historical retrieval function is only activated in the following scenarios: (1) the user explicitly issues a rollback instruction and needs to locate the target historical node; (2) the user describes the intention to reuse a certain historical editing state through natural language; (3) the system detects that the quality of the current editing path is continuously declining and automatically searches for a better historical starting point. In the above retrieval scenario, the system performs semantic parsing on the user's query text, extracting information such as editing intent, object targeting, and spatial constraints. It then maps the query text into a query vector using a natural language encoding model and matches it with the historical summary vector zi of each state node in a directed acyclic graph. The expression for this is: [expression missing]. The specific expression is: [expression missing].

[0023] Next, the history retrieval system receives the natural language query input by the user. and through semantic encoder This is mapped to a query vector, and then compared with the compressed summary vector of each historical node. Calculate cosine similarity:

[0024]

[0025] The top K nodes with the highest similarity are selected as the search results output to support semantic querying and state reuse of historical editing steps.

[0026] When a user inputs a command such as "change the background to a night scene while keeping the person's face unchanged", the system will prioritize searching historical nodes that are semantically related to "night scene background" and "facial integrity", thereby avoiding repeated trial and error.

[0027] Furthermore, the specific method of S3 is as follows:

[0028] After the rollback command is triggered, the system will target the historical node. Activate it as the current editing starting point and its subsequent closure. All nodes in the sequence undergo a truncation operation; where the successor closure... It includes the set of all nodes that can be reached from node sk via a directed path. Nodes that are removed after truncation are marked as invalid nodes or deleted from the set of valid nodes.

[0029] When the user explicitly confirms the current edit result, this node is marked as a confirmation node. Confirmed nodes serve as anchor points on the edit path, receiving the highest retention priority in subsequent pruning operations. When a node has not been accessed for an extended period and is not on the confirmed path, the system can convert it into a compressed node—compressed nodes release the complete image data, retaining only the compressed summary vector. and structured memory information This significantly reduces storage overhead, while still allowing participation in historical semantic retrieval via summary vectors. When a node is removed due to rollback truncation, it is marked as an invalid node and removed from the valid set, releasing the storage resources it occupies. These four states—valid node, confirmed node, compressed node, and invalid node—constitute the complete lifecycle of a state node.

[0030] Furthermore, the specific method of S4 is as follows:

[0031] Current image Input visual encoder Extracting visual feature maps (Right now ), will edit instructions Input text encoder Extracting text features Furthermore, a cross-modal attention mechanism is used to fuse textual features as queries and visual features as keys and values ​​to obtain multimodal fusion features. ,in:

[0032] ;

[0033] ;

[0034] in , , For learnable projection matrices, This is the scaling factor.

[0035] Text features are used to characterize the user's editing intent, while visual features are used to characterize the current image context. Cross-modal attention allows the model to focus on the region related to the instruction, thereby improving the accuracy of local editing.

[0036] Next, based on multimodal fusion features By regional positioning network Predict the category of the target semantic region and the corresponding spatial attention heatmap The attention heatmap is upsampled to the original image resolution to generate a region mask; wherein the mask is soft-preserved. With soft editable masks They respectively satisfy:

[0037] ;

[0038] ;

[0039] in This is the attention heatmap after upsampling.

[0040] Structured editing constraints are generated based on the current image state, editing instructions, and positioning results. The structured editing constraints At least including the upper bound of pixel variation amplitude ,in It is determined by the local neighborhood standard deviation, the global amplitude factor, and the task adaptive adjustment function.

[0041] Furthermore, the system can generate more fine-grained editing constraints, including color constraints, structural constraints, boundary constraints, and identity constraints. Different types of constraints can be applied to different loss terms or control branches in the image generation process, thereby improving the consistency between the editing results and the user's intent.

[0042] In one alternative approach, the system can also dynamically adjust the mask area based on negative or retention requirements in the text instructions. For example, when a user requests "only change the background, not the character," the system will increase the retention weight of the character area; when a user requests "remove the object in the upper right corner," the system will include that area in the editing area and increase the intensity of local repair.

[0043] Next, in step S4, the residual generation network With the current image Multimodal fusion features and editing constraints As input, only pixel-level changes are predicted. And through a soft editable mask The changes are constrained and then superimposed onto the current image to generate candidate edit results. ,in:

[0044] ;

[0045] And to satisfy Pixels Maintain consistency with the current image. The hard-preserving mask employs a collaborative mechanism between two types of masks as follows: The hard-preserving mask applies pixel-level hard constraints to a specific region in an "absolutely invariant" manner, meaning that at a pixel with a mask value of 1, the edited result is strictly consistent with the original image. This mask is explicitly specified by the user or automatically generated by an identity recognition model. The soft-editable mask, on the other hand, controls the intensity and range of editing outside the hard-preserving region in a "soft-response" manner, with its value adaptively adjusting according to the editing intensity parameter. The two are cascaded and collaborate through a secure editable mask, which is equal to the element-wise product of the complements of the soft-editable mask and the hard-preserving mask, thus ensuring that the absolute protection of the hard-preserving region and the adaptive adjustment of the soft-editable region do not conflict.

[0046] This method effectively reduces damage to unedited areas and maintains the stability of the original structure and main appearance by generating only local residuals instead of reconstructing the entire image.

[0047] Furthermore, the specific method of S5 is as follows:

[0048] After candidate images are generated, the system performs quality checks on them. These quality checks include at least semantic consistency detection, structure preservation detection, identity fidelity detection, boundary naturalness detection, and local preservation detection. Semantic consistency is used to determine whether the candidate image meets the text instructions; structure preservation is used to determine whether the image's main outline and spatial relationships are reasonable; identity fidelity is used to determine whether the main features have undergone excessive changes in scenes where the main subject, such as people, vehicles, and buildings, needs to be preserved; boundary naturalness is used to determine whether the transition between edited and unedited areas is smooth; and local preservation detection is used to determine whether non-target areas have been unnecessarily affected.

[0049] Semantic consistency score The structure preservation score is used to measure the semantic matching degree between candidate images and editing instructions, and can be calculated through a pre-trained image-text alignment model. Used to measure the structural stability of a candidate image relative to the current image within a non-target region, defined as: ,in This is a normalized measure of feature differences in the intermediate layers of a visual network; a higher score indicates less disruption to the original image structure. Identity fidelity score. The local preservation score is used to measure the degree to which the identity features of the subject are preserved in a candidate image. It can be obtained by extracting the subject feature vector and calculating the similarity using an identity recognition model. Used to measure the degree of consistency between the preserved region and the non-edited region, defined as = 1− Norm( ),in To maintain normalized pixel differences within the region, a higher score indicates less perturbation in non-target areas. (Boundary naturalness score) Used to measure the smoothness of the boundary between edited and unedited areas, defined as ,in For the boundary region of the soft editable mask, For variance operators, As a normalization function, a higher score indicates a more natural boundary transition.

[0050] The overall quality score Q is obtained by weighted fusion of the scores from the above five categories, and its expression is:

[0051]

[0052] in , , , , These are the weighting coefficients, and By configuring different weights, the system can emphasize semantic accuracy, structural stability, subject fidelity, or boundary naturalness for different editing tasks.

[0053] Next, the system determines the complexity factor of the editing task. , rollback depth factor and editing region factors The acceptance threshold range is dynamically adjusted, where the acceptance threshold... Editing task complexity factor Determined based on the length of the editing command, the number of target attributes, and the depth of the semantic hierarchy; Determined based on the number of backtracking steps relative to the current edit node; The threshold is determined based on the proportion of the target semantic region to the total image area. Based on these factors, the system adaptively adjusts the threshold range to accommodate editing tasks of varying difficulty and modification scales.

[0054] When the overall quality score At that time, the candidate editing results will be... Write the node into the directed acyclic graph as a new valid state node, and update the node set. With edge set ;when If the candidate edit result is rejected and the current state remains unchanged, the editing intensity parameter is adjusted and a new candidate result is generated. In this way, the system achieves a closed-loop linkage between edit generation, quality judgment, and state update.

[0055] Furthermore, to prevent continuous rejections from causing the editing process to stall, the system employs an adaptive adjustment mechanism for editing intensity. When candidate results are rejected consecutively, the sampling intensity of the residual generation network, the upper bound of pixel change amplitude, or the sensitivity of the region mask are gradually adjusted to improve the matching degree between the generated results and the target instructions in subsequent rounds. When a candidate result is accepted, the relevant adjustment parameters are reset to maintain the stability of subsequent editing.

[0056] The beneficial effects of this invention are as follows:

[0057] 1. Structured management of multi-round editing processes using directed acyclic graphs, supporting rollback, switching, and branch editing;

[0058] 2. By using historical summary compression and semantic retrieval mechanisms, storage costs can be reduced and the ability to reuse historical information can be improved;

[0059] 3. By using a constraint-guided local change prediction method, damage to non-target areas can be reduced, and editing stability can be improved;

[0060] 4. Through a multi-dimensional quality verification mechanism, the accuracy and consistency of candidate result acceptance can be enhanced;

[0061] 5. The closed-loop mechanism of generation, verification, and updating can improve the overall controllability of multi-round interactive editing. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0063] Figure 2 This is a periodic transition diagram for state nodes.

[0064] Figure 3 To edit the path diagram of the influence of the force vector. Detailed Implementation

[0065] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0066] As shown in the figure, the multi-round interactive text-guided image editing method based on directed acyclic graphs described in this invention includes the following steps in its overall process:

[0067] Step S1: After the system receives the original image uploaded by the user, it first performs format parsing, size normalization, and necessary feature preprocessing on the image to generate an initial editing state. This initial editing state is written into the directed acyclic graph (DAG) as the root node, serving as the starting point for subsequent rounds of editing.

[0068] The system will generate initial nodes for the original image. The system records basic information related to the node, including but not limited to: image identifier, image content features, upload time, user session identifier, default editing parameters, and initial summary vector. If the original image itself contains multiple objects or multiple salient regions, the system can further generate object-level or region-level feature representations for more precise localization and constraints in subsequent editing stages.

[0069] A directed acyclic graph can be represented as:

[0070]

[0071]

[0072] in, Represents the summary vector of the parent node. This represents a vector concatenation operation. The similarity between the text query vector and the state summary vector can be represented as:

[0073] The top K nodes with the highest similarity are selected as the search results output to support semantic querying and state reuse of historical editing steps.

[0074] The system not only performs searches based on text semantics, but also makes comprehensive decisions by combining the most recent editing trajectory in the current session, the user's explicit rollback points, and the quality score results. For example, when multiple historical states have similar semantics to text instructions, the system prioritizes the state node that is closer to the current session context and has a higher quality score as the basis for editing, in order to reduce error backtracking and semantic drift.

[0075] To make search results more stable, the system can also perform structured parsing of text commands, breaking them down into sub-items such as main operations, object entities, attribute changes, regional restrictions, and retention requirements. These sub-item features are then fused before being used in the search. This avoids mismatches caused by relying solely on single text embeddings.

[0076] Step S3: Rollback Management and State Lifecycle Maintenance. The rollback process is triggered when a user issues an explicit rollback command or the system detects a continuous decline in the quality of the edited path. After receiving the rollback command, the system locates the target historical node. And activate it as the current editing starting point, and also set its subsequent closure. All nodes in the process perform a truncation operation. The subsequent closure... Including all nodes The set of nodes reached via a directed path, the nodes removed after truncation are marked as invalid nodes and removed from the set of valid nodes. Delete it.

[0077] Once the user confirms the current edit result, the system performs hierarchical pruning by backtracking along the main path: only the complete status information (including complete image data) of the K most recent confirmed points is retained, and the remaining historical nodes are automatically converted into compressed nodes, retaining only the compressed summary vector and releasing the complete image data. This mechanism achieves automatic optimization of storage overhead while preserving the traceability of key editing trajectories. In long-term sessions, the system can also prioritize based on the usage frequency, confirmation probability, and rollback probability of each branch: frequently used paths retain more complete context, while infrequent and low-quality branches are preferentially compressed or cleaned up.

[0078] Step S4: The system encodes the image and text instructions corresponding to the current state respectively, and establishes the correspondence between the two through a multimodal fusion module. This fusion process is used to identify the image region pointed to by the text, the object to be modified, and the modification method, and outputs editing constraint information that can be used to control image generation.

[0079] Image features can be extracted by a visual encoder, and text features can be extracted by a language encoder. Then, a cross-modal attention mechanism is used to interactively compute these two types of features. Through attention weights, the system can assign higher responses to text-related regions in the image, thereby locating the target editing region.

[0080] The system can generate and edit heatmaps based on the multimodal fusion results. And further converted into a soft editable mask. and soft hold mask :

[0081]

[0082] ;

[0083] in, This indicates the pixel location. The mask is used to constrain the subsequent generation process, ensuring that editing operations mainly occur within the target area, while areas not indicated for modification remain as unchanged as possible.

[0084] After obtaining the above editing constraints, the system generates candidate images based on the current directed acyclic image. This generation process can be implemented using a diffusion model, conditional generation network, local inpainting network, or other image generation models. To reduce the impact on non-target regions, the system prioritizes generating changes within local regions and uses a mask to restrict the changes to the target area.

[0085] Candidate images can be represented as:

[0086]

[0087]

[0088] in, Indicates the current image. Indicates local changes. This represents element-wise multiplication. Using this method, the system can perform targeted modifications while preserving the main structure of the original image.

[0089] The system weights and combines the scores from the above multiple sources to form a comprehensive score Q:

[0090]

[0091] All quality scores have been normalized, with values ​​ranging from [0, 1], and all scores are assigned in a "higher is better" direction. The semantic consistency score measures the degree of semantic matching between the candidate image and the editing instructions; a higher value indicates greater semantic consistency. The structure preservation score measures the structural stability of the candidate image relative to the current image in the non-target region. A higher value indicates less damage to the original image's structure. The identity fidelity score measures the degree to which the identity features of the subject are preserved in the candidate image; the higher the value, the more complete the subject features. This represents the local preservation score, which measures the area preserved. The stability of the inner pixel; a higher value indicates less interference in the non-editing area. The boundary naturalness score measures the smoothness of the transition between the edited and preserved regions, and is defined as follows: (k) = 1 − Norm(Var(|∇ (k)|∂ (k))), where (k) represents the candidate image in the kth round, ∂ (k) represents the boundary region of the soft editable mask, and Norm(·) ∈ [0, 1] is the normalization function. A higher value indicates a smoother boundary gradient change and a more natural transition. w1, w2, w3, w4, and w5 are the corresponding weights, and w1+w2+w3+w4+w5=1. The weights can be adjusted according to different application scenarios. For example, in portrait editing, the identity fidelity weight w3 can be increased; in background replacement, the semantic consistency weight w1 and the structure preservation weight w2 can be increased; and in fine retouching scenarios, the boundary naturalness weight w5 can be increased.

[0092] The system makes an acceptance decision for candidate results based on the comprehensive quality score Q. The acceptance decision adopts a dual verification mechanism combining a one-sided quality threshold and a disturbance barrier. The first verification is the quality threshold verification: let the quality acceptance threshold be θlow, which is adaptively and dynamically adjusted based on the following three factors: the editing task complexity factor λcomplex, determined based on the length of the editing instruction, the number of target attributes, and the semantic level depth; the rollback depth factor λdepth, determined based on the number of rollback steps of the current edit relative to the historical node; and the editing area factor λarea, determined based on the proportion of the target semantic area to the total map area. The threshold adjustment formula is θlow = θlow0 · λcomplex · λdepth · λarea, and the calculation result is truncated: θlow = clip(θlow0 · λcomplex · λdepth · λarea, 0, 1), to ensure that the threshold is always within the effective range [0, 1], where θlow0 is the default baseline threshold. Each adaptive factor is truncated and limited to a preset range: λcomplex ∈ [0.5, 2.0], λdepth ∈ [0.5, 1.5], and λarea ∈ [0.3, 2.0], all of which are positive numbers. The technical meaning of each factor is as follows: λcomplex increases with the complexity of the editing task; the more complex the task (e.g., longer instruction length, more target attributes, deeper semantic levels), the higher the acceptance threshold for editing quality. λdepth increases with the number of backtracking steps; the deeper the backtrack, the worse the previous path quality, and the higher the quality of the current path must be to be accepted. λarea increases with the proportion of the target editing area to the total area of ​​the image; the larger the editing area, the higher the requirements for structure preservation and local consistency. All factors are truncated to avoid excessively large thresholds due to continuous multiplication. When Q ≥ θlow, the first verification is passed; the second verification is the disturbance guardrail verification: checking whether the local preservation score slocal meets the requirements. ≥ and boundary naturalness score Does it meet the requirements? ≥ . and This is a hard constraint threshold independent of the Q-weighted system. Its function is to prevent the overall high score from being masked by the abnormality of individual perturbation components when Q is high; only when both verifications pass, the candidate editing result is accepted and written into the directed acyclic graph as a new valid state node, updating the node set V and the edge set E; when Q < θlow, or < ,or < When any verification condition is not met, the candidate result is rejected and the current state remains unchanged, while the following adaptive adjustment mechanism for editing intensity is triggered. Through this closed-loop approach, the system achieves coordinated control between editing generation, quality judgment, and state update.

[0093] To avoid continuous rejections causing the editing process to stall, the system introduces an adaptive adjustment mechanism for editing intensity. When the overall quality score Q < θlow, or the local preservation rate... < or boundary naturalness < Instead of discarding candidate results directly, an adaptive editing intensity adjustment process is initiated. Through closed-loop adjustment of key control parameters during the editing process, subsequent generated results gradually approach the target editing state. The core of this mechanism is to model the editing intensity as a four-dimensional parameterized control vector, defining the editing intensity vector for the k-th round of editing as:

[0094] ;

[0095] The meanings of each component are as follows: ΔI is the residual amplitude coefficient, used to control the overall scaling intensity of the output change ΔI of the residual generation network. This is the text conditional guidance coefficient, used to adjust the guidance strength of text features on visual features in the cross-modal attention mechanism; This is the editable area expansion factor, used to control the editing mask. The boundary response is enhanced or weakened based on the attention heatmap; This is the boundary smoothing coefficient, used to adjust the smoothness of the transition between the editing area and the preserved area. All force components have safety boundaries. ∈ [ , ]、 ∈ [ , ]、 ∈ [ , ]、 ∈ [ , The lower bound ensures basic editing capabilities, while the upper bound prevents excessive editing from causing image corruption.

[0096] Before delving into the specific intensity modulation mechanism, it is necessary to clearly distinguish between two types of hold masks: soft hold masks. From attention heatmap Generated by upsampling ( [p] = 1 − [p]), the area it identifies is the "preservable area" automatically inferred by the system based on text instructions, and this mask can be adjusted according to the editing strength parameters. Adjust the response range by regulating the mask; hard hold mask The mask, explicitly specified by the user or automatically labeled by the identity recognition model, identifies core protected areas that the user explicitly requires to remain "absolutely unchanged" (such as a person's face, a specific object, etc.). This mask cannot be modified under any editing intensity. Mathematically, the hard-preservation constraint is expressed as: for all... At pixel position p where [p] = 1, there is always... [p] = [p], and edit mask Forced to be zeroed within the hard-hold region — that is (k)[p] = (k)[p] · (1 − [p]). All adjustments to the intensity vector—including residual amplitude scaling, text guide intensity adjustment, soft mask response range adjustment, and boundary smoothing—are strictly limited to (1 − ) / 2. Within the marked operable area, no penetration into the hard retaining area is permitted in any way. Soft retaining mask. Provides editing stability assurance, hard-hold mask It provides editorial security, and the two work together.

[0097] ;

[0098] When the semantic response of the candidate edit results is insufficient, increase To enhance the range of editing changes; when there is excessive editing or insufficient preservation of non-target areas, reduce... To reduce editing workload.

[0099] ;

[0100] when When the value is > 1, the attention distribution tends to be sharper, and the model focuses more on the semantic regions specified in the text, making it suitable for editing tasks that require precise semantic alignment; when When the value is less than 1, the attention distribution tends to be smoother, and the model retains more global contextual information during editing, making it suitable for editing tasks that require maintaining overall consistency; where Qatt = T·WQ, Katt = X·WK, Vatt = X·WV, WQ, WK, and WV are learnable projection matrices, and dk is a scaling factor. Here, X = Fvis( ) is the visual feature map output by the visual encoder.

[0101] ;

[0102] in This is an element-wise truncation function that ensures the mask value always lies within the interval [0,1]; when When > 0, increase the soft response weights of the target region and its boundary neighborhood to cover potential boundary diffusion effects; when When < 0, the response weight of the editing region boundary is reduced to minimize the impact on non-target regions; correspondingly, the soft-preserving mask is updated synchronously. The system maintains the mask through hard hold. Apply pixel-level constraints to the expansion of the editing area to ensure that adjustments to the editing mask do not penetrate areas that the user explicitly requests to be protected. This means that the editing mask value within the hard-preserved region is forcibly set to zero, and any adjustment of the response weight in the editing region is strictly limited to the non-hard-preserved region.

[0103] (4) Boundary smoothing: Boundary smoothing coefficient By performing linear interpolation between the original residual stacking result and the Gaussian filtering result, artifacts and discontinuities at the editing region boundaries are eliminated. The candidate image generation formula for the k-th round is then modified as follows:

[0104] ;

[0105] ;

[0106] After boundary smoothing, a hard-hold mask is used. A final overlay is performed on the candidate images to ensure that the hard-preserved regions are unaffected by editing or smoothing operations. The final candidate images for the k-th round are:

[0107]

[0108] in As the final candidate image, For the smoothed candidate image, To maintain the mask, This is an element-wise product.

[0109] After the aforementioned adaptive adjustment of editing intensity, the system re-executes the editing generation process in step S4 with the updated intensity parameters, generating new candidate results and re-entering the quality verification stage in step S5. This closed-loop iterative process of "generation-verification-adjustment-regeneration" continues until the candidate result passes the quality verification. When the number of consecutive rejections exceeds the preset limit, the system determines that the current editing path is diverging, forcibly reverts to the nearest confirmation node, resets the editing intensity vector to the initial default value, and prompts the user to adjust the editing instructions or confirm the editing target. Once the candidate result passes the verification, it enters the following status update stage.

[0110] Once a candidate image passes quality verification, the system writes it to a new state node and establishes a parent-child connection with the current node in the directed acyclic graph. Simultaneously, it records the corresponding editing instructions, generation parameters, constraint information, scoring results, summary vector, and necessary intermediate features. In this way, each successful edit creates a new traceable state, facilitating subsequent retrieval, comparison, and rollback.

[0111] In addition to storing the image result, the new state node can also store the operation tags for that editing session, such as "local repair," "style transfer," "object replacement," and "color adjustment." The system can use these tags to assist in subsequent searches, allowing users to quickly return to a similar editing state during multiple rounds of editing.

[0112] When a user issues a rollback command, the system restores the corresponding historical version based on the target state node and truncates any unconfirmed branches after that node. If the user initiates a new edit request from an intermediate node, the system uses that node as the new starting point for editing, forming a new branch path. Thus, the system supports multi-path exploratory editing without disrupting the original historical state.

[0113] Once a user confirms a result, the system can perform pruning on the directed acyclic graph. During pruning, the system prioritizes retaining the following: confirmed nodes, nodes from the most recent few rounds of editing, key branch nodes, and summary information for each node. For redundant branches that have not been accessed or confirmed for a long time, the system can retain only the compressed summary representation instead of the complete image intermediate state, thereby reducing storage overhead and improving retrieval efficiency.

[0114] In long-term sessions, the system can also prioritize different branches based on the frequency of use, acknowledgment probability, and rollback probability of the directed acyclic graph. Frequently used paths can retain more complete context, while infrequent and low-quality branches can be gradually compressed or cleaned up to control overall resource consumption.

[0115] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A multi-round interactive text-guided image editing method based on directed acyclic graphs, characterized in that, Includes the following steps: S1. The image result generated by each editing is abstracted into a state node. A directed acyclic graph is constructed with the root node corresponding to the original image. The state node is used to record the image state and its parent-child editing relationship, and supports the rollback, switching and branch comparison of the edited version through the node path information. S2. Save editing instructions, editing constraints, quality scores, parent node references, and historical summary vectors for each status node, and generate a fixed-dimensional summary representation through the memory compression module to support natural language retrieval of historical edit nodes; S3. After receiving the rollback instruction, locate the target historical node and truncate the invalid branches after it; after confirming the current result, prune the historical link, retain only the necessary confirmation nodes and summary information, and then continue to generate new branches from the current node; S4. After receiving a new text editing instruction, the current image state and the text instruction are fused and modeled to locate the target area and generate editing constraints. Then, the local change is predicted by the residual generation method and superimposed on the original image to obtain a candidate image. S5. Verify the semantic consistency, structural preservation, identity fidelity, local preservation, and boundary naturalness of the candidate image, and determine whether to accept it based on the comprehensive score. If the candidate image does not meet the preset threshold, maintain the current state, adjust the editing intensity parameters, and regenerate the candidate result so that the subsequent editing process converges to the target image.

2. The method according to claim 1, characterized in that, In step S1, the directed acyclic graph is used. To express, among which For a set of state nodes, Let be a set of directed edges, any edge State node From state nodes Editing operation Obtain; the state node Corresponding image state Furthermore, directed acyclic graphs allow a single node to correspond to multiple successor nodes to form branch editing paths.

3. The method according to claim 1, characterized in that, In step S2, for any state node Record its structured memory information The structured memory information includes editing instructions. Editing constraints Quality score vector Parent node index and compressed history summary vector Editing constraints At least include soft holding area Editable area and the upper limit of pixel variation ; The memory compression module takes the node sequence on the preceding historical path as input and passes it through an encoder network. Generate a fixed-dimensional summary vector ,satisfy: in, Represents the summary vector of the parent node. This represents a vector concatenation operation; The history retrieval system receives natural language queries input by the user. and through semantic encoder This is mapped to a query vector, and then compared with the compressed summary vector of each historical node. Calculate cosine similarity: The top K nodes with the highest similarity are selected as the search results output to support semantic querying and state reuse of historical editing steps.

4. The method according to claim 1, characterized in that, After the rollback command is triggered in step S3, the system will target the historical node. Activate it as the current editing starting point and its subsequent closure. All nodes in the sequence undergo a truncation operation; where the successor closure... Including all nodes The set of nodes reached via a directed path; nodes removed after truncation are marked as invalid nodes or deleted from the set of valid nodes. Once the user confirms the current edit result, the current node will be marked as the confirmation point. The system then backtracks along the main path to perform hierarchical pruning, retaining only the complete state information of the K most recent confirmed points, while only retaining the compressed summary vector and releasing the complete image data for the remaining historical nodes.

5. The method according to claim 1, characterized in that, Step S4, fusion, includes: fusing the current image Input visual encoder Extracting visual feature maps Editing instructions Input text encoder Extracting text features And text features are integrated through a cross-modal attention mechanism. Attention query obtained through projection Visual features Attention keys obtained through projection Sum Multimodal fusion features are obtained through cross-modal attention fusion. ,in: ； in , , For learnable projection matrices, This is the scaling factor.

6. The method according to claim 1, characterized in that, The localization of the target region in step S4 includes: based on multimodal fusion features By regional positioning network Predict the category of the target semantic region and the corresponding spatial attention heatmap The attention heatmap is then upsampled to the original image resolution to obtain an upsampled heatmap, which is used to generate a region mask; wherein the mask is soft-preserved. With soft editable masks They respectively satisfy: Where p represents the pixel position. This is an upsampled attention heatmap; the mask is used to constrain the subsequent generation process, ensuring that editing operations mainly occur within the target area, while areas not indicated for modification remain as unchanged as possible; structured editing constraints are generated based on the current image state, editing instructions, and localization results. The structured editing constraints At least including the upper bound of pixel variation amplitude ,in It is determined by the local neighborhood standard deviation, the global amplitude factor, and the task adaptive adjustment function.

7. The method according to claim 1, characterized in that, In step S4, the residual generation method is achieved through a residual generation network. With the current image Multimodal fusion features and editing constraints As input, only pixel-level changes are predicted. And through a soft editable mask The changes are constrained and then superimposed onto the current image to generate candidate edit results. ,in: in The secure editable mask is obtained by element-wise multiplication of the complements of the soft editable mask and the hard-preserving mask, representing the effective area that is actually allowed to be edited. A hard-preserving mask represents a non-editable region explicitly specified by the user or automatically labeled by the identity recognition model. For safe editing areas, For element-wise multiplication; and for satisfying pixels Maintain consistency with the current image.

8. The method according to claim 1, characterized in that, Step S5, multi-dimensional verification, includes five dimensions: calculating semantic consistency scores using a pre-trained image-text alignment model. ; Structure preservation scores are calculated using features from the intermediate layers of a pre-trained visual network. Defined as ,in This serves as a normalized measure of feature differences in the intermediate layers of a visual network; identity fidelity scores are calculated by extracting identity feature vectors through an identity recognition model. The local preservation score is obtained by calculating the normalized pixel differences within the preservation region. Defined as = 1 − Norm( ),in To maintain normalized pixel differences within the region; And the boundary naturalness score is obtained by calculating the normalized variance of the gradient magnitude of the edit region boundary. Defined as ;in For the candidate image in the kth round, For the boundary region of the soft editable mask, The image gradient magnitude at the boundary of the editing region. For variance operators; The overall quality score Q is obtained by weighted fusion of the scores of the five items, satisfying the following conditions: ;in The method employs a dual verification mechanism combining a single-sided quality threshold and a disturbed guardrail. The acceptance decision rule is: when the overall quality score... And local preservation score Boundary naturalness score At that time, the candidate editing results will be... Accept and write the node into the directed acyclic graph as a new valid state node, and update the node set. With edge set ; when ,or ,or When this happens, the candidate result is rejected and the current state remains unchanged, triggering the adaptive adjustment mechanism for editing intensity; and θlow is the hard barrier threshold independent of the Q-weighted system, and θlow is the quality acceptance threshold.

9. The method according to claim 8, characterized in that, The adaptive adjustment mechanism for editing intensity defines the editing intensity vector for the k-th round of editing as follows: ； in, This is the residual amplitude coefficient, used to control the pixel-level variation. The overall scaling intensity; This is the text conditional guidance coefficient, used to adjust the guidance strength of text features on visual features in the cross-modal attention mechanism; This is the editable area expansion factor, used to control the editing mask. Soft response weights based on attention heatmaps; This is the boundary smoothing coefficient, used to adjust the smoothness of the boundary transition between the editing area and the preserved area; all force components have safety boundaries. , , , The lower bound ensures basic editing capabilities, while the upper bound prevents excessive editing from causing image corruption.

10. The method according to claim 9, characterized in that, The editing intensity vector is embedded into the image editing process in step S4 in the following four ways to achieve parameterized intensity control of the image generation process, wherein gradient transfer is supported in the residual amplitude modulation, text-guided modulation, and soft mask modulation stages: (1) Residual amplitude modulation: residual amplitude coefficient The residual change in the k-th round is corrected by applying the product directly to the output of the residual generation network as follows: ； in, This represents the pixel-level residual change in the k-th round of editing. This represents the residual amplitude coefficient in the k-th round. Represents a residual generation network. This indicates the current state of the image, i.e., the state before this round of editing. Indicates multimodal fusion features, This indicates structured editing constraints; (2) Text guidance intensity modulation: text condition guidance coefficient The sharpness of the attention distribution is adjusted by scaling the query-key inner product in cross-modal attention. When attention distribution becomes sharper, the model focuses more on the semantic regions specified in the text; when At this time, the attention distribution tends to be smoother, and the model retains more global contextual information during editing; (3) Editable area expansion: Editable area expansion coefficient By scaling the attention heatmap pixel by pixel, the response range and editing weight of the editing mask are adjusted, and the editable region mask in the k-th round is corrected as follows: ； in This is an element-wise truncation function to ensure that the mask value is always in the range [0,1]. (4) Boundary smoothing: Boundary smoothing coefficient By performing linear interpolation between the original residual stacking result and the Gaussian filtering result, artifacts and discontinuities at the boundaries of the edited region are eliminated, and the candidate image generated in the k-th round is: ； ； in The standard deviation is expressed as Gaussian convolution kernel, This represents a two-dimensional convolution operation. The original candidate image is not smoothed. After boundary smoothing, a hard-hold mask is used. Perform a final overlay on the candidate images to ensure that the hard-preserved regions are unaffected by editing or smoothing operations. The final candidate images for the k-th round are: in As the final candidate image, For the smoothed candidate image, To maintain the mask, This is an element-wise product.

Images