An image editing method based on geometric distribution correction and space-gated routing

Abstract

The application discloses an image editing method based on geometric distribution correction and space-gated routing, and belongs to the field of computer vision and AIGC. The method comprises the following steps: extracting a source image semantic mask, using rotation position encoding and random geometric projection to construct a structure prior graph, modulating an initial Gaussian noise to generate a correction noise and deriving a dynamic geometric anchor displacement term; at each integration time step, fusing a spatial gradient and a cross-attention background confidence to construct a multi-modal routing mask, and performing soft routing weighted fusion on Q / K / V in the deep layer of a pre-trained generative model; using the geometric displacement term as a coordinate compensation and a soft routing feature for target velocity field evaluation, constructing a decoupled differential system and iteratively updating, and finally decoding and outputting an edited image. The application can maintain high background fidelity while giving sufficient generation freedom to the editing area, and has fast inference speed and low video memory occupation.

Description

Technical Field

[0001] This invention relates to the field of computer vision and artificial intelligence generated content (AIGC) technology, and in particular to an image editing method based on geometric distribution correction and spatial gating routing. Background Technology

[0002] With the rapid development of large-scale text-guided generation models and stream matching techniques, breakthroughs have been achieved in the field of high-fidelity image generation. Building on this foundation, how to perform accurate and efficient text-guided editing of existing images has become a research hotspot in computer vision and artificial intelligence. Currently, mainstream image editing techniques are mainly divided into inversion-based paradigms and non-inversion paradigms:

[0003] Inversion-based image editing techniques: such as Figure 2 As shown, such methods typically require iterative solutions to ordinary differential equations or complex feature optimization processes to map the source image back to the latent space of the generative model. While these methods perform well in zero-shot operations and preserving image structure, their extremely high dependence on the inversion path results in a computationally intensive and time-consuming inference process, making it difficult to meet the needs of real-time deployment and interactive editing.

[0004] Non-inversion image editing techniques: such as Figure 3 As shown, in order to completely avoid cumbersome inversion calculations, researchers have proposed inversion-free editing paradigms in recent years. These methods leverage the mathematical computability of stream matching to significantly improve inference speed by directly coupling the source image with the trajectory generated from the target.

[0005] However, existing inversion-free image editing techniques still suffer from a fundamental theoretical flaw in practical applications, severely limiting their structural fidelity:

[0006] Structure and randomness mismatch: Existing inversion-free methods typically use content-independent standard Gaussian noise to initialize the surrogate source trajectory. This naive initialization completely ignores the geometric manifold information implicit in the source image itself, thus leading to a mismatch between structure and random noise.

[0007] Severe trajectory deviation: Due to the mismatch between initialization noise and the features of the original manifold, the generation process evaluates the velocity field at coordinates that deviate from the source potential manifold, resulting in severe trajectory deviation.

[0008] Background degradation and geometric distortion: This offset inevitably manifests visually as geometric distortion and texture collapse in the non-edited areas (i.e., the background area), severely compromising the fidelity of image editing.

[0009] The contradiction between editing freedom and fidelity: In existing technologies, if uniform structural constraints are forcibly imposed, the contextual relationship between the editing subject and the background will be forcibly severed, excessively restricting the generative freedom of the model, making it difficult to perform large-scale semantic modifications; conversely, if attention editing is performed without constraints, it often leads to serious boundary leakage and background distortion.

[0010] In summary, how to establish macroscopic structural anchor points to prevent trajectory deviation and achieve dynamic decoupling of microscopic features to ensure editing freedom within a non-inversion framework is a pressing technical challenge in the field of image editing. Summary of the Invention

[0011] To address the "structure-random mismatch" phenomenon caused by initialization noise being unrelated to the source image content in existing inversion-free image editing techniques, which leads to problems such as generated trajectory offset, low background region fidelity, and geometric distortion, this invention provides an image editing method based on geometric distribution correction and spatial gating routing. By constructing a mask-based geometric prior, the method effectively suppresses trajectory offset in inversion-free flow matching. Furthermore, leveraging the feature decoupling capability of the spatial gating routing mechanism, it overcomes the excessive constraints imposed by traditional structure protection methods, balancing structural fidelity and semantic generation freedom during the editing process.

[0012] The objective of this invention is achieved through the following technical solution:

[0013] This invention provides an image editing method based on geometric distribution correction and spatial gating routing, comprising the following steps:

[0014] S1: Obtain the source image, source text prompts, and target text prompts; initialize the trajectory and decompose the semantic region of the source image to obtain a set of semantic region masks;

[0015] S2: Using the semantic region mask set obtained in S1, construct a structural prior graph through rotational position encoding and random geometric projection;

[0016] S3: At the initial moment of trajectory generation, the standard Gaussian noise is linearly modulated using the structure prior map to generate correction noise, ensuring that the target generated trajectory is anchored on the geometric manifold of the source image.

[0017] S4: Based on the corrected noise, derive the corrected proxy source trajectory state and dynamic geometric anchor displacement term at time step t respectively;

[0018] S5: Calculate the spatial gradient and semantic background confidence of the structural prior graph, and construct a spatial gated routing mask through nonlinear fusion;

[0019] S6: In a predetermined depth layer of the pre-trained generative model, the query vector, key vector, and value vector of the source trajectory feature stream and the target trajectory feature stream are weighted and fused using the spatial gated routing mask to obtain the routed target features;

[0020] S7: The dynamic geometric anchor point displacement term is used as coordinate compensation, and the routed target features are used for target velocity field evaluation. A decoupled flow matching differential system is constructed and discretized iteratively performed. Finally, the edited image is decoded and output.

[0021] Further, S1 includes:

[0022] S11. Obtain the source image and its corresponding source text prompts and target text prompts ; through pre-trained encoder Map the source image to the latent space to obtain the source image. True coordinates on the potential manifold ;

[0023] S12. Initialize the starting point of the target editing trajectory, and initialize the initial state of the target editing trajectory to the true latent representation of the source image:

[0024] ;

[0025] in, The starting point for editing the target trajectory; N is the total number of sampling steps; FE represents flow matching editing; Represents the initial time step for trajectory solving;

[0026] S13. Use an image segmentation model to decompose the source image into semantic regions, filter them, and extract the set of semantic region masks. :

[0027] ;

[0028] in, K is a single semantic region mask; K is the total number of masks.

[0029] Further, S2 includes:

[0030] S21. For the semantic region mask set obtained in S1 Each single semantic region mask in By using rotational position encoding, the coordinates of pixels inside the semantic region mask are mapped to a high-dimensional feature space, and the regional geometric signature of each semantic region is calculated using the following formula. :

[0031] ;

[0032] in:

[0033] A single semantic region mask; The area coefficient represents the mask of a single semantic region. The total number of valid pixels inside; Normalized pixel coordinates, representing the source image Relative coordinates on a two-dimensional plane; For rotational position encoding function;

[0034] S22. A deterministic structural hashing mechanism is employed to map the regional geometric signature calculated in S21 into a scalar intensity map using a preset and frozen linear projection vector. And assign a unique intensity identifier to each spatial region;

[0035] S23. Perform scale alignment and normalization on the scalar intensity map obtained in S22 to construct a numerically constrained map. Prior diagrams of structures within the scope .

[0036] Furthermore, in step S22, a single-channel scalar intensity map is constructed using the following formula. :

[0037] ;

[0038] in: For projected scalar values; The total number of semantic region masks; For an indicator function, if and only if the coordinates Located in the mask When the object is inside the physical boundary, the function takes a value of 1; otherwise, it takes a value of 0.

[0039] The linear projection calculation method is to use the linear projection vector W and... The inner product of W is initialized from a uniform distribution. And it remains frozen throughout the entire inference and execution process of the algorithm. The dimension of the high-dimensional feature space after the aforementioned rotation position encoding mapping.

[0040] Further, S3 includes:

[0041] At the initial time t=1 of trajectory generation, the sampled standard Gaussian noise is analyzed using the aforementioned structural prior map. Perform linear modulation to generate correction noise :

[0042] ;

[0043] in: and These are the structural fidelity coefficient and the random flexibility coefficient, respectively. The corrected noise for the final output; This is the a priori diagram of the structure.

[0044] Furthermore, in S4, the calculation formulas for correcting the proxy source trajectory state and the dynamic geometric anchor point displacement term are as follows:

[0045] ;

[0046] ;

[0047] in:

[0048] To correct the proxy source trajectory status; The dynamic geometric anchor point displacement term; t is the time step; These are the true coordinates of the source image on the latent manifold; The corrected noise.

[0049] Further, S5 includes:

[0050] S51. Based on the structural prior map constructed in S2, the physical boundary contours of the physical objects in the source image are located by calculating the spatial gradient magnitude of the structural prior map:

[0051] Spatial gradient magnitude The calculation formula is:

[0052] ;

[0053] Where x and y represent the coordinate axes in two-dimensional space; This is the a priori diagram of the structure;

[0054] S52. Extract the dynamic target attention map corresponding to the target text prompt from the cross-attention layer of the target generation trajectory in real time. The background confidence level is calculated using the following formula. :

[0055] ;

[0056] in: The normalization function maps the extracted dynamic target attention map to... The probability space;

[0057] S53. The spatial gradient magnitude is determined through a nonlinear competition mechanism. and background confidence Fusion, constructing the final spatial gating routing mask :

[0058] ;

[0059] in: This is the threshold shift hyperparameter, used to set the baseline for the activation boundary; This is a sharpness temperature hyperparameter used to adjust the sensitivity and sharpness of the boundary response; The logistic sigmoid activation function maps the physical boundary gradient, after linear transformation, to... In a continuous nonlinear probability space, make it have the same mathematical dimensions as the confidence matrix; This is the maximum value competition operator.

[0060] Furthermore, in step S6, a soft routing operation is performed on the query vector Q, key vector K, and value vector V of the attention mechanism according to the following formula:

[0061] ;

[0062] ;

[0063] ;

[0064] in:

[0065] , , These are the query, key, and value features of the source trajectory, respectively; , , These are the query, key, and value features of the target trajectory, respectively. The Hadamard product operator is used to determine the mixing ratio of source and target features; , ,

[0066] These are the target features of the query, key, and value after routing; For spatial gating routing mask;

[0067] When the value of the spatial gated routing mask approaches 1 at the background or physical boundary, the target feature after routing is forced to retain the source trajectory feature; when the spatial gated routing mask approaches 0 inside the edited object, the target feature after routing fully adopts the target trajectory feature, thus achieving decoupling between structural protection and generation freedom.

[0068] Further, S7 includes:

[0069] Construct the decoupled flow-matched differential system as shown below:

[0070] ;

[0071] in: For editing the trajectory velocity field, it represents the instantaneous velocity vector of the final edited trajectory at the current time step t;

[0072] For pre-trained velocity field evaluation networks; To edit the current state of the trajectory; This refers to the dynamic geometric anchor point displacement term; Intervention for spatial gating routing; For the corrected proxy source trajectory state; and These represent the target text prompt and the source text prompt, respectively.

[0073] A step-by-step solution is obtained using a discretized numerical integration method:

[0074] ;

[0075] in: The discrete time step; This represents the trajectory state at the next moment;

[0076] After the iterations are completed, the final latent representation is obtained. Call the pre-trained decoder Convert it back to pixel space to obtain the final edited image output.

[0077] Compared with the prior art, the present invention has the following significant advantages:

[0078] Extremely high fidelity of the non-editable region (background): This invention effectively suppresses trajectory offset in inversion-free flow matching by constructing a mask-based geometric prior. In terms of objective evaluation metrics, compared with existing flow matching baseline methods, the structure distance of this invention is significantly reduced to [value missing]. Meanwhile, the peak signal-to-noise ratio (PSNR) in the background region increased to 26.82, the structural similarity (SSIM) reached 89.15%, and the perceptual loss (LPIPS) decreased to [missing value]. This demonstrates, from a quantitative perspective, an extremely strong ability to suppress geometric distortion.

[0079] Superior semantic editing alignment: Thanks to the feature decoupling capability of the spatial gating routing mechanism, this invention breaks through the excessive constraints imposed by traditional structural protection methods. When performing complex text-guided editing, neither conceptual forgetting nor edge artifacts occur. The contrastive language-image pre-training (CLIP) similarity of the local edited region reaches 22.61, achieving a perfect balance between generative freedom and structural fidelity.

[0080] Significant computational efficiency advantages: This invention completely eliminates the time-consuming potential inversion and feature optimization processes. The geometric correction module is executed only once at the trajectory starting point, and the spatial routing module requires no additional large-scale computation. Tests show that when processing standard resolution images, the inference time on a single consumer-grade RTX 5080 graphics card (16GB VRAM) is only about 7.15 seconds, with VRAM usage controlled at 15.18GB, greatly reducing the computational barrier and deployment cost for high-fidelity image editing. Attached Figure Description

[0081] Figure 1 This is a schematic diagram illustrating the principle of inversion-free image editing based on geometric distribution correction and spatial gating routing proposed in this invention.

[0082] Figure 2 This is a schematic diagram of the inversion-based image editing principle in existing technologies.

[0083] Figure 3 This is a schematic diagram illustrating the principle of traditional non-inversion image editing.

[0084] Figure 4 The overall flowchart of an image editing method based on geometric distribution correction and spatial gating routing described in this embodiment of the invention.

[0085] Figure 5 This is a schematic diagram illustrating the principle of random geometric projection in geometric distribution correction in an embodiment of the present invention.

[0086] Figure 6 This is a sensitivity analysis curve of the correction noise ratio parameter in an embodiment of the present invention. Detailed Implementation

[0087] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present application will now be described in detail with reference to the accompanying drawings and embodiments.

[0088] Example 1:

[0089] This embodiment provides an image editing method based on geometric distribution correction and spatial gating routing, primarily applied to text-guided high-fidelity image editing scenarios without inversion. In specific implementation, it combines... Figure 4 The overall architecture diagram shown illustrates that the core editing process of this invention is achieved through inverse integration of ordinary differential equations (ODEs), specifically including the following detailed steps:

[0090] Step 1: Macroscopic structural anchoring of the source image based on geometric distribution correction, including:

[0091] S1. Input initialization and semantic mask extraction: Obtain the source image, source text prompts and target text prompts, perform trajectory initialization and semantic region decomposition on the source image, and obtain a set of semantic region masks.

[0092] S11. Data Acquisition and Latent Mapping: Acquiring Source Images and its corresponding source text prompts and target text prompts ; through pre-trained encoder Map the source image to the latent space to obtain the initial source latent representation:

[0093] ;

[0094] in, Source image The true coordinates on the latent manifold;

[0095] The source image, i.e. the original image input by the user and to be edited, is a data matrix belonging to the pixel space, which is the physical operation object of the entire editing algorithm;

[0096] These are source text prompts, indicating that they are used to objectively describe the source image. The natural language conditional sequence of visual content serves as a guiding condition for the generation of the source trajectory in the subsequent stream matching process, ensuring that the network can accurately reconstruct the features of the original image.

[0097] The target text prompt is a natural language conditional sequence representing the editing effect the user intends to achieve. This variable serves as a guiding condition for generating the target trajectory and is responsible for driving the network to generate new semantic content.

[0098] The pre-trained encoder is the encoder module in the pre-trained variational autoencoder (VAE) that is called by the system. To reduce the computational complexity of generating high-resolution images, the system uses an encoder. The operation domain is transferred from the pixel space to the potential space.

[0099] S12. Target Trajectory Initialization: Subsequently, the starting point of the target edit trajectory is initialized. :

[0100]

[0101] in, Source image The true coordinates on the latent manifold are the core benchmark for subsequent determination of whether the trajectory has been distorted; N is the total number of sampling steps, which represents the total number of discretization time steps of the ordinary differential equation (ODE) solver in the inverse integration process. This hyperparameter determines the accuracy and computational cost of the flow matching process. FE stands for Flow Match Editing, which marks the starting point for editing the target trajectory. The initial time step represents the trajectory solution; in the inversion-free flow matching paradigm, the system directly and forcibly initializes the initial state of the target edit trajectory to the true latent representation of the source image, that is, lets This establishes a strong coupling relationship between the source trajectory and the target trajectory at the starting point of time.

[0102] S13. Source Image Instance-Level Semantic Decomposition: The source image is decomposed into semantic regions using the SAM2 image segmentation model, and a series of stable semantic region masks are filtered and extracted. :

[0103] ,

[0104] in, For a single semantic region mask, it represents the binary matrix of the k-th instance region output by the image segmentation model. The pixel value inside the mask is 1, and the pixel value outside the mask is 0. Its function is to accurately delineate the physical boundaries of independent objects or backgrounds in the image. K is the total number of masks, which represents the total number of masks whose area exceeds the preset threshold and whose structure is stable after filtering by the algorithm. The semantic region mask set is a collection of all the single semantic region masks extracted above. This set serves as a snapshot of the topological structure of the source image and is passed to the subsequent geometric distribution correction module to constrain the randomness of Gaussian noise.

[0105] The system calls a pre-trained large visual model as the image segmentation model SAM2, using the source image... Perform panoramic or interactive segmentation on the input.

[0106] S2, Geometric Distribution Correction (GDC) and Prior Construction: Using the semantic region mask set obtained in S1, a structural prior graph is constructed through rotational position encoding and random geometric projection.

[0107] To overcome the structural shift caused by pure Gaussian noise initialization in standard inversion-free methods, as shown in Figures 4 and 5, this embodiment constructs a structured geometric prior and performs noise modulation in the first stage. This stage completely eliminates time-consuming and memory-intensive heavy visual encoders (such as convolutional neural networks or visual transformers). The specific implementation process and symbol definitions are as follows:

[0108] S21. Spatial Signature Calculation: This step aims to transform the physical topological information of the source image into computable geometric features, based on the semantic region mask set obtained in step S1. Each single semantic region mask in By utilizing the Rotation Position Encoding (RoPE) mechanism, the coordinates of pixels inside the semantic region mask are directly mapped to a high-dimensional feature space. The regional geometric signature of each semantic region is then calculated using the following formula. :

[0109]

[0110] in:

[0111] It is a single semantic region mask, derived from the set of semantic region masks output by the image segmentation model in step S1. The kth element in the equation serves to strictly define the two-dimensional physical space boundary of an independent object or background region.

[0112] The area coefficient represents the mask of a single semantic region. The total number of internal effective pixels (i.e., pixel value of 1) is used as the normalization denominator in the formula. Its key role is to eliminate the scale effect caused by the difference in physical size of different objects on feature convergence, and to ensure that the extracted features have scale invariance.

[0113] Normalized pixel coordinates, representing the source image The relative coordinate points on the two-dimensional plane serve to provide basic location input identifiers for RoPE encoding;

[0114] For the rotation position encoding function, a rotation matrix mechanism from the field of natural language processing is introduced to normalize the pixel coordinates. The core function of this computation is to endow each pixel coordinate with a high-dimensional absolute position and relative distance perception capability with extremely low computational overhead.

[0115] The region geometric signature is a high-dimensional vector obtained by aggregating and averaging the above operations. Its physical meaning is as a "digital fingerprint" of the object instance, uniquely and abstractly representing the centroid position and macroscopic shape distribution of the object in the image.

[0116] S22. Random geometric projection and structural prior map generation: A deterministic structural hashing mechanism is adopted, and the regional geometric signature calculated in step S21 is mapped to a scalar intensity map using a preset and frozen linear projection vector, and a unique intensity identifier is assigned to each spatial region.

[0117] Obtain the region geometric signature in step S21 Afterwards, combined Figure 5 Based on the principle of random geometric projection, a deterministic structural hash mechanism is used for dimensionality reduction mapping, and a single-channel scalar intensity map is constructed using the following formula. :

[0118]

[0119] in: The projection scalar value represents the linear projection operation, which instantly compresses the geometric signature of a high-dimensional region into a unique single-channel scalar value to distinguish different semantic regions in the grayscale dimension. The linear projection is calculated by the linear projection vector W and... The inner product of W is initialized from a strictly defined uniform distribution. Furthermore, it remains "frozen" throughout the entire inference and execution process of the algorithm (i.e., it does not participate in any gradient updates or learning). Its role is to act as a hash key, ensuring the determinism and random separability of the dimensionality reduction operation. This refers to the high-dimensional feature space dimension after the aforementioned rotation position encoding mapping; The total number of semantic region masks is derived from the total number of valid masks determined in step S1, and its function is to define the upper limit of the loop for traversing all entities in the entire graph. An indicator function is a mathematical logic gate function that is defined if and only if the coordinates... Located in the mask When the feature is inside the physical boundary, the function takes a value of 1; otherwise, it takes a value of 0. The core function of the indicator function is to ensure the local isolation of the feature and guarantee the calculated projection scalar value. It strictly fills only within its corresponding physical contour, and there will never be feature penetration between regions; It is a single-channel scalar intensity map, and the full-size two-dimensional matrix is ​​obtained by accumulating k instances of the whole image. It forms a "painting" that contains the physical structure outlines of the source image on a macroscopic level.

[0120] S23. Perform scale alignment and normalization on the scalar intensity map obtained in step S22 to construct a map whose values ​​are strictly limited to... A priori diagram of the structure within the range.

[0121] Perform size alignment and scalar strength diagram The intensity map was adjusted to the exact same resolution size as the Diffusion Transformer (DiT) latent space using bilinear interpolation and other downsampling algorithms. Then, the values ​​in the adjusted intensity map were strictly scaled down using a min-max normalization algorithm. The bounded intervals form the final structural prior graph. By normalizing, the intensity map is strictly constrained to... Within a certain range, this is the fundamental mechanism for ensuring system stability from a mathematical perspective; this operation ensures that it can be used in subsequent applications. When modulating Gaussian noise, the overall noise energy scale after modulation will not go out of bounds, thus always remaining within the effective working range of the activation function of the pre-trained large model, completely preventing image generation collapse caused by feature distribution drift.

[0122] S3. Initial Noise Modulation and Dynamic Trajectory Anchoring: At the initial moment of trajectory generation, the structural prior map constructed in step S2 is used. Linear modulation of standard Gaussian noise generates correction noise with an energy scale similar to that of Gaussian noise, ensuring that latent features are always within the effective activation range of the generative model and preventing distribution shift.

[0123] In order to transform the structural prior diagram constructed in the previous step This is transformed into corrective noise that can guide the dynamic system at the initial moment of trajectory generation. (i.e., the starting point of the inverse integration of the ordinary differential equation), the system utilizes the aforementioned structural prior diagram. Standard Gaussian noise of the sample Perform linear modulation to generate correction noise :

[0124]

[0125] in: and Defined as structural fidelity coefficient and stochastic flexibility coefficient respectively, these two orthogonal scalar parameters constitute the core control lever between "geometric structure preservation" and "text editing freedom" of the system; This is the corrected noise for the final output. Through this modulation operation, the originally blind random noise is injected with the physical topological information of the source image. Furthermore, thanks to the structural prior map... Strictly bound to Within the interval, It maintains an energy scale that is highly consistent with the standard Gaussian distribution, which ensures that the latent features are always within the effective working range of the activation function of the pre-trained model, thus avoiding system collapse caused by distribution shift (OOD) from a mathematical perspective.

[0126] t=1 is the initial time of trajectory generation (i.e., the starting point of the denoised trajectory). A single modulation is performed at this point, avoiding the repeated introduction of complex feature extraction calculations in subsequent integration. This macroscopic geometric constraint application process has extremely high computational efficiency, and the additional latency overhead is almost negligible (approximately 1 second).

[0127] The parameter optimization mechanism in this embodiment is based on sensitivity analysis of hyperparameters, such as... Figure 6 As shown, when When the value is 0.8, the structural consistency of the generated image and the semantic alignment with the target text achieve the optimal mathematical trade-off. Therefore, in the preferred embodiment, this parameter is set to 0.8.

[0128] S4. Dynamic trajectory anchoring: Based on the aforementioned noise correction... Derive the corrected agent source trajectory state at time step t. and dynamic geometric anchor point displacement term This ensures that the target generation trajectory is firmly anchored to the geometric manifold of the source image:

[0129]

[0130]

[0131] in:

[0132] To correct the surrogate source trajectory state, in traditional flow matching frameworks, directly using pure Gaussian noise for linear interpolation inevitably leads to a significant deviation of the trajectory from the source's latent manifold. This embodiment introduces... The interpolation path was redefined so that the spatial distribution of the surrogate source trajectory state closely matches the geometric features of the real source image while satisfying the time evolution equation.

[0133] The dynamic geometric anchor point displacement term is determined by the modified surrogate source trajectory state. The difference between the vector of the source image and the source image at the true coordinates of the potential source is calculated.

[0134] Temporal expansion and global kinematic stability in this embodiment: Although The calculation involves a time variable t, but its core physical significance lies in successfully extending the static spatial prior generated by the GDC module to the time domain. In each subsequent integration step of the small ordinary differential equation (ODE), This is injected directly into the evaluation coordinates of the target velocity field as a continuously effective constant structure correction term. This dynamic coordinate compensation mechanism ensures that the generated target trajectory is firmly anchored to the geometric manifold of the source image, completely eliminating trajectory offset and background degradation from the root of the dynamics.

[0135] Step Two: Building upon the macroscopic structural anchoring achieved in Step One, a spatial gating routing mechanism is introduced to decouple structural protection and editing freedom at the microscopic level, including:

[0136] S5: Dynamic Construction of Spatial Gated Routing Mask: The spatial gradient and semantic background confidence of the structural prior graph are calculated, and a spatially gated routing mask is constructed through nonlinear fusion. This step aims to define the execution structure of the spatially gated routing mechanism within the generative model (Stable Diffusion 3.5). The specific implementation process is as follows:

[0137] S51. Locate the physical boundary contours of the physical objects in the source image by calculating the spatial gradient of the structure prior map:

[0138] The system first utilizes the structural prior diagram constructed in step S2. It accurately captures the outline of an object by calculating its spatial gradient, and its spatial gradient magnitude The calculation formula is:

[0139]

[0140] Here, x and y represent the coordinate axes in two-dimensional space.

[0141] Due to the structural prior diagram The gradient values ​​within each semantic region exhibit piecewise constant characteristics, with the gradient values ​​approaching 0. However, at the boundaries between different objects (i.e., physical boundaries), the gradient values ​​undergo drastic changes. Therefore, this gradient magnitude matrix can, like a scalpel, precisely delineate the source image input in step S1 without relying on any external edge detection network. The physical boundary contours of all independent object instances within.

[0142] S52. Based on the target generation trajectory anchored in step one, extract the semantic background confidence from the cross-attention layer of the target generation trajectory:

[0143] To determine which regions in the source image are allowed to be modified and which regions must be protected, the system needs to extract the real-time attention distribution from the target generation branch.

[0144] From the cross-attention layers of the target generation trajectory, a two-dimensional attention response matrix directly corresponding to the target text prompt word is extracted in real time, i.e., a dynamic target attention map. The confidence level of the background region is calculated using the following formula:

[0145]

[0146] in:

[0147] The dynamic target attention map refers to the two-dimensional attention response matrix extracted from the cross-attention layers of the target generation trajectory (the dynamic trajectory anchored in step S4) at the current time step t, which directly corresponds to the "edit token" input by the user. Its physical meaning represents the spatial locations in which the generative model is currently trying to generate new editing semantics. The normalization function maps the extracted dynamic target attention map to a minimum-maximum iso-scaling method. The probability space; This is the background confidence matrix, obtained by performing a logical NOT operation. The core calculation function of the background confidence matrix is ​​to indicate when the edit attention in a certain region is extremely low (i.e., the model does not intend to modify that region). When the value approaches 1, the system determines that the area has a very high "background confidence" and needs to be strictly protected; otherwise, it approaches 0, indicating that the area is the core editing area.

[0148] S53. Nonlinear competitive fusion of multimodal masks: After obtaining the spatial gradient magnitude and background confidence matrix Then, the system merges the two through a nonlinear competition mechanism to construct the final spatial gating routing mask. :

[0149]

[0150] in:

[0151] The threshold shift hyperparameter is used to set the baseline for the activation boundary and filter out pseudo-gradients caused by small noise. This is a scaling hyperparameter used to normalize the gradient magnitude so that it conforms to the input domain of the activation function. The temperature hyperparameter for sharpness acts as a multiplication factor, and its function is to adjust the sensitivity and sharpness (steepness) of the boundary response. The larger the value, the more stark and clear the boundary isolation effect; The logistic sigmoid activation function maps the physical boundary gradient, after linear transformation, to... In a continuous nonlinear probability space, make it have the same mathematical dimensions as the confidence matrix; The maximum value competition operator is a logical OR gate mechanism whose decisive role is that the operator will output a high value as long as a pixel either belongs to the non-edit background or is located on the physical boundary of the object (gradient surge).

[0152] For multimodal routing masks, the system's final micro-decision matrix, each scalar value in the matrix dynamically determines whether the corresponding pixel should retain the source image features (protection) or be covered by the target features (editing) in the next feature soft routing step.

[0153] S6: Feature soft routing of deep attention layer: In a predetermined depth layer of the pre-trained generative model, the query vector, key vector and value vector of the source trajectory feature stream and the target trajectory feature stream are weighted and fused using the spatial gated routing mask to obtain the routed target features.

[0154] After obtaining the multimodal routing mask Subsequently, in this embodiment, at each sampling time step t of the inverse integral of the ordinary differential equation (ODE), a dynamic soft routing operation is performed within the pre-trained generative model (Stable Diffusion 3.5). The routing mask is used to weight and fuse the query vectors, key vectors, and value vectors of the source trajectory feature stream and the target trajectory feature stream, achieving feature-level dynamic routing. This step is designed based on the characteristic of "deep networks controlling high-frequency semantics" and seamlessly embedded into the model's attention calculation mechanism.

[0155] S61. Selection of Routing Activation Domain: The generative model used in this embodiment is specifically the pre-trained large model (Stable Diffusion 3.5) based on the Diffusion Transformer (DiT) architecture defined in step S1. Extensive empirical analysis has found that different layers of the Diffusion Transformer architecture have different focuses in controlling the generated images. Deep networks play a decisive role in the generation of high-frequency textures, local geometric structures, and core semantic attributes. Therefore, to avoid disrupting the global macro-tone, the spatial gating routing mechanism is configured to activate only within these specific deep attention layers (layers 13-23). ​​In shallow networks, the model maintains conventional flow matching evaluation without any feature intervention.

[0156] S62. Weighted Fusion and Decoupling of Q / K / V Deep Attention Features: Within a specific deep layer where the routing mechanism is activated, the system utilizes the spatially gated routing mask generated in step S5. The attention mechanism's three core elements—the query vector (Q), the key vector (K), and the value vector (V)—are subjected to soft routing operations according to the following formula:

[0157]

[0158]

[0159]

[0160] in:

[0161] These are the query, key, and value feature vectors of the source trajectory, representing the corrected proxy source trajectory state obtained in step S4. The feature vectors calculated during forward propagation at the current time step t and the current network layer contain the source image. Original physical structure and texture information; These are the initial target trajectory query, key, and value feature vectors, representing the target edit trajectory initialized in step S1 and subsequently updated by integration. In conjunction with the dynamic geometric anchor point displacement term obtained in step S4 Perform coordinate compensation (i.e., input coordinates are) And after receiving the target text prompt, the feature vector that was originally intended to be output by this network layer; The Hadamard product operator, which performs element-wise matrix multiplication, serves here to allow masking. (Its dimensions are aligned with the Q / K / V matrix) At the pixel / feature block level, the blending ratio of source features and target features is precisely adjusted; These are the target features of the query, key, and value after routing, respectively. They are new feature vectors obtained after weighted fusion. They will directly replace the target features of the source image and be input into the subsequent feedforward network or attention evaluation module of this layer.

[0162] S63. Mathematical Guarantee of Feature Decoupling and Editing Freedom: The above set of formulas thoroughly solves the pain point of "over-constraint caused by structural protection" from a mathematical perspective. Its decoupling logic is rigorously reflected in the spatial gate routing mask. On the extreme value distribution:

[0163] Background protection logic: Spatial gate routing mask in non-editable background areas of an image, or at the physical boundaries of objects.

[0164] The value of approaches 1, at which point the formula... As the term approaches 0, the weights for generating the target feature are reduced to a minimum. This forces the model to only receive the feature stream from the source image in that region (i.e., ...). This achieves pixel-level background protection without any distortion.

[0165] Generative Free Logic: Spatial Gate Routing Masks within an object undergoing drastic semantic modification (i.e., a region far from physical boundaries and with highly focused attention). The value will rapidly decay and approach 0. At this point, the system will completely block the source feature flow (source feature weights tend to 0), allowing the target branch to regain 100% generation dominance (i.e., This mechanism allows the model to still enjoy absolute generative freedom to perform complex semantic modifications (such as color conversion and texture replacement) within the anchored geometric shell, fundamentally decoupling structural constraints from semantic generation.

[0166] S7: Integration and Update of the Decoupled Dynamic System: The dynamic geometric anchor point displacement term is used as coordinate compensation, and the routed target features are used for target velocity field evaluation. A decoupled flow matching differential system is constructed and discretized iteratively performed. Finally, the edited image is decoded and output. This step aims to seamlessly integrate macroscopic geometric correction and microscopic feature routing into the flow matching solver, perform discretized iterative trajectory calculation, and output a high-fidelity edited image after the iteration is completed. The specific implementation process and the mathematical symbols involved are defined as follows:

[0167] S71. Construction of Decoupled Flow-Matched Differential Systems:

[0168] In traditional inversion-free editing, the evaluation coordinates of the target velocity field often result in severe trajectory offsets. This embodiment introduces coordinate compensation and feature routing to construct the following decoupled flow matching differential system:

[0169]

[0170] in: The velocity field for the edited trajectory represents the instantaneous velocity vector of the final edited trajectory at the current time step t. It determines the direction of image evolution from the source state to the target state; The pre-trained velocity field evaluation network represents the core neural network of the flow matching model (such as the diffusion transformer architecture), whose function is to predict the velocity of sample points in the latent space based on given conditions. The current state of the edit trajectory is derived from the initialization in step S1 and the accumulation of the preceding integration steps. It represents the coordinates of the edited image in the latent space at the current moment. The dynamic geometric anchor point displacement term, derived from the displacement amount based on geometric distribution correction (GDC) in step S4, is used to evaluate the target velocity field. Perform coordinate correction to forcibly anchor it to the geometric manifold of the source image, eliminating trajectory offset at the physical level; Spatial gating routing intervention represents intervention in the network. During the internal inference process, the attention layer soft routing operation defined in steps S5 and S6 is executed, the function of which is to utilize the mask. Decouple structural protection features from semantic generation features at the micro level; To correct the surrogate source trajectory state, the state calculated based on the correction noise in step S4 is used as a reference benchmark for the source branch to ensure that the subtraction term (i.e., the source velocity field) has accurate structural guidance. and These represent the target text prompt and the source text prompt, respectively.

[0171] S72. Discretization, Iteration, and Update of Trajectories:

[0172] Since differential systems are difficult to solve directly in the continuous-time domain, discretized numerical integration methods (such as Euler's integral method) are used to solve them step by step.

[0173]

[0174] in: The discrete time step represents the interval between two sampling times, and its value is determined by the total number of sampling steps N (i.e., ...). ); The trajectory state at the next moment represents the updated position of the image in the latent space after one step of velocity integration.

[0175] Computational logic: The system starts a loop from t=1 to t=0, and repeatedly performs velocity field evaluation and coordinate update in each step until all sampling steps are completed.

[0176] S73. Theoretical stability and manifold consistency guarantee: This embodiment demonstrates the stability of the integrated system from a mathematical perspective:

[0177] Lipschitz Continuity: Due to macroscopic compensation terms Derived from bounded priors And micro-routing mechanism The operation performed is a pure convex combination operation, therefore the corrected velocity field The Lipschitz property of the pre-trained vector field is strictly preserved. That is: .

[0178] This mathematical property ensures that the system does not generate divergent truncation errors during iteration. Although this invention deeply decouples geometry and semantics, the synthesized latent features always closely match the natural image manifold learned by the pre-trained model (StableDiffusion 3.5), fundamentally preventing the generation of artifacts.

[0179] S74. Decoding Output: After the iteration, the system obtains the final latent representation. Call the pre-trained decoder Convert it back to pixel space:

[0180]

[0181] in, This refers to the final output image, which has a high-fidelity background and accurate target semantics.

[0182] Effect verification and experimental analysis:

[0183] To verify the superiority of the technical solution of this invention in actual image editing tasks, this embodiment conducted a comprehensive quantitative comparison experiment on the internationally recognized image editing standard benchmark dataset (PIE-Bench).

[0184] 1. Baseline Comparison Method

[0185] To comprehensively evaluate the performance of this invention, this embodiment selects nine mainstream baseline methods covering three different underlying architectures for comparison: methods based on the early latent diffusion architecture (SD version 1.5), including P2P (based on cross-attention control), MasaCtrl (based on mutual self-attention control), PnP (plug-and-play diffusion features), and FreeDiff (based on frequency domain truncation). These methods mainly rely on computationally expensive feature inversion or strong attention interventions.

[0186] Methods based on the large-scale modified flow architecture (FLUX model) include RF-Inv, StableFlow, and RF-Edit. These methods represent the benchmark level of current computationally intensive non-inversion editing technologies.

[0187] Methods based on a novel diffusion converter architecture (SD version 3.5) include FSI-Edit and FlowEdit. These serve as advanced baselines with the same underlying architecture as this invention, used for direct comparison of the merits of the algorithmic logic.

[0188] 2. Definition of Evaluation Indicators

[0189] To scientifically quantify the various physical and perceptual attributes of the generated image, this embodiment adopts the following three sets of core evaluation indicators:

[0190] Structure fidelity metric: Structure distance is used to quantify the degree of distortion in the macroscopic geometric topology of the image before and after editing. The lower the value, the more completely the macroscopic structure is preserved.

[0191] Background preservation capability metrics: These comprehensively employ Peak Signal-to-Noise Ratio (PSNR), Mean Squared Error (MSE), Structural Similarity (SSIM), and Perceptual Loss (LPIPS). These four metrics measure the texture and lighting consistency of the non-edited area (background) from pixel-level error to the level of human visual cortex perception. Higher PSNR and SSIM are better, while lower MSE and LPIPS are better.

[0192] Semantic alignment metric: CLIP Similarity, calculated using a contrastive language-image pre-trained model, is divided into whole-image similarity and edited-area similarity. A higher value indicates a stronger semantic fit between the generated image and the target text prompt, meaning a higher "editing success rate."

[0193] 3. Quantitative data validation and superiority analysis

[0194] Based on the data test results of the comparative experiment, this invention demonstrates significant technical advantages in all core indicators, with its overall average ranking (Rank Avg.) reaching an optimal level of 1.29. Specific analysis is as follows: Extremely strong anti-distortion and background protection capabilities: In terms of structural fidelity, the structural distance of this invention is significantly reduced to an optimal level. This is far superior to RF-Inv based on the FLUX model and FlowEdit based on SD 3.5. In terms of background preservation, the perceptual loss (LPIPS) of this invention achieves the best performance. The mean squared error (MSE) is reduced to the optimal level. The structural similarity (SSIM) reaches an optimal 89.15%. This set of data objectively and directly proves that the embodiments of the present invention can successfully anchor the dynamically generated trajectory firmly onto the source image manifold by constructing a high-dimensional coordinate prior, eliminating the serious trajectory drift and background collapse problems in traditional non-inversion flow matching methods. Breaking excessive constraints and achieving precise semantic alignment: In existing technologies, some methods that emphasize structural protection often sacrifice editing freedom. For example, although the traditional P2P method achieves a background peak signal-to-noise ratio (PSNR) of 27.13, its CLIP similarity in the local edited region is only 21.38, resulting in serious "editing failure" or "feature solidification" phenomena. In contrast, the present invention, while performing strict background protection, achieves a globally optimal CLIP similarity (Edited) of 22.61 in the local edited region and an extremely high CLIP similarity of 25.77 across the entire image. This data confirms the advanced nature of the "Spatial Gated Routing (SGR) module" in this invention: by calculating the physical gradient and confidence mask, the module successfully truncates the source feature flow within the editing area in the deep attention layer, enabling the target semantics to be generated without constraints, thus solving the zero-sum game problem of "background protection" and "generative freedom" in image editing.

[0195] Table 1

[0196]

[0197] Analysis of memory usage and generation speed: The inference time for a single image in this invention is approximately 7.15 seconds (tested with an RTX 5080 GPU), and the memory usage is only 15.18 GB. It can run smoothly on consumer-grade hardware (such as a graphics processor with 16 GB of memory per image).

[0198] Table 2

[0199]

[0200] Ablation experiments verified that, as shown in Table 2, neither geometric distribution correction (GDC) nor spatial gated routing (SGR) alone could achieve optimal results. The synergistic effect of the two is the key to achieving high-fidelity editing.

Claims

1. An image editing method based on geometric distribution correction and spatial gating routing, characterized in that, Includes the following steps: S1: Obtain the source image, source text prompts, and target text prompts; initialize the trajectory and decompose the semantic region of the source image to obtain a set of semantic region masks; S2: Using the semantic region mask set obtained in S1, construct a structural prior graph through rotational position encoding and random geometric projection; S3: At the initial moment of trajectory generation, the standard Gaussian noise is linearly modulated using the structure prior map to generate correction noise, ensuring that the target generated trajectory is anchored on the geometric manifold of the source image. S4: Based on the corrected noise, derive the corrected proxy source trajectory state and dynamic geometric anchor displacement term at time step t respectively; S5: Calculate the spatial gradient and semantic background confidence of the structural prior graph, and construct a spatial gated routing mask through nonlinear fusion; S6: In a predetermined depth layer of the pre-trained generative model, the query vector, key vector, and value vector of the source trajectory feature stream and the target trajectory feature stream are weighted and fused using the spatial gated routing mask to obtain the routed target features; S7: The dynamic geometric anchor point displacement term is used as coordinate compensation, and the routed target features are used for target velocity field evaluation. A decoupled flow matching differential system is constructed and discretized iteratively performed. Finally, the edited image is decoded and output.

2. The image editing method based on geometric distribution correction and spatial gating routing as described in claim 1, characterized in that, S1 includes: S11. Obtain the source image and its corresponding source text prompts and target text prompts ; through pre-trained encoder Map the source image to the latent space to obtain the source image. True coordinates on the potential manifold ; S12. Initialize the starting point of the target editing trajectory, and initialize the initial state of the target editing trajectory to the true latent representation of the source image: ； in, The starting point for editing the target trajectory; N is the total number of sampling steps; FE represents flow matching editing; Represents the initial time step for trajectory solving; S13. Use an image segmentation model to decompose the source image into semantic regions, filter them, and extract the set of semantic region masks. : ； in, K is a single semantic region mask; K is the total number of masks.

3. The image editing method based on geometric distribution correction and spatial gating routing as described in claim 1, characterized in that, S2 includes: S21. For the semantic region mask set obtained in S1 Each single semantic region mask in By using rotational position encoding, the coordinates of pixels inside the semantic region mask are mapped to a high-dimensional feature space, and the regional geometric signature of each semantic region is calculated using the following formula. : ； in: A single semantic region mask; The area coefficient represents the mask of a single semantic region. The total number of valid pixels inside; Normalized pixel coordinates, representing the source image Relative coordinates on a two-dimensional plane; For rotational position encoding function; S22. A deterministic structural hashing mechanism is employed to map the regional geometric signature calculated in S21 to a scalar intensity map using a preset and frozen linear projection vector. And assign a unique intensity identifier to each spatial region; S23. Perform scale alignment and normalization on the scalar intensity map obtained in S22 to construct a structural prior map with numerical values ​​limited to the range of [-1, 1]. .

4. The image editing method based on geometric distribution correction and spatial gating routing as described in claim 3, characterized in that, In step S22, a single-channel scalar intensity map is constructed using the following formula. : ； in: For projected scalar values; The total number of semantic region masks; For an indicator function, if and only if the coordinates Located in the mask When the object is inside the physical boundary, the function takes a value of 1; otherwise, it takes a value of 0. The linear projection calculation method is to use the linear projection vector W and... The inner product of W is initialized from a uniform distribution. And it remains frozen throughout the entire inference and execution process of the algorithm. The dimension of the high-dimensional feature space after the aforementioned rotation position encoding mapping.

5. The image editing method based on geometric distribution correction and spatial gating routing as described in claim 1, characterized in that, S3 includes: At the initial time t=1 of trajectory generation, the sampled standard Gaussian noise is analyzed using the aforementioned structural prior map. Perform linear modulation to generate correction noise : ； in: and These are the structural fidelity coefficient and the random flexibility coefficient, respectively. The corrected noise for the final output; This is the a priori diagram of the structure.

6. The image editing method based on geometric distribution correction and spatial gating routing as described in claim 1, characterized in that, In S4, the calculation formulas for correcting the proxy source trajectory state and the dynamic geometric anchor point displacement term are as follows: ； ； in: To correct the proxy source trajectory status; The dynamic geometric anchor point displacement term; t is the time step; These are the true coordinates of the source image on the latent manifold; The corrected noise.

7. The image editing method based on geometric distribution correction and spatial gating routing as described in claim 1, characterized in that, S5 includes: S51. Based on the structural prior map constructed in S2, the physical boundary contours of the physical objects in the source image are located by calculating the spatial gradient magnitude of the structural prior map: Spatial gradient magnitude The calculation formula is: ； Where x and y represent the coordinate axes in two-dimensional space; This is the a priori diagram of the structure; S52. Extract the dynamic target attention map corresponding to the target text prompt from the cross-attention layer of the target generation trajectory in real time. The background confidence level is calculated using the following formula. : ； in: The normalization function maps the extracted dynamic target attention map to... The probability space; S53. The spatial gradient magnitude is determined through a nonlinear competition mechanism. and background confidence Fusion, constructing the final spatial gating routing mask : ； in: This is the threshold shift hyperparameter, used to set the baseline for the activation boundary; This is a sharpness temperature hyperparameter used to adjust the sensitivity and sharpness of the boundary response; The logistic sigmoid activation function maps the physical boundary gradient, after linear transformation, to... In a continuous nonlinear probability space, make it have the same mathematical dimensions as the confidence matrix; This is the maximum value competition operator.

8. The image editing method based on geometric distribution correction and spatial gating routing as described in claim 1, characterized in that, In step S6, the query vector for the attention mechanism is processed according to the following formula. Key vector Sum value vector Perform software router operations: ； ； ； in: , , These are the query, key, and value features of the source trajectory, respectively; , , These are the query, key, and value features of the target trajectory, respectively. The Hadamard product operator is used for the mixing ratio of source features and target features; , , These are the target features of the query, key, and value after routing; For spatial gating routing mask; When the value of the spatial gated routing mask approaches 1 at the background or physical boundary, the target feature after routing is forced to retain the source trajectory feature; When the spatial gated routing mask approaches 0 inside the object being edited, the target features after routing fully adopt the target trajectory features, thus decoupling structural protection from generation freedom.

9. The image editing method based on geometric distribution correction and spatial gating routing as described in claim 1, characterized in that, S7 includes: Construct the decoupled flow-matched differential system as shown below: ； in: For editing the trajectory velocity field, it represents the instantaneous velocity vector of the final edited trajectory at the current time step t; For pre-trained velocity field evaluation networks; To edit the current state of the trajectory; This refers to the dynamic geometric anchor point displacement term; Intervention for spatial gating routing; For the corrected proxy source trajectory state; and These represent the target text prompt and the source text prompt, respectively. A step-by-step solution is obtained using a discretized numerical integration method: ； in: The discrete time step; This represents the trajectory state at the next moment; After the iterations are completed, the final latent representation is obtained. Call the pre-trained decoder Convert it back to pixel space to obtain the final edited image output.

Images