Image editing method based on characteristic frequency domain fusion, program product and medium

Abstract

The invention discloses an image editing method based on characteristic frequency domain fusion, a program product and a medium, and belongs to the field of image editing. Step-by-step inversion is carried out on the encoded submerged space features to obtain a series of submerged space features of which the noise-containing degree is gradually increased; iterative denoising is carried out to obtain a final denoising feature; in the previous part of iterative denoising, fusing the high-frequency features of the reconstructed stream and the low-frequency features of the generated stream, introducing randomness, and meanwhile, adopting a residual structure based on the features of the generated stream; on the basis, a certain degree of noise is injected into query features and key features of the self-attention mechanism and the cross-attention mechanism. According to the method, effective fusion of features between the reconstructed stream and the generated stream can be realized, feature semantic gaps of different branches are reduced, content editing and background keeping are better performed, and the problems of poor content editing semantic conformity degree and insufficient background keeping degree in a non-rigid editing task in an existing method are effectively solved.

Description

Technical Field

[0001] This invention belongs to the field of image editing, and more specifically, relates to image editing methods, program products, and media based on feature frequency domain fusion. Background Technology

[0002] In recent years, diffusion models have made significant progress in the field of text-to-image (T2I) generation, especially Latent Diffusion Models (LDMs), which have demonstrated excellent performance in accurately converting text descriptions into high-quality images. Therefore, they have been widely applied in various downstream tasks such as image editing and video editing. Figure 1 As shown in (a) above. Image editing typically refers to modifying the content or style of a source image based on user-provided text prompts to generate a target image. This technology has significant practical value in applications such as social media content creation and visual effects production, and has become an important research direction in the field of visual generation.

[0003] Image editing based on the latent diffusion model typically follows a standardized processing workflow. First, the input source image is converted into its corresponding latent noise representation using the DDIM inversion method, or a more advanced inversion strategy. Then, starting from this latent noise representation, the system initiates two denoising processes in parallel: one reconstructs the source image based on the original text prompts, and the other generates a modified image based on the target text prompts. The key technical challenge in this process lies in how to introduce the required semantic modifications while preserving the original image content. To address this, an information injection mechanism is often employed, introducing information flow between the reconstruction and generation paths, thereby accurately guiding the semantic transformation while maintaining the consistency of the image structure.

[0004] A simple approach is to partially mix the latent noise and random noise of the source image in the initial stage of the generation process, usually done in the frequency domain. In recent years, several studies have further explored information injection strategies at intermediate feature levels, such as... Figure 1As shown in (b) above, this type of method primarily focuses on manipulating features such as queries, keys, and values ​​in self-attention or cross-attention modules to achieve more refined and controllable image editing. Specifically, during the generation of the target image, the system selects several key attention layers and directly replaces some of their features (such as queries, keys, or values) with corresponding features in the source image reconstruction stream. This operation provides strong visual references and structural constraints for preserving image regions that do not need to be edited, without interfering with the target semantic guidance.

[0005] Existing methods excel at handling rigid editing tasks (such as color adjustment and style transfer), but perform poorly on non-rigid editing tasks (such as object addition, deletion, or significant changes in pose / structure). Non-rigid editing tasks typically require large-scale reconstruction of the target object and its surrounding environment. In such scenarios, the editing area often naturally extends beyond the object itself, posing two main challenges to the widely adopted "direct feature replacement" strategy: First, there is a semantic difference between the reconstruction and generation flows. In most methods, directly injecting attention features from the reconstruction branch into the generation branch often fails to faithfully preserve the original image content. Especially in cases of significant pose changes or viewpoint shifts, this strategy easily introduces artifacts, structural distortions, or semantic inconsistencies, affecting the visual quality of the editing results. Second, non-rigid editing relies more heavily on the generative capabilities of the base model. These tasks require models capable of overcoming the semantic constraints imposed by strong inductive biases in the original image, enabling more free and flexible content generation. Relying solely on semantic cues for editing is often insufficient to guide the model to generate new content highly consistent with the target semantics, limiting the diversity and accuracy of the final editing results.

[0006] In summary, current mainstream methods often struggle to achieve an effective balance between "preserving the fidelity of the original image content" and "generating edited results that are consistent with the target semantics" when faced with non-rigid editing needs, and there is an urgent need for editing mechanisms with greater generalization ability and controllability. Summary of the Invention

[0007] In response to the shortcomings and improvement needs of existing technologies, this invention provides an image editing method, program product and medium based on feature frequency domain fusion, with the aim of achieving flexible and high-fidelity non-rigid editing.

[0008] To achieve the above objectives, according to one aspect of the present invention, an image editing method based on feature frequency domain fusion is provided, comprising:

[0009] S1: Transfer the source image I srcThe feature is encoded as a latent space feature x0, and then x0 is progressively noisyened to a Gaussian distribution space through inversion to obtain a series of noisy latent space features. T is the preset number of time steps;

[0010] S2: x T As latent space features after adding noise The latent space features are then subjected to a series of denoising steps to obtain the denoised features.

[0011] S3: Features of latent space Decoding back to image space yields the edited image I. tar ;

[0012] Among them, for Perform the noise reduction steps, including:

[0013] S21: If t>T th If yes, proceed to S22; otherwise, proceed to S26; 1 <T th <T;

[0014] S22: Latent space feature x t and source image I src Description text Input the query features into the diffusion model to obtain the query features output by the cross-attention module within the diffusion model. Key features Sum value characteristics and the query features output by the self-attention module Key features Sum value characteristics

[0015] S23: Features of latent space and target text Input the query features into the diffusion model to obtain the query features output by the cross-attention module within the diffusion model. Key features Sum value characteristics and the query features output by the self-attention module Key features Sum value characteristics

[0016] S24: Will low-frequency components and The high-frequency components are fused and superimposed on Obtain features Will low-frequency components and The high-frequency components are fused and superimposed on Obtain features The frequency of low-frequency components is lower than the preset cutoff frequency, and the frequency of high-frequency components is not lower than the cutoff frequency;

[0017] S25: Keep the input of the diffusion model constant, and change the input of the diffusion model... and Replace with and Obtain the denoised latent space features Denoising at the current time step is complete;

[0018] S26: Latent space features and target text Input diffusion model to obtain denoised latent space features The denoising process for the current time step is complete.

[0019] Furthermore, in step S25, the following steps will be performed: Replace with Previously included: Inject random noise, and, Replace with Previously included: Inject random noise.

[0020] Furthermore, regarding Injecting random noise, the formula is as follows:

[0021]

[0022] right Injecting random noise, the formula is as follows:

[0023]

[0024] Wherein, η1, η2, σ f1 and σ f2 All parameters are preset. This represents normally distributed noise. Indicates to Features obtained after injecting random noise Indicates to Features obtained after injecting random noise.

[0025] Furthermore, in step S25, the following steps will be performed: Replace with Previously included: Inject random noise, and, Replace with Previously included: Inject random noise.

[0026] Furthermore, regarding Injecting random noise, the formula is as follows:

[0027]

[0028] right Injecting random noise, the formula is as follows:

[0029]

[0030] Where, σ q and σ k All parameters are preset. This represents normally distributed noise. Indicates to Features obtained after injecting random noise Indicates to Features obtained after injecting random noise.

[0031] Further, in step S2, x T Latent space features after denoising Previously, it also included: for x T Make corrections;

[0032] Furthermore, in step S22, the latent space feature x t and source image I src Description text Before inputting the diffusion model, it also includes: for x t Make corrections;

[0033] Among them, for x t Make corrections, including: adjusting x t The high-frequency components and x t-1 Low-frequency component fusion, as the corrected x t .

[0034] Furthermore, regarding x t The formula has been revised as follows:

[0035]

[0036] Where FFT(·) represents the two-dimensional Fast Fourier Transform, and IFFT(·) represents the two-dimensional Inverse Fast Fourier Transform. and These are Gaussian high-pass and low-pass filters, respectively. σ represents normally distributed noise. x This represents a parameter used to control the intensity of introduced noise; Indicates the corrected x t .

[0037] Furthermore, low-frequency components and The high-frequency components are fused and superimposed on and will low-frequency components and The high-frequency components are fused and superimposed on The formula is as follows:

[0038]

[0039] Where FFT(·) represents the two-dimensional Fast Fourier Transform, and IFFT(·) represents the two-dimensional Inverse Fast Fourier Transform. and These are Gaussian high-pass and low-pass filters, respectively; α represents the preset parameter. or,

[0040] According to another aspect of the present invention, a computer program product is provided, comprising a computer program; when the computer program is executed by a processor, it implements the above-described image editing method based on feature frequency domain fusion provided by the present invention.

[0041] According to another aspect of the present invention, a computer-readable storage medium is provided, including a stored computer program; when the computer program is executed by a processor, it controls the device where the computer-readable storage medium is located to execute the above-described image editing method based on feature frequency domain fusion provided by the present invention.

[0042] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0043] (1) When generating an edited image based on the target text, this invention fuses the low-frequency components of some features in the generation branch with the high-frequency components of the corresponding features in the reconstruction branch and then superimposes them onto the features in the generation branch, replacing the corresponding features in the generation branch. This realizes a frequency domain residual fusion mechanism between the features in the generation branch and the reconstruction branch. This mechanism can effectively fuse the features between the reconstruction stream and the generation stream, thereby narrowing the feature semantic gap between different branches, so as to better edit the content and maintain the background. Thus, while synthesizing a global layout and structure that conforms to the semantic description of the target text, it maintains the fine-grained texture and identity features of the source image and improves the quality of the edited image.

[0044] (2) In a preferred embodiment of the present invention, random noise is injected into the features before feature replacement, thereby realizing a random noise injection mechanism. This mechanism can introduce controllable randomness into the attention matrix of the diffusion model, thereby further releasing the potential of the diffusion model in image editing, enhancing the generative expressive ability, and thus supporting greater freedom in structural reconstruction and semantic transformation, which is more conducive to non-rigid editing tasks.

[0045] (3) In a preferred embodiment of the present invention, after the image is inverted, the high-frequency components of the latent space features of the current time step and the low-frequency components of the latent space features of the previous time step are fused to correct the latent space features of the current time step, thereby using cleaner features and guiding the denoising of the current features.

[0046] (4) This invention achieves high-quality non-rigid image editing by performing feature frequency domain residual fusion and randomness injection between the reconstruction branch and the generation branch, without the need to train or fine-tune the diffusion model. Attached Figure Description

[0047] Figure 1 The diagrams illustrate existing image editing methods; where (a) is a schematic diagram of the text-to-image process, and (b) is a schematic diagram of existing image editing methods.

[0048] Figure 2 This is a schematic diagram of an image editing method based on feature frequency domain fusion provided in an embodiment of the present invention;

[0049] Figure 3 This is a flowchart of an image editing method based on feature frequency domain fusion provided in an embodiment of the present invention;

[0050] Figure 4 This is a quantitative comparison of the present invention with other image editing methods on PIE-Bench.

[0051] Figure 5 This is a quantitative comparison of the results of this invention with other image editing methods on non-rigid editing tasks.

[0052] Figure 6 This is a comparison chart of the image editing results of the method of this invention and other methods.

[0053] Figure 7 This is a comparison chart of the results of the method of this invention and other methods in non-rigid image editing.

[0054] Figure 8 and Figure 9 This is a comparison chart of the qualitative results of the method of the present invention on different types of editing tasks. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0056] In this invention, the terms "first," "second," etc. (if present) in the invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0057] To address the issues of poor semantic consistency and insufficient background preservation in existing image editing methods for non-rigid editing tasks, this invention provides an image editing method, program product, and medium based on feature frequency domain fusion. The related method provides an image editing framework that does not require parameter adjustment. Based on the image editing framework provided by this invention, the feature gap between the reconstruction stream and the generation stream can be reduced, and the generation capability of the diffusion model can be released, thereby achieving flexible and high-fidelity non-rigid editing.

[0058] The following is an example.

[0059] Example 1:

[0060] An image editing method based on feature frequency domain fusion, such as Figure 2 and Figure 3 As shown. This embodiment includes the following steps S1 to S3, each step of which is detailed below:

[0061] S1: Transfer the source image I src The feature is encoded as a latent space feature x0, and then x0 is progressively noisyened to a Gaussian distribution space through inversion to obtain a series of noisy latent space features. T is the preset number of time steps.

[0062] Optionally, this embodiment utilizes the encoder ε of the self-encoder VAE to process the source image I. src Encoded as latent space feature x0, where,

[0063] Optionally, this embodiment is based on DiT, therefore using the Rectified Flow inversion method, and employing descriptive statements for the source image during this process. Accordingly, the inversion formula for each time step is as follows:

[0064]

[0065] Where, σ t and σ t-1 v is a constant coefficient for the corresponding time step; θ This represents DiT. The noise level of the latent space features obtained through the above inversion gradually increases.

[0066] To achieve stable reconstruction of the source image, this embodiment obtains a series of state sequences {x} in the latent space through inversion. T ,x T-1After x1, ..., x1}, the modified latent space features will be constructed. Among them, the corrected latent space features at each time step By fusing the current moment x t The high-frequency components and the previous time x t-1 The low-frequency components are obtained, and the specific formula is as follows:

[0067]

[0068] Where FFT(·) represents the two-dimensional Fast Fourier Transform, and IFFT(·) represents the two-dimensional Inverse Fast Fourier Transform. and These are Gaussian high-pass and low-pass filters, respectively. σ = 0.3 controls the frequency domain fusion ratio. x =1e-3 controls the intensity of the introduced Gaussian noise; This represents Gaussian noise.

[0069] This embodiment converts noise to the frequency domain using a two-dimensional fast Fourier transform; high-frequency components of the corresponding features are obtained through a high-pass filter, and low-frequency components (cleaner features) of the corresponding features are obtained through a low-pass filter; through frequency domain fusion of the above latent space features, the cleaner features can be used to guide the denoising of the features at the current time step.

[0070] The modified latent space features constructed using the above modified formula It is only used to extract feature information from the source image for subsequent cross-branch interaction. Both the source image reconstruction branch and the target image generation branch are based on... As initial input, this processing method effectively stabilizes the overall structure of the source image, introducing a moderate degree of randomness while preserving details, thereby enhancing the flexibility and naturalness of editing.

[0071] S2: x T As latent space features after adding noise The latent space features are then subjected to a series of denoising steps to obtain the denoised features.

[0072] One of the core challenges in image editing lies in reconciling the semantic and structural differences between the source and target images. This difference is particularly pronounced during non-rigid editing (such as adding objects, changing poses, and adjusting structures), where the generated target image must satisfy the user's semantic prompts while maintaining visual consistency with the source image as much as possible. To address this issue, this embodiment uses the proposed Frequency Residual Fusion (FRF) feature alignment method in step S2. This method achieves feature-level coordination by fusing the attention features of the reconstruction and generation branches in the frequency domain. Its core idea is to retain the low-frequency components of the generation branch (used to synthesize a global layout and structure that conforms to the semantic description) and inject high-frequency residual information from the reconstruction branch (to maintain the fine-grained texture and identity features of the source image).

[0073] Specifically, for any latent space feature at time step t Perform the noise reduction steps, including:

[0074] S21: If t>T th If yes, proceed to S22; otherwise, proceed to S26; 1 <T th <T;

[0075] S22: Latent space feature x t and source image I src Description text Input the query features into the diffusion model to obtain the query features output by the cross-attention module within the diffusion model. Key features Sum value characteristics and the query features output by the self-attention module Key features Sum value characteristics

[0076] S23: Features of latent space and target text Input the query features into the diffusion model to obtain the query features output by the cross-attention module within the diffusion model. Key features Sum value characteristics and the query features output by the self-attention module Key features Sum value characteristics

[0077] S24: Will low-frequency components and The high-frequency components are fused and superimposed on Obtain features Will low-frequency components and The high-frequency components are fused and superimposed on Obtain features The frequency of low-frequency components is lower than the preset cutoff frequency, and the frequency of high-frequency components is not lower than the cutoff frequency;

[0078] Let the reconstructed branch feature be represented as The branch generation feature is represented as We define the frequency domain fusion operation as follows:

[0079]

[0080] Wherein, the cutoff frequency is set to σ = 0.3; C represents the number of feature channels, H and W represent the height and width of the feature, respectively; α represents the preset parameter; the fused frequency domain features are then restored to the spatial domain by inverse Fourier transform (IFFT) (·), and superimposed onto the target features in a residual manner, resulting in:

[0081]

[0082] In this embodiment, or,

[0083] S25: Keep the input of the diffusion model constant, and change the input of the diffusion model... and Replace with and Obtain the denoised latent space features Denoising at the current time step is complete;

[0084] This embodiment achieves coordinated preservation of large-scale structural deformation and background details by fusing high-frequency features from the reconstructed stream and low-frequency features from the generated stream, and by introducing randomness. A residual structure based on generated stream features is employed to enhance the fusion effect of features from different frequency bands, thereby improving the semantic consistency and image quality of the editing results.

[0085] In step S25, the FRF frequency domain residual fusion mechanism is mainly applied to the query and key features in the self-attention layer to achieve feature alignment in attention computation:

[0086] Feature substitution can be expressed by the following formula;

[0087]

[0088] in, and They represent the replaced... and It is easy to understand that in the relevant symbol representation, the superscript "self" of Q, K, and V indicates that these features belong to the self-attention mechanism, and the subscripts "tgt" and "src" indicate that the features come from the generation stream and the reconstruction stream, respectively.

[0089] The selective frequency mixing designed in this embodiment helps to narrow the source-target feature gap, enabling real and flexible editing on different contents while avoiding semantic drift or prompt misalignment.

[0090] S26: Input the latent space features and the target text into the diffusion model to obtain the denoised latent space features The denoising at the current time step ends.

[0091] In this embodiment, steps S22 to S25 are only iteratively executed for T th (1 < T th < T) time steps. Since the reconstruction branch and the generation branch perform two different tasks, interactions are only performed in some of the initial time steps of the denoising process to enhance the generation expression ability, and in the remaining time steps, the two branches perform their respective tasks to adapt to their own tasks.

[0092] In practical applications, T th can be flexibly set according to the specific image editing task. Considering that the rigid editing task makes fewer changes to the image, more interactions can be performed on the image to transfer the source image information to the target image. Therefore, in practical applications, the value of T th set for the rigid editing task is greater than that for the non-rigid editing task. For example, the total number of time steps is T = 50. For the non-rigid editing task, it is only executed in the first 25 steps, while for the rigid editing task, it is extended to the first 30 steps.

[0093] Through the above iterative denoising process, the denoised latent space features

[0094] S3: Decode the latent space features back to the image space to obtain the edited image I tar .

[0095] Corresponding to step S1, in step S26 of this embodiment, the decoder of the autoencoder VAE is used to decode it back to the image space to obtain the final edited image I tar .

[0096] Non-rigid editing, especially when involving large structural transformations (such as transforming a bird into an "X" shape), places higher demands on the flexibility of generative models. However, existing methods often fail to fully utilize the generative capabilities of latent diffusion models, resulting in limited editing scope and difficulty in achieving complex structural transformations.

[0097] The randomness DDIM model demonstrates that appropriately introducing noise can enhance the diversity of the inversion process. Inspired by this, this embodiment, while narrowing the feature gap between the reconstruction and generation streams, further introduces controllable randomness into the attention mechanism to enhance generative expressiveness, thereby supporting greater freedom in structural reconstruction and semantic transformation. To this end, this embodiment proposes a random noise injection mechanism to achieve semantic neighborhood interaction.

[0098] Specifically, in this embodiment, Gaussian noise is injected into the Query and Key of the generated branch in the cross-attention layer, and the Value of the reconstructed branch is replaced with the Value of the generated branch to maintain semantic consistency. Furthermore, in the self-attention layer, noise is injected into the Query and Key after the frequency domain fusion module (FRF) operation. Correspondingly, in step S25, noise is injected into the Query and Key. Replace with Previously included: Inject random noise, and, Replace with Previously included: Inject random noise; Replace with Previously included: Inject random noise, and, Replace with Previously included: Inject random noise. Injecting random noise, the formula is as follows:

[0099]

[0100] right Injecting random noise, the formula is as follows:

[0101]

[0102] right Injecting random noise, the formula is as follows:

[0103]

[0104] right Injecting random noise, the formula is as follows:

[0105]

[0106] Wherein, η1, η2, σ f1 and σ f2 All parameters are preset. This represents normally distributed noise. Indicates to Features obtained after injecting random noise Indicates to Features obtained after injecting random noise; σ q and σ k All parameters are preset; Indicates to Features obtained after injecting random noise Indicates to Features obtained after injecting random noise. Relevant parameters can be flexibly set according to actual needs. Optionally, in this embodiment, σ q =σ k =0.1, η1=η2=0.2, σ f1 =σ f2 =0.8.

[0107] Since the weight calculation of the attention mechanism depends on the dot product of the query and the key (e.g., QK), T Therefore, introducing bounded noise into the frequency domain fusion features does not affect the numerical stability.

[0108] This stochastic mechanism provides the model with the ability to escape the structural constraints of the source image, enabling it to adapt more flexibly to diverse and complex editing needs while maintaining semantic alignment. For rigid editing tasks, v q , σ k , σ f1 , σ f2 Gradually reduce it to 0, and let η→1 to restore the deterministic attention mechanism, thereby achieving precise preservation and restoration of the structure.

[0109] In summary, this embodiment provides a novel frequency domain fusion mechanism: feature-based frequency domain residual fusion (FRF). This mechanism enables effective fusion of features between the reconstruction and generation streams, thereby reducing the semantic gaps between features in different branches and facilitating better content editing and background preservation. Building upon this, this embodiment further proposes a random noise injection (SNI) mechanism. This mechanism introduces controllable randomness into the attention matrix of the diffusion model, further unleashing the potential of the basis generative model in image editing, making it more beneficial for non-rigid editing tasks. This embodiment achieves high-quality non-rigid image editing without training or fine-tuning by performing feature-based frequency domain residual fusion and random injection between the reconstruction and generation streams.

[0110] Example 2:

[0111] A computer program product includes a computer program; when the computer program is executed by a processor, it implements the image editing method based on feature frequency domain fusion provided in Embodiment 1 above.

[0112] Example 3:

[0113] A computer-readable storage medium includes a stored computer program; when the computer program is executed by a processor, it controls the device where the computer-readable storage medium is located to execute the image editing method based on feature frequency domain fusion provided in Embodiment 1 above.

[0114] The following explanation, based on comprehensive evaluation results on the PIE-Bench benchmark dataset, further illustrates the beneficial effects of this invention. PIE-Bench contains 700 image-text instruction pairs, covering 10 different image editing types, used to test the model's performance in both rigid and non-rigid editing tasks. To further verify the adaptability of this invention in non-rigid editing tasks, approximately 300 images involving structural deformation scenarios such as target addition, deletion, and pose changes were selected from this dataset, and corresponding editing instruction sets were constructed. Furthermore, to verify the optimal performance of this invention, the following comparisons are all based on Example 1 above. This invention has been comprehensively compared with other existing image editing methods based on LDM and DiT, and the quantitative and qualitative experimental results further demonstrate the effectiveness and superiority of the method of this invention.

[0115] To comprehensively evaluate the performance of the method of this invention, six complementary evaluation metrics were used in the experiment. Structure distance was used to measure the structural similarity between the edited image and the original image. Peak signal-to-noise ratio (PSNR), perceptual image similarity (LPIPS), mean squared error (MSE), and structural similarity index (SSIM) were used to jointly measure the content preservation of the unedited region. CLIP similarity was used to evaluate the semantic consistency between the editing result and the input text instruction, including two granularities: whole image and edited region.

[0116] Across all editing types, Figure 4 The average performance across six evaluation metrics was summarized. The method of this invention exhibits higher CLIP similarity, demonstrating superior text understanding and semantic preservation capabilities. Furthermore, metrics such as LPIPS and SSIM show that it outperforms existing methods like P2P and FreeDiff in preserving structure and detail in unedited regions. These results demonstrate that the present invention achieves an optimal balance between semantic alignment and content preservation. Comparison results for non-rigid editing tasks are shown below. Figure 5As shown, this invention achieves superior performance in non-rigid editing tasks, significantly outperforming existing methods across multiple metrics. Particularly in terms of structural consistency and text alignment, this invention effectively completes complex semantic transformations while maintaining the realism of the background region, validating the effectiveness of the proposed Feature Reconstruction Mechanism (FRF) and Random Noise Injection Mechanism (SNI) in complex editing tasks. Furthermore, based on a lightweight DiT architecture, this invention can achieve an editing speed of approximately 20 seconds per image on a single RTX 4090 GPU, combining high efficiency with deployment friendliness, making it suitable for large-scale application scenarios.

[0117] Figure 6 and Figure 7 The results show a visual comparison between the present invention and existing advanced methods under rigid and non-rigid tasks. Figure 7 As an example of a non-rigid editing scenario, in line 3, the invention accurately transforms the bird's posture into an "X" shape. In line 5, while deleting the phone, it preserves background details such as the vase and watch, demonstrating strong structural fidelity and semantic flexibility. Overall, the invention exhibits good versatility and robustness across different editing types. Particularly in non-rigid editing tasks with significant structural deformation, its accurate semantic understanding and flexible image generation capabilities enable it to complete complex instruction editing while maintaining image realism, demonstrating significant technical advantages.

[0118] Figure 8 and Figure 9 This paper presents qualitative comparison results of the method of the present invention in different types of editing tasks. To verify the generality of the method, several typical editing types were selected for testing, covering rigid editing (such as color, material, and style changes) and non-rigid editing (such as object addition, deletion, and pose adjustment). The results show that the method of the present invention can generate high-fidelity images in various editing scenarios, demonstrating excellent robustness and generalization ability.

[0119] In summary, the non-rigid image editing method proposed in this invention, based on feature frequency domain fusion and random noise injection, effectively improves the semantic alignment and structural fidelity of image editing without requiring additional training or fine-tuning. Compared to existing methods, this invention does not rely on a large network structure or complex loss function design. Instead, it enhances the model's responsiveness to complex editing instructions while maintaining the authenticity of unedited regions through a lightweight feature frequency domain residual fusion (FRF) mechanism and random noise injection (SNI) mechanism. The quantitative and qualitative results fully verify the effectiveness, versatility, and robustness of this invention in various image editing scenarios, demonstrating significant technical advantages and broad application potential.

[0120] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image editing method based on feature frequency domain fusion, characterized in that, include: S1: Transfer the source image I src The feature is encoded as a latent space feature x0, and then x0 is progressively noisyened to a Gaussian distribution space through inversion to obtain a series of noisy latent space features. T is the preset number of time steps; S2: x T As latent space features after adding noise The latent space features are then subjected to a series of denoising steps to obtain the denoised features. S3: Features of latent space Decoding back to image space yields the edited image I. tar ; Among them, for Perform the noise reduction steps, including: S21: If t>T th If yes, proceed to S22; otherwise, proceed to S26; 1 <T th <T; S22: Latent space feature x t and the source image I src Description text Input the diffusion model to obtain the query features output by the cross-attention module in the diffusion model. Key features Sum value characteristics and the query features output by the self-attention module Key features Sum value characteristics S23: Features of latent space and target text Input the diffusion model to obtain the query features output by the cross-attention module in the diffusion model. Key features Sum value characteristics and the query features output by the self-attention module Key features Sum value characteristics S24: Will low-frequency components and The high-frequency components are fused and superimposed on Obtain features Will low-frequency components and The high-frequency components are fused and superimposed on Obtain features The frequency of the low-frequency component is lower than the preset cutoff frequency, and the frequency of the high-frequency component is not lower than the cutoff frequency; S25: Keep the input of the diffusion model unchanged, and change the input of the diffusion model... and Replace with and Obtain the denoised latent space features Denoising at the current time step is complete; S26: Latent space features and target text Input the diffusion model to obtain the denoised latent space features. The denoising process for the current time step is complete.

2. The image editing method based on feature frequency domain fusion as described in claim 1, characterized in that, In step S25, Replace with Previously included: Inject random noise, and, Replace with Previously included: Inject random noise.

3. The image editing method based on feature frequency domain fusion as described in claim 2, characterized in that, right Injecting random noise, the formula is as follows: right Injecting random noise, the formula is as follows: Wherein, η1, η2, σ f1 and σ f2 All parameters are preset. This represents normally distributed noise. Indicates to Features obtained after injecting random noise Indicates to Features obtained after injecting random noise.

4. The image editing method based on feature frequency domain fusion as described in claim 1, characterized in that, In step S25, Replace with Previously included: Inject random noise, and, Replace with Previously included: Inject random noise.

5. The image editing method based on feature frequency domain fusion as described in claim 4, characterized in that, right Injecting random noise, the formula is as follows: right Injecting random noise, the formula is as follows: Where, σ q and σ k All parameters are preset. This represents normally distributed noise. Indicates to Features obtained after injecting random noise Indicates to Features obtained after injecting random noise.

6. The image editing method based on feature frequency domain fusion as described in any one of claims 1 to 5, characterized in that, In step S2, x T Latent space features after denoising Previously, it also included: for x T Make corrections; Furthermore, in step S22, the latent space feature x t and the source image I src Description text Before inputting the diffusion model, it also includes: for x t Make corrections; Among them, for x t Make corrections, including: adjusting x t The high-frequency components and x t-1 Low-frequency component fusion, as the corrected x t .

7. The image editing method based on feature frequency domain fusion as described in claim 6, characterized in that, For x t The formula has been revised as follows: Where FFT(·) represents the two-dimensional Fast Fourier Transform, and IFFT(·) represents the two-dimensional Inverse Fast Fourier Transform. and These are Gaussian high-pass and low-pass filters, respectively. σ represents normally distributed noise. x This represents a parameter used to control the intensity of introduced noise; Indicates the corrected x t .

8. The image editing method based on feature frequency domain fusion as described in any one of claims 1 to 5, characterized in that, Will low-frequency components and The high-frequency components are fused and superimposed on and will low-frequency components and The high-frequency components are fused and superimposed on The formula is as follows: Where FFT(·) represents the two-dimensional Fast Fourier Transform, and IFFT(·) represents the two-dimensional Inverse Fast Fourier Transform. and These are Gaussian high-pass and low-pass filters, respectively; α represents the preset parameter. or, 9. A computer program product, characterized in that, It includes a computer program; when the computer program is executed by a processor, it implements the image editing method based on feature frequency domain fusion as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer program includes a stored computer program; when executed by a processor, the computer program controls the device containing the computer-readable storage medium to perform the image editing method based on feature frequency domain fusion as described in any one of claims 1 to 8.

Images