Diffusion model single image generation method and system based on frequency domain guidance

Abstract

The invention discloses a diffusion model single image generation method and system based on frequency domain guidance, and relates to the technical field of computer vision, and the method comprises the steps: obtaining a single training image, and defining a reference network based on a diffusion model; establishing a composite frequency domain constraint function; establishing an iterative structure refining strategy; and performing end-to-end training on the reference network based on the composite frequency domain constraint function, after the training is completed, performing reverse denoising sampling in combination with the iterative structure refining strategy, and generating a new image similar to the training image in distribution. Through dual guidance of frequency domain constraint in a training stage and structure refining in an image generation stage, on the premise of not changing a reference network core architecture, the structural integrity and visual fidelity of the generated image can be improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and system for generating single images based on a frequency domain-guided diffusion model. Background Technology

[0002] Single-image generation is an important research direction in computer vision. Its core goal is to learn the inherent visual statistical laws based on a single image sample, and then generate a series of novel images. Existing single-image generation methods are mainly divided into two categories: non-parametric methods and parametric methods. Non-parametric methods synthesize new images by sampling and recombining source image patches. While they can preserve local textures, they struggle to generate novel global structures. Parametric methods include those based on Generative Adversarial Networks (GANs) and those based on denoising diffusion models. GAN-based methods suffer from problems such as training instability, pattern collapse, and impaired structural integrity. While diffusion model-based methods offer advantages such as high generation quality and stable training, they typically employ small receptive field models to avoid overfitting to single samples. This results in insufficient perception of global structures, leading to problems such as macroscopic structural distortion, collapse of key content, and blurring of high-frequency details in the generated images.

[0003] Frequency domain information can effectively decouple low-frequency structural components from high-frequency detail components in an image, providing strong support for improving image generation quality. In existing technologies, frequency domain information has been applied to tasks such as domain adaptation, image super-resolution, and style transfer, demonstrating its effectiveness in guiding the image generation process and optimizing specific image attributes. Therefore, how to utilize frequency domain information to compensate for the structural perception deficiencies caused by the limited receptive field in diffusion models during single image generation has become a key technical issue for improving the quality of single image generation. Summary of the Invention

[0004] The purpose of this application is to provide a single image generation method and system based on a frequency domain-guided diffusion model. By using frequency domain constraints during the training phase and structural refinement during the image generation phase as dual guidance, the structural integrity and visual fidelity of the generated images are significantly improved without changing the core architecture of the baseline network.

[0005] To achieve the above objectives, this application provides the following solution.

[0006] In a first aspect, this application provides a single image generation method based on a frequency domain-guided diffusion model, which includes the following steps.

[0007] A single training image is acquired, and a baseline network based on a diffusion model is defined; the baseline network employs a neural network architecture for noise prediction to achieve a reverse denoising process from a noisy image to a clear image.

[0008] A composite frequency domain constraint function is established; the composite frequency domain constraint function is an explicit constraint on the frequency domain components of the image added on the basis of the noise prediction loss during the training phase of the benchmark network; the composite frequency domain constraint function includes constraint terms for low-frequency components of the image and constraint terms for high-frequency components of the image.

[0009] An iterative structural refinement strategy is established; the iterative structural refinement strategy is to use the low-frequency information of the original training image to correct the image generation process in real time in each inverse denoising step of the image generation stage, so as to suppress error accumulation.

[0010] The baseline network is trained end-to-end based on the composite frequency domain constraint function. After training, reverse denoising sampling is performed in conjunction with the iterative structure refinement strategy to generate a new image with a distribution similar to the training image.

[0011] Optionally, the benchmark network is a noise prediction network based on the U-Net architecture; the training process of the benchmark network includes a fixed forward noise addition process and a backward noise reduction process that needs to be learned; in the forward noise addition process, Gaussian noise is gradually added to the training image within multiple discrete time steps; in the backward noise reduction process, the benchmark network receives the noise-added image and the time step as input and predicts the added noise.

[0012] Optionally, the constraints on the low-frequency components of the image include low-frequency spatial domain structure loss and low-frequency frequency domain amplitude loss.

[0013] The low-frequency spatial domain structure loss is achieved by imposing constraints on the low-frequency part of the predicted image in the spatial domain. Two-dimensional fast Fourier transform is used to separate the low-frequency components of the spectrum, and the loss is calculated by combining a preset low-pass filter mask and the L1 norm.

[0014] The low-frequency amplitude loss is achieved by imposing constraints on the Fourier amplitude spectrum of the low-frequency components, which is used to monitor the energy distribution and contrast of the low-frequency components.

[0015] Optionally, the formula for calculating the low-frequency spatial domain structure loss is as follows.

[0016] .

[0017] in, This indicates the low-frequency spatial domain structure loss; This represents a two-dimensional Fast Fourier Transform; This indicates the prediction of the training image; Represents real training images; This indicates element-wise multiplication; The preset ideal low-pass filter mask; This represents the L1 norm.

[0018] Optionally, the formula for calculating the low-frequency amplitude loss is as follows.

[0019] .

[0020] in, This indicates the amplitude loss in the low-frequency domain; Represents the Fourier amplitude spectrum of the image; This indicates the prediction of the training image; Represents real training images; These are the frequency coordinates after the Fourier transform of the image; This represents the mathematical expectation of the frequency coordinates; This is the preset ideal low-pass filter mask.

[0021] Optionally, the constraint term for the high-frequency components of the image is a high-frequency domain logarithmic amplitude loss.

[0022] The high-frequency domain logarithmic amplitude loss employs logarithmic loss to penalize the relative differences in high-frequency amplitudes in order to encourage the generation of clear details.

[0023] Optionally, the formula for calculating the high-frequency domain logarithmic amplitude loss is as follows.

[0024] .

[0025] in, This represents the logarithmic amplitude loss in the high-frequency domain; ε is a high-pass mask; ε is a small constant used to ensure numerical stability. These are the frequency coordinates after the Fourier transform of the image; Represents the Fourier amplitude spectrum of the image; This indicates the prediction of the training image; Represents real training images; This represents the mathematical expectation of the frequency coordinates.

[0026] Optionally, the real-time correction process of the iterative structure refinement strategy includes: calculating the difference between the predicted training image and the real training image in the low-frequency amplitude spectrum in the current denoising step, and using the difference as the structure correction gradient to update the predicted training image. The updated image is used for calculation in subsequent denoising steps.

[0027] Optionally, in the end-to-end training, the total loss function is composed of the composite frequency domain loss corresponding to the composite frequency domain constraint function and the pixel-level noise prediction loss weighted together, and an optimizer is used to iteratively train the baseline network. The pixel-level noise prediction loss is either the L1 loss or the L2 loss between the predicted noise and the real noise.

[0028] Secondly, this application provides a frequency-domain guided diffusion model single image generation system, which is used to implement the frequency-domain guided diffusion model single image generation method described in any of the first aspects. The frequency-domain guided diffusion model single image generation system includes the following modules.

[0029] A baseline network module is used to acquire a single training image and define a baseline network based on a diffusion model; the baseline network adopts a neural network architecture for noise prediction to achieve a reverse denoising process from a noisy image to a clear image.

[0030] A composite frequency domain constraint module is used to establish a composite frequency domain constraint function. The composite frequency domain constraint function is an explicit constraint on the image frequency domain components added on top of the noise prediction loss during the training phase of the baseline network. The composite frequency domain constraint function includes constraint terms for low-frequency components of the image and constraint terms for high-frequency components of the image.

[0031] The structure refinement module is used to establish an iterative structure refinement strategy. The iterative structure refinement strategy is to use the low-frequency information of the original training image to correct the image generation process in real time in each inverse denoising step of the image generation stage, so as to suppress error accumulation.

[0032] The training and generation module is used to perform end-to-end training of the baseline network based on the composite frequency domain constraint function. After training, it performs inverse denoising sampling in combination with the iterative structure refinement strategy to generate a new image with a distribution similar to the training image.

[0033] According to the specific embodiments provided in this application, this application has the following technical effects.

[0034] This application provides a single-image generation method and system based on a frequency-domain guided diffusion model. By establishing a composite frequency-domain constraint function and an iterative structural refinement strategy, the composite frequency-domain constraint function introduces explicit constraints on the image's frequency-domain components during the training phase. Simultaneously, the iterative structural refinement strategy applies real-time corrections during the image generation phase to suppress error accumulation. This achieves a dual-guided closed loop of composite frequency-domain constraints during training and iterative structural refinement during image generation. This not only solves the problems of macroscopic structural distortion and content collapse caused by limited receptive fields in existing diffusion models, but also significantly improves the structural integrity of the generated image. Furthermore, targeted constraints on high-frequency components of the image effectively enhance the detail clarity and visual fidelity of the generated image. Moreover, this application does not require changes to the core architecture of the baseline network, exhibiting strong compatibility and applicability to various single-image generation scenarios based on diffusion models. Additionally, the collaborative optimization between the training and image generation phases ensures the effective execution of the structural knowledge learned by the model during generation, improving the stability and reliability of image generation. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is an application environment diagram of a single image generation method based on a frequency domain-guided diffusion model provided in an embodiment of this application.

[0037] Figure 2 This is a flowchart illustrating a single-image generation method based on a frequency-domain guided diffusion model, provided as an embodiment of this application.

[0038] Figure 3 This is a flowchart illustrating the principle of a single image generation method based on a frequency domain-guided diffusion model provided in an embodiment of this application.

[0039] Figure 4 This is a schematic diagram of an iterative structural refining strategy provided in an embodiment of this application.

[0040] Figure 5 This is a schematic diagram of the original training samples provided in an embodiment of this application, wherein, Figure 5 Image (a) in the image is the original training sample image of "distant mountains". Figure 5 (b) in the image is the original training sample image of "playing golf".

[0041] Figure 6This is a schematic diagram illustrating the generation result of the method of this application according to an embodiment of the present application, wherein, Figure 6 Image (a) in this application is a "distant mountain" image generated by the method described in this application. Figure 6 (b) in the image is a "golfing" image generated by the method of this application.

[0042] Figure 7 A schematic diagram illustrating the comparison between the generation result of the method of this application and the generation result of a traditional baseline model, provided as an embodiment of this application, is shown below. Figure 7 (a), (d), and (g) in the image set are a set of training images. Figure 7 In the image, (b), (e), and (h) are a set of images generated by the traditional baseline model. Figure 7 (c), (f), and (i) in this application are a set of images generated by the method of this application.

[0043] Figure 8 This is a schematic diagram of the structure of a single image generation system based on a frequency domain-guided diffusion model, provided in an embodiment of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] Single-image generation aims to learn the inherent visual statistical patterns from a single image sample and generate a series of novel images with similar content but different details. This has always been a highly challenging research direction in computer vision. Previous research can be broadly categorized into non-parametric and parametric methods based on whether a learnable parametric model is required to generate new images. The core idea of ​​non-parametric methods is to synthesize new images by sampling and recombining image patches from the source image; for example, the texture synthesis algorithm proposed by Efros and Leung employs this paradigm. Parametric methods, on the other hand, learn the inherent data distribution of images by training a deep generative model. Depending on the model used, they can be further divided into methods based on generative adversarial networks (GANs) and methods based on denoising diffusion models. GAN-based methods, such as SinGAN proposed by Shaham et al., utilize an adversarial game between the generator and discriminator to learn the image distribution and construct a multi-scale pyramid structure to capture features at different levels. Diffusion model-based methods, such as SinDiffusion proposed by Wang et al., generate samples by learning an inverse process of progressively denoising from noise to a clear image. While the aforementioned methods have made some progress in single-image generation tasks, they all have inherent limitations: non-parametric methods, although able to preserve local textures well, have limited ability to generate images with novel global structures; GAN-based methods often face challenges of training instability and mode collapse, and their multi-scale architecture easily leads to the amplification of lower-level errors in upper layers, compromising the structural integrity of the final image. In contrast, diffusion-based methods exhibit superior generation quality and training stability. However, to avoid overfitting under single-sample conditions, existing methods typically employ small receptive field models. While this ensures the diversity of generated samples, it sacrifices the model's ability to perceive global structure, frequently leading to technical problems such as macroscopic structural distortion and collapse of key content in generated images.

[0046] Introducing frequency domain information provides a powerful tool for manipulating and improving specific image properties. Its core lies in decoupling low-frequency components representing the macroscopic structure of an image from high-frequency components representing local details. In generative adversarial networks (GANs), Durall et al. effectively bridged the domain gap and improved the model's generalization ability by analyzing and aligning the spectral distributions of images from different domains. In image super-resolution tasks, Lai et al.'s Laplacian pyramid network successfully reconstructed high-quality high-frequency information by progressively predicting and recovering residual details of images across different frequency subbands. Furthermore, in image style transfer tasks, fusing low-frequency components representing content from one image with high-frequency components representing style from another has become a classic technique and has been widely applied and validated in numerous studies following the pioneering work of Gatys et al. These works all demonstrate, from different perspectives, the enormous potential and effectiveness of using frequency domain information to guide and constrain the generation process.

[0047] Based on this, the purpose of this application is to provide a single-image generation method and system based on a frequency domain-guided diffusion model. This method is suitable for learning visual statistical patterns from a single image sample to generate novel images with similar content, different details, complete structure, and high fidelity. It primarily addresses the common problems of macroscopic structural distortion, key content collapse, and high-frequency detail blurring in existing image generation models. Frequency domain information is used to guide the generation process of the single-image diffusion model to compensate for its inherent structural perception deficiencies due to its small receptive field. Furthermore, a composite frequency domain constraint is introduced during the training phase, enabling the model to learn both global structure and local details simultaneously. Simultaneously, an iterative structural correction mechanism is designed during the image generation phase to correct the generation path in real time, systematically improving the structural integrity and detail fidelity of the generated images. By decoupling the global structural information and local detail information of the image in the frequency domain and applying dual guidance in both model training and image generation phases, the structural integrity and visual fidelity of the generated images can be significantly improved.

[0048] Specifically, firstly, during the model training phase, a composite frequency domain constraint function is designed. This function, in addition to the traditional noise prediction loss, adds explicit optimization objectives for low-frequency structural components and high-frequency detail components, allowing the model to learn and master the correct global layout and sharp local textures during training. Subsequently, in the image generation phase, an iterative structural refinement strategy is introduced. This strategy actively and in real-time corrects the generation process towards the correct structural trajectory at each step of denoising sampling to suppress the accumulation of small errors. The constraint function in the training phase and the refinement strategy in the image generation phase form a collaborative closed-loop system, mutually optimizing each other. The frequency domain prior knowledge injected in the training phase provides the correction module in the image generation phase with higher-quality initial predictions that are closer to the correct manifold. Simultaneously, the real-time correction mechanism in the image generation phase ensures that the macroscopic structural knowledge learned by the model during training is not lost due to error accumulation in the sampling chain. This method not only effectively solves the structural collapse problem but also significantly enhances the detail clarity and overall visual quality of the generated images, comprehensively improving the performance and reliability of single-image generation technology.

[0049] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] The single-image generation method based on a frequency-domain guided diffusion model provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up separately, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send a single training image to server 104. After receiving the single training image, server 104 defines a baseline network based on a diffusion model for that single training image; establishes a composite frequency domain constraint function; establishes an iterative structure refinement strategy; performs end-to-end training on the baseline network based on the composite frequency domain constraint function; after training, it performs inverse denoising sampling in conjunction with the iterative structure refinement strategy to generate a new image with a distribution similar to the training image. Server 104 can then feed back the obtained new image to terminal 102. In addition, in some embodiments, the frequency-domain guided diffusion model single image generation method can also be implemented by the server 104 or the terminal 102 separately. For example, the terminal 102 can directly perform frequency-domain guided diffusion model single image generation processing on a single training image, or the server 104 can obtain a single training image from the data storage system and perform frequency-domain guided diffusion model single image generation processing on the single training image.

[0051] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, and IoT devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers, or it can be a cloud server.

[0052] In one exemplary embodiment, such as Figure 2 As shown, a single-image generation method based on a frequency-domain guided diffusion model is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps S1 to S4.

[0053] S1: Obtain a single training image and define a baseline network based on the diffusion model.

[0054] The baseline network employs a neural network architecture for noise prediction to achieve a reverse denoising process from a noisy image to a clear image.

[0055] S2: Establish composite frequency domain constraint functions.

[0056] The composite frequency domain constraint function is an explicit constraint on the image frequency domain components added to the noise prediction loss during the training phase of the baseline network. The composite frequency domain constraint function includes constraint terms for low-frequency components of the image and constraint terms for high-frequency components of the image.

[0057] S3: Establish an iterative structural refinement strategy.

[0058] The iterative structural refinement strategy involves using the low-frequency information of the original training image to correct the image generation process in real time during each inverse denoising step of the image generation stage, in order to suppress error accumulation.

[0059] S4: The baseline network is trained end-to-end based on the composite frequency domain constraint function. After training, reverse denoising sampling is performed in conjunction with the iterative structure refinement strategy to generate a new image with a distribution similar to the training image.

[0060] By implementing steps S1 to S4, a dual-guided closed loop of composite frequency domain constraints during training and iterative structural refinement during image generation not only solves the problems of macroscopic structural distortion and content collapse caused by the limited receptive field in existing diffusion models, but also significantly improves the structural integrity of the generated images. Targeted constraints on high-frequency components of the image also effectively enhance the detail clarity and visual fidelity of the generated images. Furthermore, it does not require changes to the core architecture of the baseline network, exhibiting strong compatibility and applicability to various single-image generation scenarios based on diffusion models. In addition, the collaborative optimization between the training and image generation phases ensures that the structural knowledge learned by the model is effectively executed during generation, improving the stability and reliability of image generation.

[0061] As an optional implementation, in step S1, the reference network is a noise prediction network based on the U-Net architecture; the training process of the reference network includes a fixed forward noise addition process and a reverse noise reduction process that needs to be learned; in the forward noise addition process, Gaussian noise is gradually added to the training image within multiple discrete time steps; in the reverse noise reduction process, the reference network receives the noise-added image and the time step as input and predicts the added noise.

[0062] As an optional implementation, in step S2, the constraint terms for the low-frequency components of the image include low-frequency spatial domain structure loss and low-frequency frequency domain amplitude loss. The low-frequency spatial domain structure loss is achieved by imposing constraints on the low-frequency portion of the predicted image in the spatial domain. This is accomplished by using a two-dimensional fast Fourier transform to separate the spectral low-frequency components, and calculating the loss using a preset low-pass filter mask and the L1 norm. The low-frequency frequency domain amplitude loss is achieved by imposing constraints on the Fourier amplitude spectrum of the low-frequency components, used to monitor the energy distribution and contrast of the low-frequency components.

[0063] As an optional implementation, the formula for calculating the low-frequency spatial domain structure loss is as follows.

[0064] .

[0065] in, This indicates the low-frequency spatial domain structure loss; This represents a two-dimensional Fast Fourier Transform; This indicates the prediction of the training image; Represents real training images; This indicates element-wise multiplication; The preset ideal low-pass filter mask; This represents the L1 norm.

[0066] As an optional implementation, the formula for calculating the low-frequency amplitude loss is as follows.

[0067] .

[0068] in, This indicates the amplitude loss in the low-frequency domain; Represents the Fourier amplitude spectrum of the image; This indicates the prediction of the training image; Represents real training images; These are the frequency coordinates after the Fourier transform of the image; This represents the mathematical expectation of the frequency coordinates; This is the preset ideal low-pass filter mask.

[0069] As an optional implementation, in step S2, the constraint term for the high-frequency components of the image is a high-frequency domain logarithmic amplitude loss. This high-frequency domain logarithmic amplitude loss uses logarithmic loss to penalize the relative differences in high-frequency amplitudes to encourage the generation of sharp details.

[0070] As an optional implementation, the formula for calculating the high-frequency domain logarithmic amplitude loss is as follows.

[0071] .

[0072] in, This represents the logarithmic amplitude loss in the high-frequency domain; ε is a high-pass mask; ε is a small constant used to ensure numerical stability. These are the frequency coordinates after the Fourier transform of the image; Represents the Fourier amplitude spectrum of the image; This indicates the prediction of the training image; Represents real training images; This represents the mathematical expectation of the frequency coordinates.

[0073] As an optional implementation, in step S3, the real-time correction process of the iterative structure refinement strategy includes: calculating the difference between the predicted training image and the real training image in the low-frequency amplitude spectrum in the current denoising step, and using the difference as the structure correction gradient to update the predicted training image. The updated image is then used for calculation in subsequent denoising steps.

[0074] As an optional implementation, in step S4, during the end-to-end training, the total loss function is composed of the composite frequency domain loss corresponding to the composite frequency domain constraint function and the pixel-level noise prediction loss weighted together, and an optimizer is used to iteratively train the baseline network. The pixel-level noise prediction loss is either the L1 loss or the L2 loss between the predicted noise and the actual noise.

[0075] To make the technical solution of this application clearer, the specific implementation process of the technical solution of this application will be explained in detail below with examples.

[0076] This application proposes a single-image generation method and system based on a frequency-domain guided diffusion model. The method employs a dual-guided paradigm: applying prior constraints during model training and performing real-time correction during image generation. First, a composite frequency-domain constraint function is introduced during training. By decoupling the high- and low-frequency components of the image and applying optimization objectives separately, the model learns the correct global structural prior during parameter learning. Then, during image generation, an iterative structural refinement strategy is used, utilizing low-frequency information from a reference image as structural anchors to actively correct the generation direction and suppress error accumulation. Finally, through this dual-guided mechanism based on the composite frequency-domain constraint function and the iterative structural refinement strategy, a novel image with high structural integrity and detail fidelity is synthesized.

[0077] This application uses the SinDiffusion model as the baseline network, assuming the input single image is... ,in and These represent the height and width of the image, respectively. The principle and flow of the method in this application are as follows: Figure 3 As shown, the specific steps include:

[0078] S1. Input the image and define the baseline model.

[0079] The SinDiffusion model consists of a training process and a sampling process. The training process aims to teach the neural network to predict added noise from a noisy image. This process includes a fixed forward noise addition step and a backward noise reduction step that needs to be learned. In the forward noise addition step, the input single training image is randomly cropped, and the system... Gaussian noise is gradually added to the model within discrete time steps. In the reverse denoising process, the model trains a U-Net architecture noise prediction network. Its task is to receive noisy images. and time step As input, and predict the added noise. During the sampling phase, the model starts from an image with pure Gaussian noise. Initially, a pre-trained noise prediction network was used. Noise is removed gradually and iteratively, starting from the time step. proceed in reverse Finally, an image is generated that is identical to the training image. New images with similar distributions.

[0080] This application first uses a single training image As a goal, a noise prediction network based on the U-Net architecture is adopted. Given that single-image generation tasks are highly susceptible to overfitting on a single training sample, SinDiffusion employs a U-Net network with a relatively small receptive field to prevent overfitting. However, this limitation on the receptive field directly results in the model's inability to effectively capture global dependencies and macroscopic structural information when processing images. The frequency domain guidance mechanism subsequently introduced in this application aims to address this inherent structural perception deficiency in SinDiffusion due to its limited receptive field without altering the baseline network structure.

[0081] S2. Establish composite frequency domain constraint functions.

[0082] To enable the model to perceive and learn the global structure and local details of an image during the training phase, this application designs a composite frequency domain constraint function. In addition to the traditional noise prediction loss, this embodiment introduces explicit constraints on the image in the frequency domain space. Specifically, at each step of training, besides calculating the model's prediction noise... With real noise Pixel-level noise prediction loss In addition, it also uses the predicted training images Compared with real training images Calculate composite frequency domain loss This composite frequency domain constraint function is applied to the model at any time step. Predicted training images Above. The composite frequency domain loss corresponding to this composite frequency domain constraint function. It consists of the following three parts.

[0083] (1) Low-frequency spatial domain structure loss: In order to ensure the correctness of the macroscopic structure, this application constrains the low-frequency part of the predicted image in the spatial domain. This process can be expressed as the following formula.

[0084] .

[0085] in, This indicates the low-frequency spatial domain structure loss; This represents a two-dimensional Fast Fourier Transform; This indicates the prediction of the training image; Represents real training images; This indicates element-wise multiplication; This is a preset ideal low-pass filter mask used to separate low frequencies in the spectrum; This represents the L1 norm.

[0086] (2) Low-frequency amplitude loss: In order to monitor the energy distribution and contrast of low-frequency components, this application directly constrains the low-frequency amplitude spectrum in the frequency domain. This process can be expressed as the following formula.

[0087] .

[0088] in, This indicates the amplitude loss in the low-frequency domain; Represents the Fourier amplitude spectrum of the image; This indicates the prediction of the training image; Represents real training images; These are the frequency coordinates after the Fourier transform of the image; This represents the mathematical expectation of the frequency coordinates; This is the preset ideal low-pass filter mask.

[0089] (3) High-frequency domain logarithmic amplitude loss: To counteract the blurring effect caused by mean square error loss and encourage the generation of clear details, this application uses logarithmic loss to more effectively penalize the relative differences in high-frequency amplitudes. This process can be expressed as follows.

[0090] .

[0091] in, This represents the logarithmic amplitude loss in the high-frequency domain; ε is a high-pass mask; ε is a small constant used to ensure numerical stability. These are the frequency coordinates after the Fourier transform of the image; Represents the Fourier amplitude spectrum of the image; This indicates the prediction of the training image; Represents real training images; This represents the mathematical expectation of the frequency coordinates.

[0092] S3. Establish an iterative structural refinement strategy.

[0093] To combat structural drift caused by error accumulation during the sampling phase, this application designs an iterative structural refinement strategy. This iterative structural refinement strategy utilizes the original training images at each step of the inverse denoising process. Low-frequency information is used to correct the generation process in real time. In any denoising step... Given the current noisy image The standard sampler will predict the denoised image. And calculate accordingly This iterative structural refining strategy adds a correction step: calculating... Compared with real training images The difference in the low-frequency amplitude spectrum, and using this difference as a gradient to guide... The update process can be expressed as follows.

[0094] .

[0095] .

[0096] in, For structural correction gradient, The intensity coefficient is used as a guide. The updated denoised image. It was then used to calculate the final result. This strategy, at each step, pulls the generated direction back onto the correct macroscopic structural track, specifically as follows: Figure 4 As shown. Since this correction only applies to low-frequency components, it can maintain structural consistency while preserving the randomness and diversity of high-frequency details to the greatest extent.

[0097] S4. Training and Testing.

[0098] The overall process of this application is as follows: First, during the training phase, the U-Net network is trained end-to-end using a composite frequency domain constraint function; then, during the testing phase, a standard denoising sampler is used in conjunction with an iterative structure refinement strategy to generate new images.

[0099] During training, the total loss function Loss prediction based on pixel-level noise and composite frequency domain loss The weighted composition is expressed as follows.

[0100] .

[0101] in, This is the total loss function; For pixel-level noise prediction loss, it is represented by L1 loss or L2 loss between predicted noise and real noise; For composite frequency domain loss; Weighting coefficients to balance the two types of losses.

[0102] This application uses the AdamW optimizer to train the model for 60K iterations with an initial learning rate of 0.0005 and weight coefficients... Set it to 0.05. Perform the following operation in each iteration.

[0103] (1) Input the original training image After random cropping, another time step is randomly sampled. and noise Generate a noisy image .

[0104] (2) and Input noise prediction network The predicted noise is obtained. .

[0105] (3) Calculate pixel-level noise prediction loss .

[0106] (4) According to and Calculate the prediction training image .

[0107] (5) Calculate the composite frequency domain loss .

[0108] (6) Calculate the total loss And then perform backpropagation to update the network parameters.

[0109] The optimal model weights are saved after training. During image generation, a noisy image containing pure Gaussian noise is used. Begin, proceed Step-by-step reverse noise reduction. At each step... The following operations will be performed.

[0110] (1) Input the current noisy image The predicted noise is obtained through the trained optimal model. .

[0111] (2) According to Calculate the prediction training image .

[0112] (3) Calculate the structural correction gradient And use it to update the denoised image. The updated denoised image is obtained. .

[0113] (4) Using the standard sampler and the updated denoised image Calculated .

[0114] (5) Repeat steps (1) to (4) until The final generated image is obtained. .

[0115] This application proposes a single-image generation method and system based on a frequency-domain guided diffusion model, which focuses on solving the structural distortion problem caused by the limited receptive field in existing single-image diffusion generation models. The method first uses a composite frequency-domain constraint function during the training phase to allow the model to learn prior knowledge of the image's structure and details. Then, during the image generation phase, an iterative structure refinement strategy is employed to use low-frequency information from the original training images to correct the generation direction in real time, ultimately synthesizing a high-quality, high-fidelity new image.

[0116] In this embodiment, it is assumed that the original training image used is as follows: Figure 5 As shown, where, Figure 5 Image (a) in the image is the original training sample image of "distant mountains". Figure 5 Image (b) in the image is the original training sample image of "playing golf". Images generated by the method of this application are as follows: Figure 6 As shown, where, Figure 6 Image (a) in this application is a "distant mountain" image generated by the method described in this application. Figure 6 (b) in the image is a "golfing" image generated by the method of this application.

[0117] Using the images generated through the above steps, this embodiment compares them with the traditional baseline model SinDiffusion (trained using only pixel loss), as follows: Figure 7 As shown. Among them, Figure 7 (a), (d), and (g) in the image set are a set of training images. Figure 7 In the image, (b), (e), and (h) are a set of images generated by the traditional baseline model SinDiffusion. Figure 7In the table, (c), (f), and (i) represent a set of images generated by the method of this application. To quantitatively evaluate the performance of the method, this application selected indicators such as SIFID (Single-Image Frechet Inception Distance), NIQE (Natural Image Quality Evaluator), MUSIQ (Multi-Scale Image Quality), and NIMA (Neural Image Assessment) for evaluation. The lower the SIFID, the closer the generated image is to the original image distribution. The lower the NIQE, the more natural the image; the higher the MUSIQ, the higher the image perceptual quality; and the higher the NIMA, the higher the image aesthetic score. The results are shown in Table 1. The results show that the method proposed in this application is superior to the comparative methods in all indicators, especially in the distribution structure similarity SIFID. The SIFID value of the traditional baseline model SinDiffusion is 4.05, while the SIFID value of the method in this application is 1.14. Since the lower the SIFID, the closer the generated image is to the original image distribution, the image generated by the method in this application is closer to the original image.

[0118] Table 1 Quantitative Results Generated from a Single Image

[0119] Based on the same inventive concept, this application also provides a frequency-domain guided diffusion model single image generation system for implementing the frequency-domain guided diffusion model single image generation method described above. The solution provided by this frequency-domain guided diffusion model single image generation system is similar to the implementation scheme described in the above method. Therefore, the specific limitations in the embodiments of the frequency-domain guided diffusion model single image generation system provided below can be found in the limitations of the frequency-domain guided diffusion model single image generation method described above, and will not be repeated here.

[0120] In one exemplary embodiment, such as Figure 8 As shown, a single image generation system based on a frequency domain-guided diffusion model is provided, comprising the following modules.

[0121] A baseline network module is used to acquire a single training image and define a baseline network based on a diffusion model; the baseline network adopts a neural network architecture for noise prediction to achieve a reverse denoising process from a noisy image to a clear image.

[0122] A composite frequency domain constraint module is used to establish a composite frequency domain constraint function. The composite frequency domain constraint function is an explicit constraint on the image frequency domain components added on top of the noise prediction loss during the training phase of the baseline network. The composite frequency domain constraint function includes constraint terms for low-frequency components of the image and constraint terms for high-frequency components of the image.

[0123] The structure refinement module is used to establish an iterative structure refinement strategy. The iterative structure refinement strategy is to use the low-frequency information of the original training image to correct the image generation process in real time in each inverse denoising step of the image generation stage, so as to suppress error accumulation.

[0124] The training and generation module is used to perform end-to-end training of the baseline network based on the composite frequency domain constraint function. After training, it performs inverse denoising sampling in combination with the iterative structure refinement strategy to generate a new image with a distribution similar to the training image.

[0125] Existing single-image diffusion generation models face structural distortion and content collapse issues due to limited receptive fields. To address this, this application proposes a frequency-domain guided diffusion model-based single-image generation method and system. First, a composite frequency-domain constraint function is introduced during the model training phase. Using pre-defined low-pass and high-pass masks, different targeted optimization objectives are applied to the low-frequency components representing image structure and the high-frequency components representing microscopic details. Specifically, spatial domain loss and frequency domain amplitude loss are applied simultaneously to the low-frequency components for dual supervision; logarithmic amplitude loss is applied to the high-frequency components to encourage detail generation. This approach directly injects prior knowledge of global layout and local texture into the gradient update of model parameters.

[0126] Then, an iterative structural refinement strategy is employed during the image generation stage. At each step of the denoising process, this iterative structural refinement strategy calculates the deviation between the currently predicted sharp image and the reference image in the low-frequency amplitude spectrum, and generates a correction gradient accordingly to fine-tune the generation direction in real time, actively suppressing error accumulation along the generation path. Since this correction is specifically targeted at low frequencies, it can effectively preserve the randomness of high-frequency details.

[0127] Furthermore, this application organically combines frequency domain constraints during the training phase with structural refinement during the image generation phase, achieving synergistic optimization and mutual enhancement between the two phases through a dual-guided approach. The prior knowledge injected during training enables the model to make more accurate initial predictions during generation, providing high-quality input for subsequent structural refinement and making its correction process more efficient and stable. Conversely, the real-time correction mechanism during image generation ensures that the macroscopic structural knowledge learned by the model during training is not lost due to error accumulation in the random sampling chain, thus guaranteeing the effective execution of this knowledge. Compared to traditional baseline methods, this application addresses the limited receptive field problem of existing models by applying prior constraints during training and gradient guidance during sampling, significantly improving the structural integrity and detail clarity of the generated images.

[0128] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A single-image generation method based on a frequency-domain guided diffusion model, characterized in that, The frequency-domain guided diffusion model-based single-image generation method includes: A single training image is acquired, and a baseline network based on a diffusion model is defined; the baseline network employs a neural network architecture for noise prediction to achieve a reverse denoising process from a noisy image to a clear image. A composite frequency domain constraint function is established; the composite frequency domain constraint function is an explicit constraint on the frequency domain components of the image added on the basis of the noise prediction loss during the training phase of the benchmark network; the composite frequency domain constraint function includes constraint terms for low-frequency components of the image and constraint terms for high-frequency components of the image. An iterative structural refinement strategy is established; the iterative structural refinement strategy is to use the low-frequency information of the original training image to correct the image generation process in real time in each inverse denoising step of the image generation stage, so as to suppress error accumulation. The baseline network is trained end-to-end based on the composite frequency domain constraint function. After training, reverse denoising sampling is performed in conjunction with the iterative structure refinement strategy to generate a new image with a distribution similar to the training image.

2. The single-image generation method based on a frequency-domain guided diffusion model according to claim 1, characterized in that, The baseline network is a noise prediction network based on the U-Net architecture; the training process of the baseline network includes a fixed forward noise addition process and a backward noise reduction process that needs to be learned; in the forward noise addition process, Gaussian noise is gradually added to the training image at multiple discrete time steps; In the reverse denoising process, the baseline network receives the noisy image and time step as input and predicts the added noise.

3. The single-image generation method based on a frequency-domain guided diffusion model according to claim 1, characterized in that, The constraints on the low-frequency components of the image include low-frequency spatial domain structure loss and low-frequency frequency domain amplitude loss. The low-frequency spatial domain structure loss is achieved by imposing constraints on the low-frequency part of the predicted image in the spatial domain. Two-dimensional fast Fourier transform is used to separate the low-frequency components of the spectrum, and the loss is calculated by combining a preset low-pass filter mask and the L1 norm. The low-frequency amplitude loss is achieved by imposing constraints on the Fourier amplitude spectrum of the low-frequency components, which is used to monitor the energy distribution and contrast of the low-frequency components.

4. The single-image generation method based on a frequency-domain guided diffusion model according to claim 3, characterized in that, The formula for calculating the low-frequency spatial domain structure loss is as follows: ； in, This indicates the low-frequency spatial domain structure loss; Represents a two-dimensional Fast Fourier Transform; This represents the prediction of the training image; Represents real training images; This indicates element-wise multiplication; The preset ideal low-pass filter mask; This represents the L1 norm.

5. The single-image generation method based on a frequency-domain guided diffusion model according to claim 3, characterized in that, The formula for calculating the low-frequency amplitude loss is as follows: ； in, This indicates the amplitude loss in the low-frequency domain; Represents the Fourier amplitude spectrum of the image; This represents the prediction of the training image; Represents real training images; These are the frequency coordinates after the Fourier transform of the image; This represents the mathematical expectation of the frequency coordinates; This is the preset ideal low-pass filter mask.

6. The single-image generation method based on a frequency-domain guided diffusion model according to claim 1, characterized in that, The constraint term for the high-frequency components of the image is the high-frequency domain logarithmic amplitude loss; The high-frequency domain logarithmic amplitude loss employs logarithmic loss to penalize the relative differences in high-frequency amplitudes in order to encourage the generation of clear details.

7. The single-image generation method based on a frequency-domain guided diffusion model according to claim 6, characterized in that, The formula for calculating the high-frequency domain logarithmic amplitude loss is as follows: ； in, This represents the logarithmic amplitude loss in the high-frequency domain; ε is a high-pass mask; ε is a small constant used to ensure numerical stability. These are the frequency coordinates after the Fourier transform of the image; Represents the Fourier amplitude spectrum of the image; This represents the prediction of the training image; Represents real training images; This represents the mathematical expectation of the frequency coordinates.

8. The single-image generation method based on a frequency-domain guided diffusion model according to claim 1, characterized in that, The real-time correction process of the iterative structure refinement strategy includes: calculating the difference between the predicted training image and the real training image in the low-frequency amplitude spectrum in the current denoising step, and using the difference as the structure correction gradient to update the predicted training image. The updated image is then used for calculation in subsequent denoising steps.

9. The single-image generation method based on a frequency-domain guided diffusion model according to claim 1, characterized in that, In the end-to-end training, the total loss function is composed of the composite frequency domain loss corresponding to the composite frequency domain constraint function and the pixel-level noise prediction loss weighted together. An optimizer is used to iteratively train the baseline network. The pixel-level noise prediction loss is either the L1 loss or the L2 loss between the predicted noise and the real noise.

10. A single-image generation system based on a frequency-domain guided diffusion model, characterized in that, The frequency-domain guided diffusion model single image generation system is used to implement the frequency-domain guided diffusion model single image generation method according to any one of claims 1-9, wherein the frequency-domain guided diffusion model single image generation system comprises: A baseline network module is used to acquire a single training image and define a baseline network based on a diffusion model; the baseline network adopts a neural network architecture for noise prediction to achieve a reverse denoising process from a noisy image to a clear image; A composite frequency domain constraint module is used to establish a composite frequency domain constraint function. The composite frequency domain constraint function is an explicit constraint on the image frequency domain components added to the noise prediction loss during the training phase of the baseline network. The composite frequency domain constraint function includes constraint terms for low-frequency components of the image and constraint terms for high-frequency components of the image. The structure refinement module is used to establish an iterative structure refinement strategy. The iterative structure refinement strategy is to use the low-frequency information of the original training image to correct the image generation process in real time in each inverse denoising step of the image generation stage in order to suppress error accumulation. The training and generation module is used to perform end-to-end training of the baseline network based on the composite frequency domain constraint function. After training, it performs inverse denoising sampling in combination with the iterative structure refinement strategy to generate a new image with a distribution similar to the training image.

Images