Neural Network Image Generation Method Based on Diffusion Model and Fast Fourier Transform

Abstract

This invention provides a neural network image generation method and system based on a diffusion model and a fast Fourier transform, relating to the field of artificial intelligence image generation. The method includes acquiring noisy image data and conditional input information; performing a fast Fourier transform to obtain frequency domain feature data; decomposing it into amplitude and phase spectrum components and denoising it using a diffusion model; generating frequency modulation parameters based on the conditional input and performing modulation processing; performing an inverse transform to obtain denoised image data; and iterating until a termination condition is met to output the generated image. This invention combines frequency domain processing with a diffusion model, improving image generation quality and control accuracy.

Description

Technical Field

[0001] This invention relates to artificial intelligence image generation technology, and more particularly to a neural network image generation method and system based on a diffusion model and fast Fourier transform. Background Technology

[0002] Image generation technology is an important research direction in the field of artificial intelligence. In recent years, diffusion models, as a novel deep generation model, have achieved remarkable results in image generation. Diffusion models generate high-quality images by gradually adding noise to the original data and learning the inverse process. Its basic principle is to simulate the diffusion phenomenon in physics, transforming the data into noise through a forward process and then learning the reverse diffusion process to recover the original data.

[0003] Traditional image generation methods primarily rely on spatial domain processing, directly manipulating pixel data. Frequency domain analysis, however, a crucial tool in signal processing, uses Fourier transform to convert images from the spatial domain to the frequency domain, revealing frequency features that are difficult to observe in the spatial domain. The Fast Fourier Transform (FFT), as an efficient algorithm for calculating Fourier transforms, has been widely applied in image processing, signal analysis, and other fields.

[0004] Traditional diffusion models primarily operate in the spatial domain, making it difficult to effectively capture and process the global frequency features of an image. This results in inconsistencies in texture details and global structure in the generated images. Existing spatial domain-based diffusion models suffer from high computational complexity and lengthy generation processes when processing high-resolution images, making them unsuitable for real-time applications. Furthermore, existing image generation methods rely on limited methods for utilizing conditional inputs, restricting the diversity and controllability of the generated results and failing to flexibly adjust the frequency features of the generated image according to specific needs. Summary of the Invention

[0005] This invention provides a neural network image generation method based on a diffusion model and fast Fourier transform, which can solve the problems in the prior art.

[0006] A first aspect of this invention provides a neural network image generation method based on a diffusion model and a fast Fourier transform, comprising: Acquire noisy image data and corresponding conditional input information, perform fast Fourier transform on the noisy image data, and convert the noisy image data from spatial domain representation to frequency domain to obtain frequency domain feature data; The frequency domain feature data is decomposed into amplitude spectrum components and phase spectrum components. The amplitude spectrum components and phase spectrum components are denoised and predicted by a diffusion model to generate denoised amplitude spectrum components and denoised phase spectrum components. The denoised amplitude spectrum components and denoised phase spectrum components are then combined to form denoised frequency domain feature data. Frequency modulation parameters are generated based on the input information. The denoised frequency domain feature data is then modulated based on the frequency modulation parameters to obtain the modulated frequency domain feature data. Perform an inverse fast Fourier transform on the modulated frequency domain feature data to convert the modulated frequency domain feature data from frequency domain representation to spatial domain representation, and obtain denoised image data; The denoised image data is used as new noisy image data, and the process is repeated iteratively until the time step termination condition of the diffusion model is met, and the generated image data is output.

[0007] Acquire noisy image data and corresponding conditional input information, perform a Fast Fourier Transform on the noisy image data to convert the noisy image data from spatial domain representation to frequency domain to obtain frequency domain feature data, including: The noisy image data is divided into multiple image blocks according to a preset size. The multiple image blocks are sampled according to the sampling resolution to obtain sampled noisy image data. The sampled noisy image data is then normalized to obtain normalized noisy image data. Feature vectors are extracted from the conditional input information to construct orthogonal basis vectors, and the orthogonal basis vectors are combined according to spatial structure rules to generate a frequency domain transformation matrix; The normalized noisy image data is transformed by the frequency domain transformation matrix to obtain the initial frequency domain feature data, and the frequency domain amplitude feature and frequency domain phase feature are separated from the initial frequency domain feature data. The spatial structure rules based on orthogonal basis vectors combine frequency domain amplitude features and frequency domain phase features into frequency domain feature data.

[0008] The frequency domain feature data is decomposed into amplitude spectrum components and phase spectrum components. A diffusion model is used to perform denoising prediction on the amplitude spectrum components and phase spectrum components, generating denoised amplitude spectrum components and denoised phase spectrum components. The denoised amplitude spectrum components and denoised phase spectrum components are then combined to form denoised frequency domain feature data, including: Frequency domain decomposition is performed on the frequency domain feature data to extract the amplitude information to obtain the amplitude spectrum component, and the angle information is extracted to obtain the phase spectrum component. Construct a feature mapping space, map the amplitude spectrum components to the feature mapping space to obtain the amplitude mapping features that characterize the amplitude distribution, and map the phase spectrum components to the feature mapping space to obtain the phase mapping features that characterize the angle distribution; The feature representation is obtained by combining the amplitude mapping feature and the phase mapping feature according to the frequency domain distribution relationship; A diffusion model containing a feature extraction layer and a noise prediction layer is constructed. The feature representation is input into the feature extraction layer to extract noise features, and the noise features are input into the noise prediction layer to generate noise distribution prediction. Based on noise distribution prediction, denoising prediction is performed on the amplitude spectrum component to generate a denoised amplitude spectrum component, and denoising prediction is performed on the phase spectrum component to generate a denoised phase spectrum component. The frequency domain distribution mapping relationship is extracted from the frequency domain feature data. Based on the frequency domain distribution mapping relationship, the amplitude distribution information in the denoised amplitude spectrum component and the angle distribution information in the denoised phase spectrum component are combined to generate denoised frequency domain feature data containing complete frequency domain features.

[0009] Frequency modulation parameters are generated based on the input information. The denoised frequency domain feature data is then modulated using these parameters to obtain the modulated frequency domain feature data, which includes: Semantic and spatial features are extracted from the conditional input information, a feature mapping matrix is ​​constructed, and the semantic features are converted into semantic modulation parameters and the spatial features are converted into spatial modulation parameters through the feature mapping matrix. Frequency modulation parameters are generated based on the semantic and spatial modulation parameters, and the frequency modulation parameters are used to guide frequency domain feature modulation. Decomposition operations are performed on the denoised frequency domain feature data to extract amplitude information to obtain amplitude components, and phase information is extracted from the denoised frequency domain feature data to obtain phase components; The amplitude component and the phase component are modulated separately based on the frequency modulation parameters to generate modulated amplitude components and phase components. The modulated amplitude component and phase component are combined to generate modulated frequency domain feature data.

[0010] Performing an inverse fast Fourier transform on the modulated frequency domain feature data converts the frequency domain representation to a spatial domain representation, resulting in denoised image data including: Construct an inverse transform matrix to separate the amplitude component and the phase component from the modulated frequency domain feature data. Map the amplitude component to the inverse transform matrix to generate amplitude transform parameters, and map the phase component to the inverse transform matrix to generate phase transform parameters. Spatial domain transformation parameters are generated based on amplitude transformation parameters and phase transformation parameters; The modulated frequency domain feature data is decomposed into multi-scale frequency domain components. Based on the spatial domain transformation parameters, the multi-scale frequency domain components are inversely transformed to generate multi-scale spatial domain components. The multi-scale spatial domain components are reconstructed to obtain the initial spatial domain feature data; Compensation information is extracted from the modulated frequency domain feature data and converted into spatial domain compensation data. The spatial domain compensation data is combined with the initial spatial domain feature data to generate complete spatial domain feature data, which is then output as denoised image data.

[0011] The denoised image data is used as new noisy image data, and the process is repeated iteratively until the time step termination condition of the diffusion model is met. The output generated image data includes: Construct a feature discrimination matrix, extract local and global features from the denoised image data, map the local features to the feature discrimination matrix to calculate local difference parameters, map the global features to the feature discrimination matrix to calculate global difference parameters, and generate time step judgment parameters based on the local and global difference parameters. The denoised image data is input into the diffusion process to perform iterative calculations. The feature change data generated in each iteration is recorded. The feature change data is then converted into structural change data and texture change data through a decomposition matrix. Iterative optimization parameters are calculated based on the structural change data and texture change data. These iterative optimization parameters are used to guide the next iteration. The iterative optimization parameters and time step judgment parameters are combined and analyzed to generate termination judgment data. Based on the termination judgment data, it is determined whether the preset time step termination condition is met. Feature processing is performed on the denoised image data that meets the termination condition to extract structural and texture information, and the structural and texture information is fused to generate the final generated image data.

[0012] The denoised image data is input into the diffusion process for iterative calculation. The feature change data generated in each iteration is recorded. The feature change data is then transformed into structural change data and texture change data through a decomposition matrix, including: The denoised image data is input into the diffusion process. During the diffusion process, a forward diffusion operation is performed on the denoised image data to obtain the diffusion state features. A reverse denoising operation is performed on the diffusion state features to obtain the first round of iterative denoising results. Extract the initial feature representation of the denoised image data and the output feature representation of the first round of denoising results, calculate the feature difference value between the initial feature representation and the output feature representation, and record the feature difference value as the feature change data generated in the first round of iteration; The first round of denoising results is used as new input data to continue the diffusion process to perform iterative calculations. The forward diffusion operation and the reverse denoising operation are repeatedly executed. The feature representations of the input data and the output data of each round of iteration are extracted. The feature difference value of each round of iteration is calculated, and the feature change data generated in each round of iteration is recorded. Construct a decomposition matrix, perform linear transformation on the feature change data through the decomposition matrix to obtain the decomposed feature representation, perform subspace projection on the decomposed feature representation to extract the edge response subspace projection and detail response subspace projection respectively, and fuse the edge response subspace projection and detail response subspace projection to generate texture change data; Subspace projection is performed on the decomposed feature representation to extract the region connectivity subspace projection and the morphological distribution subspace projection respectively. The region connectivity subspace projection and the morphological distribution subspace projection are then fused to generate structural change data.

[0013] A second aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0014] A third aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0015] In this embodiment, by converting noisy image data to the frequency domain for processing, the characteristics of images in the frequency domain are fully utilized. This allows the diffusion model to more effectively capture and process the global structural features of the image. The frequency domain feature data is decomposed into amplitude and phase spectrum components, which are then used for denoising prediction separately. This achieves refined processing of different frequency components of the image, better preserving image details and texture information. Introducing frequency modulation parameters based on conditional input information to modulate the denoised frequency domain features enhances the controllability of the generation process, enabling the model to flexibly adjust the frequency characteristics of the image according to input conditions, improving the diversity and customization capabilities of the generated images. By organically combining the diffusion model with Fourier transform, an iterative process of image denoising and generation is realized in the frequency domain. Compared to traditional diffusion models that only process in the spatial domain, this method is more computationally efficient and can better handle multi-scale features of the image. It effectively avoids the problems of inconsistent local details and global structural inconsistencies that may occur in traditional diffusion models, resulting in images with better integrity and consistency. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the neural network image generation method based on the diffusion model and fast Fourier transform according to an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0019] Figure 1 This is a flowchart illustrating the neural network image generation method based on the diffusion model and fast Fourier transform according to an embodiment of the present invention. Figure 1 As shown, the method includes: Acquire noisy image data and corresponding conditional input information, perform fast Fourier transform on the noisy image data, and convert the noisy image data from spatial domain representation to frequency domain to obtain frequency domain feature data; The frequency domain feature data is decomposed into amplitude spectrum components and phase spectrum components. The amplitude spectrum components and phase spectrum components are denoised and predicted by a diffusion model to generate denoised amplitude spectrum components and denoised phase spectrum components. The denoised amplitude spectrum components and denoised phase spectrum components are then combined to form denoised frequency domain feature data. Frequency modulation parameters are generated based on the input information. The denoised frequency domain feature data is then modulated based on the frequency modulation parameters to obtain the modulated frequency domain feature data. Perform an inverse fast Fourier transform on the modulated frequency domain feature data to convert the modulated frequency domain feature data from frequency domain representation to spatial domain representation, and obtain denoised image data; The denoised image data is used as new noisy image data, and the process is repeated iteratively until the time step termination condition of the diffusion model is met, and the generated image data is output.

[0020] In one optional implementation, acquiring noisy image data and corresponding conditional input information, performing a Fast Fourier Transform on the noisy image data, and converting the noisy image data from spatial domain representation to frequency domain to obtain frequency domain feature data includes: The noisy image data is divided into multiple image blocks according to a preset size. The multiple image blocks are sampled according to the sampling resolution to obtain sampled noisy image data. The sampled noisy image data is then normalized to obtain normalized noisy image data. Feature vectors are extracted from the conditional input information to construct orthogonal basis vectors, and the orthogonal basis vectors are combined according to spatial structure rules to generate a frequency domain transformation matrix; The normalized noisy image data is transformed by the frequency domain transformation matrix to obtain the initial frequency domain feature data, and the frequency domain amplitude feature and frequency domain phase feature are separated from the initial frequency domain feature data. The spatial structure rules based on orthogonal basis vectors combine frequency domain amplitude features and frequency domain phase features into frequency domain feature data.

[0021] In the field of digital image processing, acquiring and processing noisy image data is crucial for image reconstruction and enhancement. This embodiment provides a noisy image processing method based on frequency domain transformation, which extracts effective frequency domain feature data by converting noisy image data from spatial domain representation to frequency domain representation.

[0022] First, acquire the noisy image data and the corresponding conditional input information. The noisy image data can be a raw image containing noise acquired through an image acquisition device, or it can be an artificially synthesized noisy image. The conditional input information includes descriptive information related to the image, such as image size, sampling parameters, and noise type.

[0023] The noisy image data is divided into multiple image blocks of a preset size. The preset size can be a fixed pixel size, for example, dividing a noisy image with a resolution of 1024×1024 into multiple image blocks of 128×128. The image blocks can be divided in an overlapping or non-overlapping manner. Overlapping division can reduce boundary effects and improve processing results.

[0024] Multiple image patches are sampled according to a sampling resolution to obtain sampled noisy image data. The sampling resolution can be set according to application requirements; for example, a higher sampling resolution can be used for areas with more high-frequency noise. Interpolation algorithms, such as bilinear interpolation or cubic spline interpolation, can be used during the sampling process to ensure sampling quality.

[0025] Normalized noise image data is obtained by normalizing the sampled noisy image data. Normalization maps image pixel values ​​to a specific range, such as [-1, 1] or [0, 1], which helps stabilize subsequent processing. Normalization can be achieved using methods such as maximum-minimum normalization or Z-score standardization.

[0026] Feature vectors are extracted from the conditional input information to construct orthogonal basis vectors. Feature vector extraction can be based on image characteristic parameters in the conditional input information, such as texture features and edge information. Through methods such as principal component analysis or singular value decomposition, a set of mutually orthogonal basis vectors is constructed from the feature vectors. These basis vectors can effectively represent the main features of the image.

[0027] The frequency domain transformation matrix is ​​generated by combining orthogonal basis vectors according to spatial structure rules. These rules define the arrangement of the basis vectors in the frequency domain, which can be based on frequency or energy. The generation of the frequency domain transformation matrix takes spatial location information into account, ensuring that the transformed frequency domain representation preserves the spatial structure relationships.

[0028] Initial frequency domain feature data is obtained by transforming normalized noisy image data using a frequency domain transformation matrix. The transformation process involves operating on the normalized noisy image data with the frequency domain transformation matrix, mapping the spatial domain image data to the frequency domain. This transformation is similar to the Discrete Fourier Transform, but it employs adaptive basis functions constructed based on conditional information.

[0029] Frequency domain amplitude features and frequency domain phase features are separated from the initial frequency domain feature data. The frequency domain amplitude features represent the intensity of each frequency component, and the frequency domain phase features represent the phase information of each frequency component. The separation process includes calculating the magnitude (amplitude) and phase angle (phase) of the frequency domain data in complex number representation.

[0030] Based on the spatial structure rules of orthogonal basis vectors, frequency domain amplitude features and frequency domain phase features are combined into frequency domain feature data. Taking into account the importance of different frequency components in the image, a weighted combination method can be used, with higher-energy frequency domain features having higher weights. The combined frequency domain feature data retains the main structural information and detailed features of the image.

[0031] In practical applications, such as medical image processing, when processing CT images containing Gaussian noise, the image can first be divided into 64×64 image blocks and sampled using double the sampling resolution. Normalization is performed using the Z-score method, normalizing pixel values ​​to a range with a mean of 0 and a standard deviation of 1. Feature vectors are extracted from the imaging parameters of the CT image to construct orthogonal basis vectors that consider the characteristics of human tissue. The generated frequency domain transformation matrix is ​​specifically optimized for representing the boundaries of human tissue. In the frequency domain feature data obtained from the transformation, the low-frequency part reflects the general structure of the tissue, while the high-frequency part contains details and noise information. Appropriate frequency domain processing can effectively remove noise while preserving tissue details.

[0032] The frequency domain feature data obtained by the above method provides a good data foundation for subsequent tasks such as image denoising, feature extraction, and image reconstruction, and can effectively handle noise problems while preserving key image information.

[0033] In one optional implementation, the frequency domain feature data is decomposed into amplitude spectrum components and phase spectrum components. A diffusion model is used to perform denoising prediction on the amplitude spectrum components and phase spectrum components to generate denoised amplitude spectrum components and denoised phase spectrum components. Combining the denoised amplitude spectrum components and denoised phase spectrum components into denoised frequency domain feature data includes: Frequency domain decomposition is performed on the frequency domain feature data to extract the amplitude information to obtain the amplitude spectrum component, and the angle information is extracted to obtain the phase spectrum component. Construct a feature mapping space, map the amplitude spectrum components to the feature mapping space to obtain the amplitude mapping features that characterize the amplitude distribution, and map the phase spectrum components to the feature mapping space to obtain the phase mapping features that characterize the angle distribution; The feature representation is obtained by combining the amplitude mapping feature and the phase mapping feature according to the frequency domain distribution relationship; A diffusion model containing a feature extraction layer and a noise prediction layer is constructed. The feature representation is input into the feature extraction layer to extract noise features, and the noise features are input into the noise prediction layer to generate noise distribution prediction. Based on noise distribution prediction, denoising prediction is performed on the amplitude spectrum component to generate a denoised amplitude spectrum component, and denoising prediction is performed on the phase spectrum component to generate a denoised phase spectrum component. The frequency domain distribution mapping relationship is extracted from the frequency domain feature data. Based on the frequency domain distribution mapping relationship, the amplitude distribution information in the denoised amplitude spectrum component and the angle distribution information in the denoised phase spectrum component are combined to generate denoised frequency domain feature data containing complete frequency domain features.

[0034] Frequency domain feature data can be decomposed into amplitude spectrum components and phase spectrum components. Denoising prediction is performed on these two components using a diffusion model to finally generate denoised frequency domain feature data.

[0035] First, frequency domain decomposition is performed on the frequency domain feature data. Frequency domain feature data is typically represented as a complex number F(u, v), where the real part represents the cosine component of the signal, and the imaginary part represents the sine component. The decomposition operation extracts the amplitude information to obtain the amplitude spectrum component |F(u, v)|, calculated as the square root of the sum of the squares of the real and imaginary parts. The angle information is extracted to obtain the phase spectrum component φ(u, v), calculated as the arctangent of the imaginary and real parts. The amplitude spectrum component describes the intensity of each frequency component of the signal, while the phase spectrum component describes the relative positional relationship between the frequency components.

[0036] Next, a feature mapping space is constructed to map the amplitude and phase spectral components to this space. For the amplitude spectral components, their dynamic range can be enhanced through a logarithmic transformation, generating an amplitude mapping feature A(u, v) = log(1 + |F(u, v)|) representing the amplitude distribution. For the phase spectral components, considering that the phase values ​​are distributed in the interval [-π, π], sine and cosine transformations can be used for encoding, generating phase mapping features Pcos(u, v) = cos(φ(u, v)) and Psin(u, v) = sin(φ(u, v)) representing the angular distribution. This encoding method avoids the discontinuity problem of phase values ​​near -π and π.

[0037] The amplitude mapping feature and the phase mapping feature are combined according to their frequency domain distribution relationship to obtain the feature representation. The combination method can be channel concatenation, where the amplitude mapping feature A(u, v), the phase cosine feature P_cos(u, v), and the phase sine feature Psin(u, v) are concatenated along the channel dimension to form a three-channel feature representation T(u, v) = [A(u, v), Pcos(u, v), Psin(u, v)]. This combination method preserves the spatial relationship of the frequency domain distribution while organically combining amplitude and phase information.

[0038] A diffusion model comprising a feature extraction layer and a noise prediction layer is constructed. The feature extraction layer can employ a multi-layer convolutional neural network structure, containing multiple residual blocks and attention modules, used to extract noise features from the input feature representation. Specifically, the feature extraction layer first maps the input feature dimension to the model's hidden dimension through a convolutional layer, and then performs feature extraction through multiple residual blocks, each containing two convolutional layers and a skip connection. During feature extraction, noise level is introduced as conditional information, injected into the feature extraction process through temporal embedding. The noise prediction layer consists of multiple convolutional layers, used to map the extracted noise features to a noise distribution prediction with the same dimensions as the input feature representation.

[0039] The feature representation is input into the feature extraction layer to extract noise features, and then the noise features are input into the noise prediction layer to generate noise distribution predictions. During the training phase, Gaussian noise is added to the feature representation, and the model learns to predict the added noise. During the inference phase, starting from pure noise, a clean feature representation is gradually recovered through an iterative denoising process.

[0040] Denoising prediction is performed on the amplitude and phase spectrum components based on noise distribution prediction. The generated noise distribution prediction is decomposed along the channel dimension to obtain amplitude spectrum noise prediction and phase spectrum noise prediction. The denoised amplitude spectrum component is generated by subtracting the predicted noise from the noisy amplitude spectrum component; similarly, the denoised phase spectrum component is generated by subtracting the predicted noise from the noisy phase spectrum component. For the phase spectrum component, the denoising results of cosine and sine coding need to be converted back to angle values, calculated as φ'(u, v) = atan2(Psin'(u, v), Pcos'(u, v)), where Psin' and Pcos' are the denoised sine and cosine phase features, respectively.

[0041] The frequency domain distribution mapping relationship is extracted from the frequency domain feature data. Based on this mapping relationship, the amplitude distribution information in the denoised amplitude spectrum component and the angular distribution information in the denoised phase spectrum component are combined to generate denoised frequency domain feature data. Specifically, the denoised amplitude spectrum component |F'(u,v)| and the denoised phase spectrum component φ'(u,v) are combined using Euler's formula: F'(u,v)=|F'(u,v)|·(cos(φ'(u,v))+i·sin(φ'(u,v))), where i is the imaginary unit. This combination method preserves the complex form of the frequency domain features, ensuring the integrity of the frequency domain features.

[0042] In audio signal processing applications, audio signals can be converted to a time-frequency representation using a Short-Time Fourier Transform (STFT) to obtain frequency domain feature data. After denoising the frequency domain feature data using the method described above, the denoised frequency domain feature data is then converted back to the time domain using an Inverse Short-Time Fourier Transform (ISTFT) to obtain the denoised audio signal. Experimental results show that, compared to directly applying a diffusion model to the frequency domain feature data, the method of decomposing the signal into amplitude and phase spectra for denoising separately can more effectively preserve the detailed features of the audio signal and improve the denoising effect, especially in low signal-to-noise ratio environments.

[0043] In one optional implementation, frequency modulation parameters are generated based on conditional input information, and the denoised frequency domain feature data is modulated based on the frequency modulation parameters to obtain modulated frequency domain feature data including: Semantic and spatial features are extracted from the conditional input information, a feature mapping matrix is ​​constructed, and the semantic features are converted into semantic modulation parameters and the spatial features are converted into spatial modulation parameters through the feature mapping matrix. Frequency modulation parameters are generated based on the semantic and spatial modulation parameters, and the frequency modulation parameters are used to guide frequency domain feature modulation. Decomposition operations are performed on the denoised frequency domain feature data to extract amplitude information to obtain amplitude components, and phase information is extracted from the denoised frequency domain feature data to obtain phase components; The amplitude component and the phase component are modulated separately based on the frequency modulation parameters to generate modulated amplitude components and phase components. The modulated amplitude component and phase component are combined to generate modulated frequency domain feature data.

[0044] In one embodiment, conditional input information is first received, which may include text descriptions, image tags, or user preferences. Extracting semantic and spatial features from the conditional input information is a fundamental step in achieving frequency modulation. Semantic features mainly reflect the theme, sentiment, or style attributes of the content; spatial features reflect the geometric attributes of the content, such as its spatial distribution and structural organization.

[0045] Semantic features can be extracted using deep neural network models, such as pre-trained models like Bidirectional Encoder Representations from Transformers (BERT) or Robustly Optimized BERT, which process textual information and output high-dimensional feature vectors as semantic features. For spatial features, convolutional neural networks can be used to extract the spatial distribution information of images, or positional encoding methods can be used to obtain the positional information of words in text.

[0046] Constructing the feature mapping matrix is ​​a crucial step in connecting the semantic space and the modulation parameter space. This matrix can be implemented using an attention mechanism or a fully connected layer. Specifically, the extracted semantic features and spatial features are input into their respective mapping networks to generate the semantic mapping matrix Msem and the spatial mapping matrix Mspa. The semantic features are transformed through Msem to obtain the semantic modulation parameters Psem, and the spatial features are transformed through Mspa to obtain the spatial modulation parameters Pspa.

[0047] The semantic modulation parameter Psem primarily controls the content attributes of frequency domain features, such as topic consistency and style intensity; the spatial modulation parameter Pspa affects the structural attributes of frequency domain features, such as edge sharpness and texture detail. By fusing Psem and Pspa, the final frequency modulation parameter Pfreq can be obtained. The fusion operation can be implemented using methods such as weighted summation, element-wise multiplication, or adaptive gating mechanisms.

[0048] The frequency modulation parameter Pfreq includes an amplitude modulation factor Pamp and a phase modulation factor Pphase, which are used to modulate the amplitude and phase information of the frequency domain features, respectively. These two modulation factors typically have dimensions that match the frequency domain features to allow for element-wise operations.

[0049] After obtaining the frequency modulation parameters, the denoised frequency domain feature data is decomposed. This denoised frequency domain feature data is typically obtained by performing a Fourier transform on the original data and then undergoing noise suppression processing. The denoised frequency domain feature data Fd is decomposed into an amplitude component |Fd| and a phase component φd for separate modulation operations.

[0050] The amplitude component |Fd| reflects the energy distribution of the frequency domain information, while the phase component φd carries structural information. The decomposition process can be represented as the extraction of magnitude and phase from the complex domain features. In practice, polar coordinates can be used to convert the complex frequency domain features into amplitude and phase components.

[0051] The decomposed amplitude and phase components are modulated based on frequency modulation parameters. During amplitude modulation, the amplitude component |Fd| is element-wise operated on using the amplitude modulation factor Pamp to obtain the modulated amplitude component |Fmod|. Common modulation methods include product modulation, weighted modulation, or conditional scaling. Phase modulation uses the phase modulation factor Pphase to adjust the phase component φd to obtain the modulated phase component φmod. Phase modulation is typically achieved using phase shifting or phase scaling.

[0052] In a specific example, amplitude modulation can be expressed as |Fmod|=|Fd|·(1+α·Pamp), where α is the modulation intensity coefficient; phase modulation can be expressed as φmod=φd+β·Pphase, where β is the phase modulation intensity coefficient. α and β can be adjusted according to application requirements to control the strength of the modulation effect.

[0053] Finally, the modulated amplitude component |Fmod| and phase component φmod are recombine to generate the modulated frequency domain feature data Fmod. The recombination process is the inverse operation of decomposition and can be achieved by converting the amplitude and phase components into recursive forms. The modulated frequency domain feature data retains the basic structure of the original features while adjusting its content and style based on the input conditions.

[0054] In practical applications, the modulated frequency domain feature data can be further subjected to inverse Fourier transform to convert it back to the time or spatial domain to obtain the final processing result. The entire modulation process can be optimized according to different application scenarios, enabling flexible control from text description to image generation, audio processing, or other signal transformations.

[0055] This invention is particularly suitable for conditional generation tasks, such as text-guided image generation, style transfer, or content editing. Through a frequency modulation mechanism, it can precisely control signal features based on conditional input information while preserving the basic structure of the original signal, achieving more flexible and controllable generation results.

[0056] In one optional implementation, an inverse fast Fourier transform is performed on the modulated frequency domain feature data to convert the modulated frequency domain feature data from a frequency domain representation to a spatial domain representation, resulting in denoised image data including: Construct an inverse transform matrix to separate the amplitude component and the phase component from the modulated frequency domain feature data. Map the amplitude component to the inverse transform matrix to generate amplitude transform parameters, and map the phase component to the inverse transform matrix to generate phase transform parameters. Spatial domain transformation parameters are generated based on amplitude transformation parameters and phase transformation parameters; The modulated frequency domain feature data is decomposed into multi-scale frequency domain components. Based on the spatial domain transformation parameters, the multi-scale frequency domain components are inversely transformed to generate multi-scale spatial domain components. The multi-scale spatial domain components are reconstructed to obtain the initial spatial domain feature data; Compensation information is extracted from the modulated frequency domain feature data and converted into spatial domain compensation data. The spatial domain compensation data is combined with the initial spatial domain feature data to generate complete spatial domain feature data, which is then output as denoised image data.

[0057] The process of performing an inverse fast Fourier transform (IFT) on modulated frequency domain feature data first requires constructing an inverse transform matrix. During IFT construction, amplitude and phase components are separated from the modulated frequency domain feature data. The amplitude component is typically represented as a complex absolute value, while the phase component is represented as a complex phase angle. Mapping the amplitude component to the IFT matrix involves processing the amplitude component using a mapping function to generate amplitude transform parameters. The mapping function can be an exponential, logarithmic, or linear function, the specific choice depending on the image characteristics. For example, for images containing high-frequency noise, an exponential decay function can be used to reduce the influence of high-frequency amplitude and improve the retention of low-frequency information. Similarly, mapping the phase component to the IFT matrix involves generating phase transform parameters through a phase adjustment function to ensure accurate reconstruction of phase information.

[0058] Based on the generated amplitude and phase transformation parameters, spatial domain transformation parameters are generated using corresponding weighting coefficients. These parameters can be expressed as complex numbers, where the real part comes from the amplitude transformation parameters and the imaginary part comes from the phase transformation parameters. The weighting coefficients are selected based on the statistical characteristics of the image; for example, for images containing local noise, increasing the weighting coefficients for the corresponding regions can improve the denoising effect.

[0059] When performing decomposition operations on the modulated frequency domain feature data, a multi-scale decomposition strategy is employed. First, the frequency domain feature data is decomposed using a bandpass filter bank to obtain multi-scale frequency domain components in different frequency ranges. For example, the frequency domain data can be decomposed into low-frequency, mid-frequency, and high-frequency components. For each frequency domain component, an inverse transform operation is performed based on the previously generated spatial domain transform parameters to obtain the corresponding multi-scale spatial domain component. During the inverse transform operation, a strong regularization constraint is applied to the high-frequency components to suppress the influence of noise; while for the low-frequency components, more original information is preserved to ensure that the main structure of the image is not destroyed.

[0060] The obtained multi-scale spatial domain components are reconstructed to generate initial spatial domain feature data. The reconstruction process employs a weighted fusion strategy, assigning different weights to components of different scales based on their importance. Typically, low-frequency components contain the main structural information of the image and are given higher weights; mid-frequency components contain the texture information of the image and are given medium weights; high-frequency components contain edge and detail information, and their weights are dynamically adjusted according to the noise level. During the fusion process, a nonlinear activation function is used to enhance the useful signal and suppress residual noise.

[0061] Compensation information is extracted from the modulated frequency domain feature data, primarily targeting the details that may be lost during frequency domain modulation. The extraction process involves analyzing the differences between the data before and after frequency domain modulation, establishing a mapping relationship, and generating a compensation matrix. This compensation matrix is ​​then converted into spatial domain compensation data through an inverse transform. The same inverse transform method as used for the main data is employed during the transformation to ensure that the compensation data and the initial spatial domain feature data reside in the same representation space.

[0062] An adaptive fusion strategy is employed when combining spatial domain compensation data with initial spatial domain feature data. For smooth regions in the image, the influence of compensation data is reduced to avoid introducing artifacts; for edge and textured regions, the contribution of compensation data is enhanced to restore details that may have been over-smoothed. Local variance is used as a weighting metric during the fusion process to ensure a natural transition in the fusion result. Finally, complete spatial domain feature data is generated as the output denoised image data.

[0063] In practical applications, the parameter settings in the above process can be adjusted for different types of image noise. For example, for additive white Gaussian noise, the suppression of high-frequency components can be strengthened in the frequency domain modulation stage; for mixed noise, more complex modulation strategies can be designed by combining the characteristics of the frequency domain and the spatial domain. In addition, the inverse transform parameters can be dynamically adjusted according to the content characteristics of the image, such as texture richness and edge complexity, to improve the denoising effect while preserving the image's detailed information.

[0064] In medical image processing applications, CT images are often affected by noise under low-dose imaging conditions. When applying the above method, the characteristics of CT images can be considered: during the frequency domain modulation stage, anatomically relevant frequency components can be preserved, while during the inverse transformation, stronger constraints can be applied to noise-sensitive frequency components, thereby reducing noise while maintaining the critical details required for diagnosis.

[0065] Overall, by performing inverse fast Fourier transform on the modulated frequency domain feature data, combined with multi-scale decomposition and reconstruction, and the extraction and fusion of compensation information, the modulated frequency domain feature data can be effectively converted into denoised image data, thereby improving image quality.

[0066] In one optional implementation, the denoised image data is used as new noisy image data, and the iteration is repeated until the time step termination condition of the diffusion model is met, and the output generated image data includes: Construct a feature discrimination matrix, extract local and global features from the denoised image data, map the local features to the feature discrimination matrix to calculate local difference parameters, map the global features to the feature discrimination matrix to calculate global difference parameters, and generate time step judgment parameters based on the local and global difference parameters. The denoised image data is input into the diffusion process to perform iterative calculations. The feature change data generated in each iteration is recorded. The feature change data is then converted into structural change data and texture change data through a decomposition matrix. Iterative optimization parameters are calculated based on the structural change data and texture change data. These iterative optimization parameters are used to guide the next iteration. The iterative optimization parameters and time step judgment parameters are combined and analyzed to generate termination judgment data. Based on the termination judgment data, it is determined whether the preset time step termination condition is met. Feature processing is performed on the denoised image data that meets the termination condition to extract structural and texture information, and the structural and texture information is fused to generate the final generated image data.

[0067] In one implementation, a feature discrimination matrix is ​​first constructed to evaluate the differences in image features. The feature discrimination matrix is ​​a multi-dimensional tensor containing multiple pre-defined feature templates, each corresponding to a different type of image feature. Local features, including edges, textures, and color gradients, are extracted from the denoised image data; simultaneously, global features, including overall attributes such as color distribution, contrast, and brightness, are extracted. Multi-scale convolutional kernels are used for sliding window operations during feature extraction to obtain feature response values ​​at different scales.

[0068] The extracted local features are mapped to a feature discrimination matrix to calculate local difference parameters. Specifically, the similarity score is calculated between the feature vector extracted from each local region and the corresponding template in the feature discrimination matrix. Similarity calculation can be implemented using methods such as cosine similarity or Euclidean distance. The local difference parameter is represented as a weighted average of the similarity scores of all local regions, with the weights dynamically adjusted according to the importance of the regions.

[0069] Similarly, global features are mapped to a feature discrimination matrix to calculate global difference parameters. Global feature vectors are matched with global templates in the feature discrimination matrix to calculate overall similarity. Global difference parameters reflect the difference in overall style and structure between the current denoised image and the ideal target image.

[0070] Time step decision parameters are generated based on local and global difference parameters. These parameters are scalar values, obtained by weighting the local and global difference parameters: Time step decision parameter = m × local difference parameter + n × global difference parameter, where m and n are weighting coefficients that are dynamically adjusted according to the image generation stage. In the early stages of generation, n is larger to ensure a reasonable global structure; in the later stages, m is increased to focus on optimizing local details.

[0071] The denoised image data is input into the diffusion process for iterative computation. In each iteration, the noisy image is denoised using the prediction network of the diffusion model, generating a new denoised image. Simultaneously, the feature change data generated in each iteration is recorded, including the changes in feature maps at different levels.

[0072] Feature variation data is transformed into structural variation data and texture variation data using a decomposition matrix. The decomposition matrix is ​​a pre-trained transformation matrix that maps the feature space to the structural and texture spaces. Structural variation data reflects changes in basic elements such as the image skeleton and layout; texture variation data reflects changes in surface features such as details and materials.

[0073] Iterative optimization parameters are calculated based on structural and textural variation data. These parameters consist of two parts: a structural optimization factor and a texture optimization factor. The structural optimization factor is dynamically adjusted based on the convergence rate of the structural variation data; when the structural variation is small, it indicates that the structure is basically stable, and the step size for structural optimization can be reduced. The texture optimization factor is adjusted based on the magnitude of changes in the texture variation data to ensure that the detail richness achieves the expected effect. These iterative optimization parameters guide the adjustment of the learning rate and attention mechanism in the next iteration.

[0074] The iterative optimization parameters and time step judgment parameters are combined for analysis to generate termination judgment data. The combined analysis employs a weighted decision tree model, using both sets of parameters as input features to predict whether the current iteration should terminate. The termination judgment data is a probability value reflecting the likelihood that the current iteration meets the termination condition. When the termination judgment data exceeds a preset threshold, the preset time step termination condition is considered met.

[0075] Feature processing is performed on the denoised image data that meets the termination criteria to extract structural and texture information. Structural information is extracted using a low-pass filter and edge detection algorithm; texture information is extracted using a high-pass filter and texture descriptors. The structural and texture information are then fused through a feature fusion network. This network employs an attention mechanism to adaptively weight structural and texture features, ensuring rich detail while preserving structural integrity. The fused features are then decoded to generate the final generated image data.

[0076] In a specific application scenario, when generating high-resolution portrait images, the above method effectively balances the accuracy of facial structure with the realism of skin texture. The constructed feature discrimination matrix includes facial keypoint positional relationship templates and skin texture templates. During the iteration process, local difference parameters mainly focus on the accuracy of facial features such as eyes, nose, and mouth, while global difference parameters focus on the naturalness of overall skin tone and lighting. Through the analysis of structural change data and texture change data, natural skin texture and hair details can be generated while ensuring facial proportion harmony. When the termination condition is met, the feature processing stage further enhances the clarity of facial contours and the realism of skin texture, ultimately outputting a high-quality portrait image.

[0077] The method described above achieves adaptive termination judgment in the diffusion model generation process through feature discrimination matrix and iterative optimization strategy, effectively improving the quality and efficiency of generated images. Compared with the traditional fixed-time-step generation method, this method can dynamically adjust the number of iterations according to the image content, avoiding the problems of detail loss due to excessive iteration or unclear structure caused by insufficient iteration.

[0078] In one optional implementation, the denoised image data is input into the diffusion process for iterative calculation, and the feature change data generated in each iteration is recorded. The feature change data is then converted into structural change data and texture change data through a decomposition matrix, including: The denoised image data is input into the diffusion process. During the diffusion process, a forward diffusion operation is performed on the denoised image data to obtain the diffusion state features. A reverse denoising operation is performed on the diffusion state features to obtain the first round of iterative denoising results. Extract the initial feature representation of the denoised image data and the output feature representation of the first round of denoising results, calculate the feature difference value between the initial feature representation and the output feature representation, and record the feature difference value as the feature change data generated in the first round of iteration; The first round of denoising results is used as new input data to continue the diffusion process to perform iterative calculations. The forward diffusion operation and the reverse denoising operation are repeatedly executed. The feature representations of the input data and the output data of each round of iteration are extracted. The feature difference value of each round of iteration is calculated, and the feature change data generated in each round of iteration is recorded. Construct a decomposition matrix, perform linear transformation on the feature change data through the decomposition matrix to obtain the decomposed feature representation, perform subspace projection on the decomposed feature representation to extract the edge response subspace projection and detail response subspace projection respectively, and fuse the edge response subspace projection and detail response subspace projection to generate texture change data; Subspace projection is performed on the decomposed feature representation to extract the region connectivity subspace projection and the morphological distribution subspace projection respectively. The region connectivity subspace projection and the morphological distribution subspace projection are then fused to generate structural change data.

[0079] First, prepare the denoised image data to be processed. This image data can be an image that has undergone preliminary denoising but still needs further optimization. Use this denoised image data as the initial input data for iterative calculations.

[0080] The denoised image data is input into a pre-trained diffusion model, and a forward diffusion operation is performed. The forward diffusion operation is a process of gradually adding noise to the image data. Specifically, a Markov chain structure can be used, adding Gaussian noise at each step according to a predefined noise schedule. For example, for the input image x0, using the formula x... t =(α t ) 1 / 2 ·x (t-1) +(1-α t ) 1 / 2 ·ε calculates the diffusion state, where α t Here, ε represents the scheduling parameter and Gaussian noise. Multi-step forward diffusion is used to obtain the diffusion state characteristics of the image under different noise levels.

[0081] The obtained diffusion state features are subjected to inverse denoising, which employs a pre-trained neural network model that predicts and removes noise components added at each step. The inverse denoising process starts from the end of the diffusion chain and gradually recovers the original image information, ultimately yielding the denoising result of the first iteration.

[0082] The initial feature representation of the denoised image data and the output feature representation of the first iteration of denoising are extracted. Feature extraction can be accomplished using a convolutional neural network, and the extracted features include multi-level image features, such as low-level texture features and high-level semantic features. The feature difference value between the initial feature representation and the output feature representation is calculated. The difference value can be obtained using metrics such as Euclidean distance or cosine similarity, and this difference value is recorded as the feature change data generated in the first iteration.

[0083] The denoising result from the first iteration is used as new input data to continue the diffusion process, repeating the forward diffusion and backward denoising operations. In each iteration, the feature representations of the input and output data for the current round are extracted, and the feature difference between them is calculated and recorded as the feature change data for the current round. The iteration process can continue for multiple rounds until a predetermined termination condition is met, such as the feature change falling below a threshold or reaching the maximum number of iterations.

[0084] A decomposition matrix is ​​constructed for transforming feature-varying data. This decomposition matrix can be pre-learned using methods such as principal component analysis or nonnegative matrix factorization, with the aim of decomposing the feature space into multiple subspaces with specific semantics. The recorded feature-varying data is then linearly transformed using this decomposition matrix to obtain the decomposed feature representation.

[0085] Subspace projection is performed on the decomposed feature representation to extract edge response subspace projection and detail response subspace projection. The edge response subspace mainly captures edge change information in the image, while the detail response subspace captures texture detail change information. In specific implementation, a dedicated projection matrix can be designed, and matrix multiplication operations are performed on the decomposed feature representation to achieve projection. The edge response subspace projection and detail response subspace projection are then fused using a weighted method to generate texture change data, and the fusion weights can be adjusted according to the specific application scenario.

[0086] Similarly, another set of subspace projections is applied to the decomposed feature representation, extracting the region connectivity subspace projection and the morphological distribution subspace projection. The region connectivity subspace mainly focuses on changes in the connectivity relationships between regions in the image, while the morphological distribution subspace focuses on changes in the overall morphological structure. Projection is achieved through similar projection matrix operations, and the region connectivity subspace projection and the morphological distribution subspace projection are fused to generate structural change data.

[0087] In practical applications, the above methods can be applied to the field of medical image processing. For example, when processing low-dose CT images, structural change data extracted through the iterative diffusion process can help preserve key structural information such as organ boundaries, while texture change data helps restore tissue details, thus maintaining the diagnostic value of the image while reducing noise.

[0088] Furthermore, in remote sensing image processing, structural change data can be used to preserve the spatial structure of surface features, such as road networks and building outlines, while texture change data helps to restore detailed features such as vegetation and water bodies, thereby improving the visual quality and information content of images.

[0089] The structural and textural change data obtained through the above methods can be further used for downstream tasks such as image enhancement, image segmentation, or object recognition, improving the performance and robustness of these tasks.

[0090] A second aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0091] A third aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0092] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A neural network image generation method based on diffusion model and fast Fourier transform, characterized in that, include: Acquire noisy image data and corresponding conditional input information, perform Fast Fourier Transform on the noisy image data, and convert the noisy image data from spatial domain representation to frequency domain to obtain frequency domain feature data; The frequency domain feature data is decomposed into amplitude spectrum components and phase spectrum components. The amplitude spectrum components and phase spectrum components are denoised and predicted by a diffusion model to generate denoised amplitude spectrum components and denoised phase spectrum components. The denoised amplitude spectrum components and denoised phase spectrum components are then combined to form denoised frequency domain feature data. Frequency modulation parameters are generated based on the input information. The denoised frequency domain feature data is then modulated based on the frequency modulation parameters to obtain the modulated frequency domain feature data. Perform an inverse fast Fourier transform on the modulated frequency domain feature data to convert the modulated frequency domain feature data from frequency domain representation to spatial domain representation, and obtain denoised image data; The denoised image data is used as new noisy image data, and the process is repeated iteratively until the time step termination condition of the diffusion model is met, and the generated image data is output.

2. The method according to claim 1, characterized in that, Acquire noisy image data and corresponding conditional input information, perform a Fast Fourier Transform on the noisy image data to convert the noisy image data from spatial domain representation to frequency domain to obtain frequency domain feature data, including: The noisy image data is divided into multiple image blocks according to a preset size. The multiple image blocks are sampled according to the sampling resolution to obtain sampled noisy image data. The sampled noisy image data is then normalized to obtain normalized noisy image data. Feature vectors are extracted from the conditional input information to construct orthogonal basis vectors, and the orthogonal basis vectors are combined according to spatial structure rules to generate a frequency domain transformation matrix; The normalized noisy image data is transformed by the frequency domain transformation matrix to obtain the initial frequency domain feature data, and the frequency domain amplitude feature and frequency domain phase feature are separated from the initial frequency domain feature data. The spatial structure rules based on orthogonal basis vectors combine frequency domain amplitude features and frequency domain phase features into frequency domain feature data.

3. The method according to claim 1, characterized in that, The frequency domain feature data is decomposed into amplitude spectrum components and phase spectrum components. A diffusion model is used to perform denoising prediction on the amplitude spectrum components and phase spectrum components, generating denoised amplitude spectrum components and denoised phase spectrum components. The denoised amplitude spectrum components and denoised phase spectrum components are then combined to form denoised frequency domain feature data, including: Frequency domain decomposition is performed on the frequency domain feature data to extract the amplitude information to obtain the amplitude spectrum component, and the angle information is extracted to obtain the phase spectrum component. Construct a feature mapping space, map the amplitude spectrum components to the feature mapping space to obtain the amplitude mapping features that characterize the amplitude distribution, and map the phase spectrum components to the feature mapping space to obtain the phase mapping features that characterize the angle distribution; The feature representation is obtained by combining the amplitude mapping feature and the phase mapping feature according to the frequency domain distribution relationship; A diffusion model containing a feature extraction layer and a noise prediction layer is constructed. The feature representation is input into the feature extraction layer to extract noise features, and the noise features are input into the noise prediction layer to generate noise distribution prediction. Based on noise distribution prediction, denoising prediction is performed on the amplitude spectrum component to generate a denoised amplitude spectrum component, and denoising prediction is performed on the phase spectrum component to generate a denoised phase spectrum component. The frequency domain distribution mapping relationship is extracted from the frequency domain feature data. Based on the frequency domain distribution mapping relationship, the amplitude distribution information in the denoised amplitude spectrum component and the angle distribution information in the denoised phase spectrum component are combined to generate denoised frequency domain feature data containing complete frequency domain features.

4. The method according to claim 1, characterized in that, Frequency modulation parameters are generated based on the input information. The denoised frequency domain feature data is then modulated using these parameters to obtain the modulated frequency domain feature data, which includes: Semantic and spatial features are extracted from the conditional input information, a feature mapping matrix is ​​constructed, and the semantic features are converted into semantic modulation parameters and the spatial features are converted into spatial modulation parameters through the feature mapping matrix. Frequency modulation parameters are generated based on the semantic and spatial modulation parameters, and the frequency modulation parameters are used to guide frequency domain feature modulation. Decomposition operations are performed on the denoised frequency domain feature data to extract amplitude information to obtain amplitude components, and phase information is extracted from the denoised frequency domain feature data to obtain phase components; The amplitude component and the phase component are modulated separately based on the frequency modulation parameters to generate modulated amplitude components and phase components. The modulated amplitude component and phase component are combined to generate modulated frequency domain feature data.

5. The method according to claim 1, characterized in that, Performing an inverse fast Fourier transform on the modulated frequency domain feature data converts the frequency domain representation to a spatial domain representation, resulting in denoised image data including: Construct an inverse transform matrix to separate the amplitude component and the phase component from the modulated frequency domain feature data. Map the amplitude component to the inverse transform matrix to generate amplitude transform parameters, and map the phase component to the inverse transform matrix to generate phase transform parameters. Spatial domain transformation parameters are generated based on amplitude transformation parameters and phase transformation parameters; The modulated frequency domain feature data is decomposed into multi-scale frequency domain components. Based on the spatial domain transformation parameters, the multi-scale frequency domain components are inversely transformed to generate multi-scale spatial domain components. The multi-scale spatial domain components are reconstructed to obtain the initial spatial domain feature data; Compensation information is extracted from the modulated frequency domain feature data and converted into spatial domain compensation data. The spatial domain compensation data is combined with the initial spatial domain feature data to generate complete spatial domain feature data, which is then output as denoised image data.

6. The method according to claim 1, characterized in that, The denoised image data is used as new noisy image data, and the process is repeated iteratively until the time step termination condition of the diffusion model is met. The output generated image data includes: Construct a feature discrimination matrix, extract local and global features from the denoised image data, map the local features to the feature discrimination matrix to calculate local difference parameters, map the global features to the feature discrimination matrix to calculate global difference parameters, and generate time step judgment parameters based on the local and global difference parameters. The denoised image data is input into the diffusion process to perform iterative calculations. The feature change data generated in each iteration is recorded. The feature change data is then converted into structural change data and texture change data through a decomposition matrix. Iterative optimization parameters are calculated based on the structural change data and texture change data. These iterative optimization parameters are used to guide the next iteration. The iterative optimization parameters and time step judgment parameters are combined and analyzed to generate termination judgment data. Based on the termination judgment data, it is determined whether the preset time step termination condition is met. Feature processing is performed on the denoised image data that meets the termination condition to extract structural and texture information, and the structural and texture information is fused to generate the final generated image data.

7. The method according to claim 1, characterized in that, The denoised image data is input into the diffusion process for iterative calculation. The feature change data generated in each iteration is recorded. The feature change data is then transformed into structural change data and texture change data through a decomposition matrix, including: The denoised image data is input into the diffusion process. During the diffusion process, a forward diffusion operation is performed on the denoised image data to obtain the diffusion state features. A reverse denoising operation is performed on the diffusion state features to obtain the first round of iterative denoising results. Extract the initial feature representation of the denoised image data and the output feature representation of the first round of denoising results, calculate the feature difference value between the initial feature representation and the output feature representation, and record the feature difference value as the feature change data generated in the first round of iteration; The first round of denoising results is used as new input data to continue the diffusion process to perform iterative calculations. The forward diffusion operation and the reverse denoising operation are repeatedly executed. The feature representations of the input data and the output data of each round of iteration are extracted. The feature difference value of each round of iteration is calculated, and the feature change data generated in each round of iteration is recorded. Construct a decomposition matrix, perform linear transformation on the feature change data through the decomposition matrix to obtain the decomposed feature representation, perform subspace projection on the decomposed feature representation to extract the edge response subspace projection and detail response subspace projection respectively, and fuse the edge response subspace projection and detail response subspace projection to generate texture change data; Subspace projection is performed on the decomposed feature representation to extract the region connectivity subspace projection and the morphological distribution subspace projection respectively. The region connectivity subspace projection and the morphological distribution subspace projection are then fused to generate structural change data.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images