An image super-resolution method based on dynamic texture noise synthesis
Patent Information
- Application Number
- CN202611030307.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-11
- Publication Date
- 2026-10-27
AI Technical Summary
[0006]本发明的目的在于克服目前扩散模型的超分辨率方法在应对训练与推理信号不匹配、高频细节恢复不足以及退化模式干扰等方面，尚未形成高效且统一的解决方案的技术缺陷，提供一种基于动态纹理噪声合成的图像超分辨率方法
（1）本发明基于低分辨率图像的局部梯度统计特性，通过条件生成网络合成与图像内容自适应、空间变化的纹理噪声，并将该合成噪声与纯高斯噪声融合构建推理初始噪声。该初始噪声嵌入了低分辨率图像的高频先验信息，使推理起点与训练阶段的信号分布更加接近，从根本上解决了传统扩散模型因训练-推理信号不匹配而导致的高频细节恢复困难问题，显著提升了重建图像的动态范围与纹理丰富度。
Smart Images
Figure CN122989420A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision and image processing technology, specifically to an image super-resolution method based on dynamic texture noise synthesis. Background Technology
[0002] Image super-resolution aims to recover high-resolution images with rich high-frequency details from low-resolution images. It is a fundamental task in the field of computer vision and is widely used in practical scenarios such as video enhancement, medical imaging, satellite remote sensing, and mobile phone photography. In recent years, diffusion-based super-resolution methods have attracted widespread attention due to their high generation quality and strong detail recovery capabilities, becoming one of the mainstream technical directions in this field.
[0003] Image super-resolution methods based on diffusion models typically learn high-frequency details by incorporating the residual signal from the corresponding high-resolution image during training. However, due to the non-zero signal-to-noise ratio at the diffusion termination time step in commonly used noise scheduling schemes, the model can perceive and rely on the residual signal from the high-resolution image (containing low-frequency information such as the global mean) during training. In the inference phase, the generation process begins with pure Gaussian noise with a zero mean, lacking the aforementioned residual signal. This training-inference signal mismatch prevents the model from effectively recovering high-frequency details in the image during inference, resulting in limited dynamic range, blurred textures, and a tendency to generate mediocre results with moderate brightness, severely restricting the perceptual quality of super-resolution images in practical applications.
[0004] Existing technologies have attempted to address these issues from different angles. For example, ControlNet-based conditional diffusion methods attempt to directly inject low-resolution images into the initial noise during inference to provide structure guidance. However, when the input image is severely degraded (e.g., containing noise or compression artifacts), this strategy introduces the degraded components from the low-resolution image into the initial state, leading to unreasonable artifacts in the generated image and exacerbating image distortion rather than improving it. Other methods reduce the training-inference gap by adjusting the noise scheduling scheme, but these methods often require retraining the model and struggle to fundamentally address the problem of missing high-frequency priors.
[0005] In summary, existing super-resolution methods based on diffusion models have not yet formed an efficient and unified solution to the problems of training and inference signal mismatch, insufficient high-frequency detail recovery, and degradation mode interference. There is an urgent need for a method that can effectively bridge the signal gap and provide high-quality high-frequency priors for the generation process. Summary of the Invention
[0006] The purpose of this invention is to overcome the technical shortcomings of current diffusion model super-resolution methods in dealing with issues such as mismatch between training and inference signals, insufficient high-frequency detail recovery, and interference from degradation modes, which have not yet formed an efficient and unified solution. This invention provides an image super-resolution method based on dynamic texture noise synthesis.
[0007] To achieve the above objectives, the present invention employs the following technical solution: an image super-resolution method based on dynamic texture noise synthesis, comprising the following steps: S1. Obtain a low-resolution image, extract the local gradient statistical features of the low-resolution image, and synthesize texture noise that is adapted to the image content through a conditional generation network based on the local gradient statistical features. S2. Fuse texture noise and pure Gaussian noise at a preset ratio to construct the initial noise for inference; S3. Starting with the initial noise of inference and using the low-resolution image as a condition, perform iterative denoising through a pre-trained diffusion model to generate a high-resolution image. S4. Introduce a noise decoupling loss function and combine it with the standard noise prediction loss of the diffusion model to form a joint optimization objective. Train the model to make the generated results far away from the degradation mode in the feature space.
[0008] Furthermore, the "extracting local gradient statistical features of the low-resolution image" mentioned in step S1 specifically refers to using a pre-trained gradient estimation network to calculate the gradient magnitude of each local region of the low-resolution image. G ( x , y and gradient direction θ ( x , y This forms a gradient statistical feature map; the gradient magnitude... G ( x , y and gradient direction θ ( x , y The calculation formulas for ) are as follows: , in, This is a low-resolution image.
[0009] The phrase "synthesizing texture noise adapted to image content through a conditional generation network based on the local gradient statistical features" in step S1 specifically refers to: using the gradient statistical feature map as a condition, a conditional generation network learns to convert the gradient statistical feature map into texture noise. mapping Generate texture noise aligned with the gradient statistical feature map space. This allows the spatial distribution of noise to match the local structure of the image.
[0010] In step S2, the process of "fusing texture noise and pure Gaussian noise at a preset ratio to construct initial inference noise" is described. The formula for calculating this initial noise is as follows: ,in, For texture noise, It is pure Gaussian noise. This is the fusion coefficient.
[0011] The step S3, "performing iterative denoising using a pre-trained diffusion model," follows a Markov chain denoising process. Step-by-step noise reduction output By the Output and noise prediction network The decision is made jointly, and the calculation formula is as follows: ,in These are scheduling parameters.
[0012] The calculation formula for "introducing the noise decoupling loss function" mentioned in step S4 is as follows: ,in, For true high-resolution images, To Degraded reference image obtained after applying artificial degradation operations. This is a pre-trained feature extraction network.
[0013] Compared with the prior art, the beneficial effects of the present invention are: (1) Based on the local gradient statistical characteristics of low-resolution images, this invention synthesizes texture noise that is adaptive and spatially varied with image content through a conditional generation network, and then fuses this synthesized noise with pure Gaussian noise to construct the initial noise for inference. This initial noise embeds high-frequency prior information of the low-resolution image, making the signal distribution of the inference starting point closer to that of the training stage. This fundamentally solves the problem of difficulty in recovering high-frequency details caused by the mismatch between training and inference signals in traditional diffusion models, and significantly improves the dynamic range and texture richness of the reconstructed image.
[0014] (2) This invention introduces a noise decoupling loss function, constructs a set of degraded reference images by applying artificial degradation to real high-resolution images, and calculates the negative similarity between the generated image and the degraded image in the deep feature space, forcing the generated result to be far away from known degradation patterns such as blurring and noise in feature representation. This mechanism gives the model the ability to actively avoid low-quality textures from the source of optimization, ensuring the visual naturalness and texture fidelity of the generated image.
[0015] (3) This invention uses low-resolution images as a condition and synthetic noise rich in high-frequency priors as the starting point for inference. It gradually recovers high-frequency details through an iterative denoising process using a diffusion model. Synthetic noise provides semantic awareness as the starting point for the denoising process, and conditional injection ensures the accurate transmission of structural information. The synergistic effect of the two makes the generated image superior to existing methods in terms of texture sharpness, structural integrity, and dynamic range.
[0016] (4) This invention encapsulates core computing units such as dynamic texture noise synthesis and conditional diffusion inference into independent modules, which can be flexibly combined through a streaming data bus and deployed in a SoC parallel computing architecture that employs INT8 quantization, double-buffered weight prefetching, and multi-level DMA scheduling. This hardware and software co-design achieves low-power real-time inference while ensuring reconstruction quality, and is suitable for application scenarios with stringent requirements for size, power consumption, and real-time performance, such as automotive, airborne, spaceborne, and edge terminals. Attached Figure Description
[0017] Figure 1 This is a schematic diagram of the overall process of the present invention. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example
[0019] The core of this embodiment lies in: instead of directly using low-resolution image signals, it extracts local texture statistics and dynamically synthesizes a spatially varying texture noise that adapts to the image content, simulating the high-frequency residual signal present during the training phase. This synthesized noise is embedded into the initial pure Gaussian noise for inference, providing a semantically aware starting point for the diffusion model, thereby effectively bridging the gap between training and inference. The specific steps are as follows: Figure 1 As shown.
[0020] See Figure 1 As shown, the image super-resolution method based on dynamic texture noise synthesis described in this embodiment includes four steps, S1 to S4, specifically: S1. Obtain a low-resolution image, extract the local gradient statistical features of the low-resolution image, and synthesize texture noise that is adapted to the image content through a conditional generation network based on the local gradient statistical features.
[0021] This step is dynamic texture noise synthesis, which aims to solve the signal mismatch problem in the training and inference stages. Its core is to dynamically synthesize a texture noise rich in high-frequency prior information based on the content of the low-resolution image, so as to provide a semantically aware starting point for the subsequent generation process of the diffusion model.
[0022] This step specifically includes the following sub-steps: S11. Extract local gradient statistical features.
[0023] For the input low-resolution image, a pre-trained gradient estimation network (such as the Sobel operator or a lightweight CNN) is used to calculate the gradient magnitude and gradient direction of each local region, forming a gradient statistical feature map.
[0024] The gradient statistical features can characterize the local structural complexity of an image (such as edge and texture density), providing prior guidance for spatial variations in subsequent noise synthesis; while the gradient statistical feature map (denoted as...) The intermediate data representation proposed in this embodiment is essentially a two-dimensional structured matrix formed by arranging the local gradient information (gradient magnitude and gradient direction) at each pixel position of the input low-resolution image according to spatial coordinates.
[0025] The gradient magnitude G ( x , y and gradient direction θ ( x , y The calculation formulas for ) are as follows: ,in, This is a low-resolution image.
[0026] Step S12: Generate content-adaptive texture noise.
[0027] This step involves synthesizing texture noise adapted to the image content using a conditional generation network based on the local gradient statistical features. Specifically, the conditional generation network learns to convert the gradient statistical feature map into texture noise, using the gradient statistical feature map as a condition. mapping Generate texture noise aligned with the gradient statistical feature map space. This mapping matches the spatial distribution of noise to the local structure of the image. Mathematically, this mapping is represented as: .
[0028] in, The gradient statistical feature map is used, and the conditional generation network is preferably a noise synthesis network based on the UNet architecture. Because UNet has the characteristics of multi-scale feature extraction, skip connections preserving details, powerful image inductive bias, and high flexibility, it is very suitable as the backbone of the conditional generation network in this invention to synthesize texture noise that highly matches the local structure of the image.
[0029] S2. Fuse texture noise and pure Gaussian noise at a preset ratio to construct the initial noise for inference.
[0030] The purpose of this step is to embed the high-frequency structural prior information of the image into the initial noise of the inference, so that the diffusion model can obtain the distribution guidance of high-frequency details at the starting point of denoising. It cannot directly copy low-resolution images (which would introduce degradation), nor can it start empty (pure Gaussian noise lacks priors). Instead, it extracts structural clues and embeds them into the starting point in the form of noise, so that the model knows the approximate location of high-frequency details from the beginning, thereby effectively bridging the training-inference signal gap.
[0031] The formula for calculating the initial noise is: ,in, For texture noise, It is pure Gaussian noise. This is the fusion coefficient.
[0032] Through the above steps, the initial noise in this embodiment embeds high-frequency prior information from the low-resolution image, which can effectively avoid the defect of signal mismatch between the inference starting point and the training starting point.
[0033] S3. Starting with the initial noise of inference and using the low-resolution image as a condition, perform iterative denoising through a pre-trained diffusion model to generate a high-resolution image.
[0034] This step starts with initial inference noise and uses a low-resolution image as a structural guide condition to drive a pre-trained diffusion model to perform an iterative inverse denoising process. In each denoising step, the structural contours and high-frequency details of the image are gradually restored, ultimately generating a high-resolution reconstructed image with rich texture and clear details. In other words, it transforms "initial noise containing high-frequency structural cues" into "a high-resolution image with rich texture details".
[0035] The iterative denoising process performed using a pre-trained diffusion model follows a Markov chain, where the denoising process... Step-by-step noise reduction output By the Output and noise prediction network The decision is made jointly, and the calculation formula is as follows: ,in These are scheduling parameters.
[0036] This step enables the generated high-resolution image to achieve significant improvements in four dimensions: texture fidelity, structural accuracy, dynamic range, and perceptual quality.
[0037] S4. Introduce a noise decoupling loss function and combine it with the standard noise prediction loss of the diffusion model to form a joint optimization objective. Train the model to make the generated results far away from the degradation mode in the feature space.
[0038] The core function of this step is to introduce an additional optimization objective during the model training process, guiding the diffusion model to actively avoid generating low-quality degradation patterns such as blurring and artifacts while accurately denoising.
[0039] The formula for calculating the noise decoupling loss function is as follows: ,in, For true high-resolution images, To Degraded reference image obtained after applying artificial degradation operations. This is a pre-trained feature extraction network.
[0040] The expression for the joint optimization objective is: ,in, The standard noise prediction loss for the diffusion model, The noise decoupling loss function is... , These are the weighting coefficients.
[0041] As described above, the present invention can be well implemented.
Claims
1. An image super-resolution method based on dynamic texture noise synthesis, characterized in that, Includes the following steps: S1. Obtain a low-resolution image, extract the local gradient statistical features of the low-resolution image, and synthesize texture noise that is adapted to the image content through a conditional generation network based on the local gradient statistical features. S2. Fuse texture noise and pure Gaussian noise at a preset ratio to construct the initial noise for inference; S3. Starting with the initial noise of inference and using the low-resolution image as a condition, perform iterative denoising through a pre-trained diffusion model to generate a high-resolution image. S4. Introduce a noise decoupling loss function and combine it with the standard noise prediction loss of the diffusion model to form a joint optimization objective. Train the model to make the generated results far away from the degradation mode in the feature space.
2. The image super-resolution method based on dynamic texture noise synthesis according to claim 1, characterized in that, The step S1, "extracting the local gradient statistical features of the low-resolution image," specifically refers to using a pre-trained gradient estimation network to calculate the gradient magnitude of each local region in the low-resolution image. G ( x , y and gradient direction θ ( x , y This forms a gradient statistical feature map; the gradient magnitude... G ( x , y and gradient direction θ ( x , y The calculation formulas for ) are as follows: ， in, This is a low-resolution image.
3. The image super-resolution method based on dynamic texture noise synthesis according to claim 2, characterized in that, The phrase "synthesizing texture noise adapted to image content through a conditional generation network based on the local gradient statistical features" in step S1 specifically refers to: using the gradient statistical feature map as a condition, a conditional generation network learns to convert the gradient statistical feature map into texture noise. mapping Generate texture noise aligned with the gradient statistical feature map space. This allows the spatial distribution of noise to match the local structure of the image.
4. The image super-resolution method based on dynamic texture noise synthesis according to claim 3, characterized in that, In step S2, "the texture noise and pure Gaussian noise are fused at a preset ratio to construct the initial inference noise." The calculation formula for this initial noise is as follows: ,in, For texture noise, It is pure Gaussian noise. This is the fusion coefficient.
5. The image super-resolution method based on dynamic texture noise synthesis according to claim 4, characterized in that, The step S3, "performing iterative denoising using a pre-trained diffusion model," follows a Markov chain denoising process. Step-by-step noise reduction output By the Output and noise prediction network The decision is made jointly, and the calculation formula is as follows: ,in These are scheduling parameters.
6. The image super-resolution method based on dynamic texture noise synthesis according to claim 5, characterized in that, The calculation formula for "introducing the noise decoupling loss function" mentioned in step S4 is as follows: ,in, For true high-resolution images, To Degraded reference image obtained after applying artificial degradation operations. This is a pre-trained feature extraction network.