Image denoising method based on optimal transmission constraint
By introducing optimal transport theory and an iterative residual update module, the image denoising method solves the problems of distribution consistency and structure preservation in the existing image denoising process, and achieves high-quality image denoising effect.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-02-26
- Publication Date
- 2026-05-15
AI Technical Summary
Existing image denoising methods struggle to simultaneously balance image distribution consistency and perceptual quality in their optimization objectives, leading to over-smoothing or structural distortion.
An image denoising framework based on optimal transmission theory is adopted. The optimal transmission loss is used to measure the distribution difference between the denoised image and the high-definition image. Combined with an encoder-decoder structure and an iterative residual update module, the image distribution consistency and structure preservation are achieved.
While suppressing noise, it preserves the structural and detailed features of the image, thus improving the denoising effect and perceived quality.
Smart Images
Figure CN122048713A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, and in particular relates to an image denoising correlation method based on deep learning. Background Technology
[0002] In recent years, optimal transport theory (OT), as an important tool connecting mathematics, statistics, and computer science, has experienced rapid development and widespread application. Originally proposed by Monge and further generalized by Kantorovich through the introduction of a linear programming framework, it laid the foundation for modern optimal transport theory. Its core idea is to find an optimal way to map one probability distribution to another by minimizing transport costs, given a cost function.
[0003] In the field of statistics, optimal transport has been systematically studied in two main scenarios: on the one hand, optimal transport (OT) is considered a core estimation problem, such as estimating the Wasserstein distance between two distributions using finite samples, or further recovering their corresponding optimal transport mappings; on the other hand, OT is an analytical tool used to construct and interpret statistical models, showing significant advantages, especially in complex data structures where distributions serve as the basic unit of analysis. These studies demonstrate that OT can not only measure distributional differences but also explicitly characterize the geometric structure in the data space.
[0004] Based on the above characteristics, the connection between optimal transmission theory and image denoising has gradually attracted attention in recent years. Traditional image denoising methods mainly rely on regularization techniques, filter design or deep learning models, which can suppress noise interference to a certain extent, but still have obvious limitations: (1) Many methods rely on specific noise model assumptions, and their performance degrades under complex or unknown noise conditions; (2) Most methods focus on local pixel relationships and are difficult to characterize the image structure at the global level; (3) In terms of high-frequency details and structure preservation, a trade-off between smoothness and fidelity is often required.
[0005] In contrast, optimal transport-based image denoising offers a novel perspective beyond traditional pixel-level modeling. By viewing an image as a spatial distribution of pixel quality, OT (Optimal Transport Mapping) can model the overall distributional differences between noisy and target images while considering geometric structures. Within this framework, the image denoising problem can be viewed as finding the optimal transport mapping between the noise and noise-free distributions, thereby better preserving the structural and detailed features of the image while suppressing noise, showing potential advantages, especially in high-frequency noise and structure transfer scenarios.
[0006] Numerous studies have focused on image denoising, with methods broadly categorized into traditional model-based approaches and data-driven learning methods. On one hand, traditional model-based methods do not rely on large-scale training data but instead utilize prior assumptions about the signal and statistical properties of noise, such as sparsity, low rank, smoothness, and self-similarity. Typical methods include Gaussian filtering, median filtering, and bilateral filtering, which suppress noise through local or weighted averaging. Nonlocal means (NLM) methods improve denoising performance by weighted estimation of similar image patches, while BM3D achieves high-quality denoising through collaborative filtering of similar patches. However, the performance of these methods is highly dependent on manually designed priors and hyperparameter selection, resulting in limited generalization ability in complex noisy scenarios. On the other hand, with the development of deep learning, learning-based image denoising methods have made significant progress. Deep image priors (DIP) treat the neural network structure itself as an implicit prior; the Noise-to-Noise (N2N) method has proven that, under zero-mean noise conditions, it can achieve performance comparable to supervised learning using only "noise-noise" image pairs and has been extended to video denoising tasks. When paired noisy images are unavailable, methods such as Noisy-as-clean and GAN2GAN achieve unsupervised training by constructing synthetic noise or learning the noise distribution. Furthermore, denoising methods based on unbiased risk estimation (SURE) also demonstrate good performance under the assumption of independent and identically distributed Gaussian noise.
[0007] Although the above methods have been successful under different settings, the existing denoising paradigm still has fundamental flaws in terms of optimization objectives.
[0008] For supervised methods that rely on "noise-clean" image pairs, commonly used pixel-level losses primarily focus on minimizing the pointwise error between the denoised result and the corresponding clean image. This fails to capture the differences in their overall distribution, often leading to oversmoothing and decreased perceptual quality. Conversely, for unsupervised methods that do not rely on paired data, models typically constrain the output through distribution matching or statistical consistency, but lack direct constraints on the fidelity between the denoised result and the input observations, potentially introducing structural distortion or detail shift. Therefore, how to simultaneously maintain image distribution consistency during denoising has become one of the key challenges in current image denoising research.
[0009] Based on the above analysis, this paper proposes an image denoising framework based on optimal transmission theory, which constrains the gap between the denoising result and the input observation, as well as the consistency between the denoised image distribution and the high-quality image distribution, thereby achieving high-quality image denoising that balances structure preservation and perceptual quality. Summary of the Invention
[0010] The purpose of this invention is to provide an image denoising method based on optimal transmission constraints to overcome the problem that the consistency of image distribution cannot be taken into account simultaneously during the denoising process.
[0011] To achieve the above objectives, the present invention provides an image denoising method based on optimal transmission constraints, comprising:
[0012] Step 1: Perform the necessary image preprocessing on the input image dataset.
[0013] This dataset uses high-quality images from a specified folder as its data source. During the loading phase, the images are first read and uniformly converted to RGB format. The images are then uniformly resized to 256×256 resolution and converted to tensor format (C,H,W), where C represents the channel dimension, H represents the height dimension, and W represents the width dimension. If no transformation method is explicitly specified, such as vertical or horizontal flipping, no change is made by default. For the processed image tensors, if Gaussian noise with variances of 15, 25, and 50 is being processed, Gaussian noise with a mean of 0 and a standard deviation of 15, 25, and 50 is added to the tensor respectively. If real-world Gaussian noise is being processed, a random Gaussian noise with a mean of 0 and a standard deviation between 15 and 50 is added to each batch.
[0014] Step 2: Design the denoising neural network, whose structure mainly consists of three modules:
[0015] Modeling based on optimal transmission theory: The optimal transmission theory is fused with traditional image denoising methods to construct a new denoising objective. This objective minimizes the pixel-level differences between the denoised image and its corresponding high-resolution image, while also minimizing the differences between the distribution of the denoised image and its corresponding high-resolution image.
[0016] The denoising and reconstruction module employs a denoising network based on an encoder-decoder structure as the generator. The network first maps the input noisy image to a high-dimensional feature space using a shallow feature extraction module. Then, it utilizes a multi-scale encoder-decoder structure to extract and fuse feature representations at different resolutions layer by layer. To enhance feature representation, a channel-space joint attention mechanism is introduced at each layer to adaptively emphasize feature responses more important for the denoising task. During the decoding stage, skip connections fuse high-resolution structural information with deep semantic features, effectively restoring image details. Finally, a supervised attention module (SAM) generates intermediate denoising results, and an iterative residual update module further refines the reconstruction results, achieving a balance between noise suppression and structure preservation.
[0017] Denoising Network Design Based on Image Residual Iterative Optimization: To further eliminate residual noise and refine the reconstruction results, an iterative residual update module is introduced based on the initially denoised image. This module explicitly models the residual information between adjacent iteration results and uses gated recurrent units (GRUs) to model the residual evolution process, thereby gradually compensating for denoising errors in multiple iterations and achieving refined denoising while preserving the structure.
[0018] Step 3: Train the neural network. The specific method is as follows:
[0019] The preprocessed noisy image is input into the neural network and passed through the denoising and reconstruction module and the residual iterative optimization module in sequence to obtain the denoised image;
[0020] The model is constructed based on the optimal transmission theory to calculate the error between the denoised network output and its corresponding high-definition image, as well as the combined error between the mean square error of the denoised image and the corresponding high-definition image.
[0021] The error formula is as follows:
[0022]
[0023] in A clean image. This represents the image after denoising. This represents the mean square error between the denoised image and its corresponding high-resolution image. To balance the weights of fidelity and distributional differences, express and The sinkhorn distance between distributions.
[0024] Step 4: Repeat step 3 until the number of iterations reaches the preset limit; finally, save the model parameters as a pre-trained model file.
[0025] Step 5: Load the saved neural network parameters, take a new blurred image, perform the same preprocessing as in Step 1, input it into the neural network, and output a denoised image. Attached Figure Description
[0026] Figure 1 Flowchart of an image denoising method based on optimal transmission constraints
[0027] Figure 2 Schematic diagram of an image denoising method based on optimal transmission constraints Detailed Implementation
[0028] To more clearly illustrate the image denoising method based on optimal transmission constraints of the present invention, the specific real-time mode of the present invention will be described in detail below with reference to the embodiments. This embodiment is implemented under the premise of the technical solution of the present invention, and provides detailed implementation methods and specific operation processes, but the protection scope of the present invention is not limited to the following embodiments.
[0029] Step 1: Image dataset preprocessing and noise addition
[0030] This dataset uses high-quality images from a specified folder as its data source. During the loading phase, the images are first read and uniformly converted to RGB format. The images are then uniformly resized to 256×256 resolution and converted to tensor format (C,H,W), where C represents the channel dimension, H represents the height dimension, and W represents the width dimension. If no transformation method is explicitly specified, such as vertical or horizontal flipping, no change is made by default. For the processed image tensors, if Gaussian noise with variances of 15, 25, and 50 is being processed, Gaussian noise with a mean of 0 and a standard deviation of 15, 25, and 50 is added to the tensor respectively. If real-world Gaussian noise is being processed, a random Gaussian noise with a mean of 0 and a standard deviation between 15 and 50 is added to each batch.
[0031] Step 2: Building a Denoising Neural Network Model
[0032] When a blurred image is input into a deep learning model, the neural network mainly consists of three parts: modeling based on optimal transport theory, a denoising and reconstruction module, and a denoising network design based on iterative optimization of image residuals. The structure of each module is described in detail below:
[0033] 1) Modeling Based on Optimal Transmission Theory: This method integrates optimal transmission theory with traditional image denoising methods. While traditional pixel-level mean square loss-based image denoising methods perform well in noise suppression, they have inherent limitations: they only minimize the point-by-point differences between the denoised image and its corresponding high-resolution image in pixel space, ignoring the overall image distribution structure and higher-order statistical information. This often leads to over-smoothing of the denoising results, resulting in loss of texture and detail, especially in complex noisy scenes where pixel-level constraints cannot guarantee that the distribution of the denoised image is consistent with that of the high-resolution image. To overcome this deficiency, optimal transmission theory is introduced to simultaneously model pixel-level errors and image distribution-level differences. Specifically, based on traditional mean square loss, optimal transmission loss is used to measure the distance between the distribution of the denoised image and the high-resolution image, thus enabling the network to maintain global statistical characteristics and texture structure while ensuring pixel-level reconstruction accuracy. This method naturally unifies pixel-level fidelity and distribution-level consistency, achieving high-fidelity, texture-rich, and structurally stable image denoising. It minimizes the pixel-level differences between the denoised image and its corresponding high-resolution image while also minimizing the differences between the distribution of the denoised image and its corresponding high-resolution image.
[0034] 2) Denoising and Reconstruction Module: This denoising network adopts an encoder-decoder framework and introduces multiple attention mechanisms in the feature extraction, cross-scale fusion, and image reconstruction stages to improve the modeling ability for noise suppression and structure restoration. In the shallow stage, the input noisy image first undergoes initial feature extraction through convolution and channel and spatial attention blocks to obtain a low-level representation containing local texture and edge information. Subsequently, the features are fed into a multi-scale encoder structure. The encoder consists of multiple layers. Each layer gradually reduces the spatial resolution and increases the channel dimension through downsampling operations, thereby expanding the receptive field and capturing richer contextual information. Each encoding layer consists of multiple channels and spatial attention blocks, combining channel attention and spatial attention mechanisms to adaptively adjust the importance of different feature channels and spatial locations. The decoder structure is symmetrical to the encoder, restoring spatial resolution through layer-by-layer upsampling and using skip connections to fuse high-resolution features from the encoding stage with features from the decoding stage. To avoid interference from irrelevant features, an attention module is introduced before the skip connections to selectively enhance features, thereby more effectively preserving structural and detailed information. In the feature reconstruction stage, the network employs a supervised attention module (SAM), which generates intermediate denoising results by mapping decoded features to image space and forming residual connections with the input image. SAM also uses image-level supervision signals to guide feature learning in reverse, making the network pay more attention to regions that contribute to the reconstruction quality. Finally, to further improve the denoising effect, the network introduces an iterative residual update module to refine and update the initial denoising results in multiple steps, thereby gradually eliminating residual noise and enhancing the consistency of image structure. This overall design effectively avoids over-smoothing while ensuring noise suppression capabilities.
[0035] 3) Denoising Network Design Based on Image Residual Iterative Optimization: This module aims to iteratively refine the initial denoising results to further suppress residual noise and enhance structural consistency. Given an initial denoised image, the module iteratively updates the denoising results through multiple iterations. In the t-th iteration, the residual between the current estimate and the result of the previous iteration is first calculated. ,in Let represent the residual of the t-th iteration. This represents the denoising result of the t-th iteration. This represents the denoising result of the (t-1)th iteration.
[0036] The residual explicitly characterizes the changing trend during the denoising process. Subsequently, the residual image is fed into a lightweight convolutional encoder to extract local structural features, which are then mapped to the hidden state space of the GRU via linear projection. To reduce computational complexity and aggregate global contextual information, the encoded features undergo global average pooling in the spatial dimension and are used as input to drive the update of the GRU hidden state. In this way, the GRU can implicitly model the temporal dependencies of residual evolution during iteration. Based on the updated hidden state, the module generates an image domain compensation term through a residual prediction head and adds it to the current estimate to obtain a new denoising result. ,in This represents the compensation term in the image domain generated by the residual prediction head. Through multiple iterations, this module can gradually correct the systematic errors in the initial denoising results, effectively suppressing noise while avoiding over-smoothing, thereby improving the quality and structural consistency of the final denoised image.
[0037] Step 3: Training the denoising neural network model
[0038] The specific methods for training a neural network are as follows:
[0039] First, the blurred image preprocessed in step 1 is input into the neural network, and a preliminary denoised image is obtained through the denoising and reconstruction module.
[0040] Then, through a denoising network based on image residual iterative optimization, the final denoised image with residual compensation is obtained by continuous iterative optimization using GRU.
[0041] Finally, a model based on optimal transmission theory is used to calculate the combined error between the denoised image output by the network and its corresponding high-resolution image, as well as the mean square error between the denoised image and its corresponding high-resolution image. The error values are then calculated, and based on these error values, the RMSprop optimization algorithm is used to calculate the gradient of the loss function with respect to the neural network parameters. Using this gradient information, the network parameters are gradually adjusted to minimize the loss function value.
[0042] Step 4: Iterative model training and parameter saving
[0043] Repeat step 3 until the number of iterations reaches the preset limit; finally, save the model parameters as a pre-trained model file.
[0044] Step 5: Loading the pre-trained model and image denoising inference
[0045] Load the saved neural network parameters, take a new blurred image, perform the same preprocessing as in step one, input it into the neural network, and output a denoised image.
[0046] Through the above implementation methods, the present invention realizes an image denoising method based on optimal transmission constraints, which effectively solves the problem of focusing on pixel-level differences in denoised images without taking into account the differences between the distributions of denoised images.
[0047] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. An image denoising method based on optimal transmission constraints, comprising the following steps: Step 1: Perform the necessary image preprocessing on the input image dataset; By changing the image size, normalizing, and adding different amounts of noise to the high-resolution image, noise-high-resolution image pairs can be formed; Step 2: Design the denoising neural network, whose structure mainly consists of three modules: Modeling based on optimal transmission theory: The optimal transmission theory is fused with traditional image denoising methods to construct a new denoising objective. This objective minimizes the pixel-level differences between the denoised image and its corresponding high-resolution image, while also minimizing the differences between the distribution of the denoised image and its corresponding high-resolution image. The denoising and reconstruction module employs a denoising network based on an encoder-decoder structure as the generator. The network first maps the input noisy image to a high-dimensional feature space using a shallow feature extraction module. Then, it utilizes a multi-scale encoder-decoder structure to extract and fuse feature representations at different resolutions layer by layer. To enhance feature representation capabilities, a channel-space joint attention mechanism is introduced at each layer to adaptively emphasize feature responses more important for the denoising task. During the decoding stage, skip connections fuse high-resolution structural information with deep semantic features, effectively restoring image details. Finally, a supervised attention module (SAM) generates intermediate denoising results, and an iterative residual update module further refines the reconstruction results, achieving a balance between noise suppression and structure preservation. Denoising Network Design Based on Image Residual Iterative Optimization: To further eliminate residual noise and refine the reconstruction results, an iterative residual update module is introduced based on the initially denoised image. This module explicitly models the residual information between adjacent iteration results and uses gated recurrent units (GRUs) to model the residual evolution process, thereby gradually compensating for denoising errors in multiple iterations and achieving refined denoising while preserving the structure. Step 3: Train the neural network. The specific method is as follows: The preprocessed noisy image is input into the neural network and passed through the denoising and reconstruction module and the residual iterative optimization module in sequence to obtain the denoised image; The model is constructed based on optimal transmission theory to calculate the error between the denoised network output and its corresponding high-definition image, as well as the combined error between the mean square error of the denoised image and the corresponding high-definition image. The error formula is as follows: ,in A clean image. This represents the image after denoising. This represents the mean square error between the denoised image and its corresponding high-resolution image. To balance the weights of fidelity and distributional differences, express and Sinkhorn distance between distributions; Step 4: Repeat step 3 until the number of iterations reaches the preset limit. Finally, save the model parameters as a pre-trained model file. Step 5: Load the saved neural network parameters, take a new blurred image, perform the same preprocessing as in Step 1, input it into the neural network, and output a denoised image.
2. The image denoising method based on optimal transmission constraints according to claim 1, characterized in that, In step one, the image preprocessing includes the following specific steps: The dataset uses high-quality images from a specified folder as the data source. During the loading stage, these images are first read and uniformly converted to RGB format. The images are uniformly adjusted to a resolution of 256×256 and converted to tensor format (C,H,W), where C represents the channel dimension, H represents the height dimension, and W represents the width dimension. If no transformation method is explicitly specified, such as vertical or horizontal flipping, no change is made by default. For the processed image tensor, if Gaussian noise with variances of 15, 25, and 50 is processed, Gaussian noise with a mean of 0 and a standard deviation of 15, 25, and 50 is added to the tensor respectively. If real-world Gaussian noise is processed, a random Gaussian noise with a mean of 0 and a standard deviation between 15 and 50 is added to each batch, generating noise-high-resolution image pairs.
3. The image denoising method based on optimal transmission constraints according to claim 1, characterized in that, In step two, the channel-space joint attention mechanism in the denoising and reconstruction module adaptively emphasizes feature responses that play a crucial role in image denoising, achieving a balance between noise suppression and image structure preservation. The encoder-decoder structure of the denoising and reconstruction module is multi-scale, capable of extracting image features at different resolutions layer by layer and performing feature fusion, effectively restoring image detail. Skip connections fuse the high-resolution structural information extracted by the encoder with the deep semantic features of the decoder during the decoding stage, preventing the loss of image details during feature extraction.
4. The image denoising method based on optimal transmission constraints according to claim 1, characterized in that, In step two, the denoising network module based on image residual iterative optimization operates on the basis of the initially denoised image, and eliminates residual noise through multiple iterations to achieve refined denoising that preserves the image structure.
5. The image denoising method based on optimal transmission constraints according to claim 1, characterized in that, In step three, the combination error is obtained by weighted summation of the mean square error and the Sinkhorn distance, using weights... Adjustable proportions of pixel-level consistency and distribution-level consistency in model training.