Image noise reduction method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of computer vision, in particular to an image noise reduction method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring image data to be denoised, and preprocessing the image data to be denoised to obtain preprocessed image data; the preprocessed image data is input to a preset image noise reduction model, a final noise reduction image is obtained, the preset image noise reduction model is obtained based on a gated texture convolutional neural network architecture and through training by using a preset joint loss function, and the final noise reduction image is obtained. The preset joint loss function comprises a weighted sum of a mean square error loss item and a learning perception image fragment similarity loss item. Therefore, the lightweight gated texture convolutional neural network is adopted as a noise reduction core architecture, and model optimization is performed in combination with the joint loss function fusing the pixel-level fidelity and the perception similarity, so that the problems of excessive smoothness of the image texture and reduction of the perception quality caused by a noise reduction method based on mean square error loss are solved.

Description

Image noise reduction methods, apparatus, electronic devices and storage media Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to an image noise reduction method, apparatus, electronic device and storage medium. Background Technology

[0002] In the fields of computer vision and digital image processing, image quality is a key factor affecting the performance of subsequent analysis, recognition, and understanding tasks. However, in actual imaging processes, images are often introduced with noise due to factors such as equipment limitations, insufficient lighting, and signal transmission, leading to a significant deterioration in image quality. Image denoising, as a fundamental image restoration task, aims to reconstruct the original clear image as accurately as possible from noisy observation images, and is of paramount importance for improving the robustness and accuracy of vision systems.

[0003] In related technologies, mainstream methods can be broadly divided into two categories: one is the single-scale method represented by denoising convolutional neural networks; the other is the multi-scale encoder-decoder method represented by U-Net (Convolutional Networks for Biomedical Image Segmentation, U-shaped network). However, when processing images with high noise levels, the output of the above methods often loses detail and texture, exhibiting an overly smooth visual defect, which degrades the perceptual quality and urgently needs to be addressed. Summary of the Invention

[0004] This application provides an image denoising method, apparatus, electronic device, and storage medium to solve the problem of excessively smoothed image texture and decreased perceptual quality caused by denoising methods based on mean square error loss, so as to generate denoised images with rich details and excellent visual perceptual quality under an efficient computing framework.

[0005] To achieve the above objectives, a first aspect of this application proposes an image denoising method, comprising the following steps: acquiring image data to be denoised, and preprocessing the image data to be denoised to obtain preprocessed image data; inputting the preprocessed image data into a preset image denoising model to obtain a final denoised image, wherein the preset image denoising model is based on a gated texture convolutional neural network architecture and is trained using a preset joint loss function, the preset joint loss function comprising a weighted sum of a mean squared error loss term and a learned perceptual image segment similarity loss term.

[0006] According to one embodiment of this application, before inputting the preprocessed image data into the preset image denoising model to obtain the final denoised image, the method further includes: acquiring an original image dataset and a noisy image dataset corresponding to the original image data, constructing a training dataset based on the original image dataset and the noisy image dataset, preprocessing the training dataset to obtain a preprocessed training dataset; dividing the preprocessed training dataset into a training set, a validation set, and a test set based on a preset partitioning ratio; constructing a target neural network based on the gated texture convolutional neural network architecture, and inputting the training set into the target neural network for training to obtain initial model parameters; based on the initial model parameters, inputting the validation set into the target neural network for performance evaluation, and adjusting the initial model parameters according to the performance evaluation results until the preset joint loss function of the validation set converges to obtain optimal model parameters; based on the optimal model parameters, inputting the test set into the target neural network for model testing, and obtaining the preset image denoising model if the test results meet preset requirements.

[0007] According to one embodiment of this application, the step of inputting the preprocessed image data into a preset image denoising model to obtain a final denoised image includes: using the multi-scale context extraction module of the preset image denoising model to perform multi-level downsampling and upsampling processing on the preprocessed image data to extract contextual structure feature information of the preprocessed image data; using the single-scale noise estimation module of the preset image denoising model to perform convolution processing on the preprocessed image data to obtain a preliminary noise distribution estimation result of the preprocessed image data; converting the contextual structure feature information into modulation coefficients through the gated feature fusion module of the preset image denoising model; and performing weighted modulation processing on the preliminary noise distribution estimation result based on the modulation coefficients to generate the final denoised image.

[0008] According to an embodiment of this application, the above-described image denoising method further includes: calculating the pixel-level difference between the current denoised image output by the preset image denoising model and the original image corresponding to the current denoised image to obtain the mean square error loss term; and obtaining the learning-aware image segment similarity loss term based on the distance between the current denoised image output by the preset image denoising model and the original image corresponding to the current denoised image in a multi-layer deep feature space.

[0009] According to one embodiment of this application, the step of obtaining the learning-aware image segment similarity loss term by measuring the distance between the current denoised image and the original image corresponding to the current denoised image in a multi-layer deep feature space based on the preset image denoising model includes: inputting the current denoised image and the original image corresponding to the current denoised image into a preset deep convolutional network to obtain multiple intermediate layer feature map pairs, wherein the feature map pair includes a first feature map corresponding to the current denoised image and a second feature map corresponding to the original image corresponding to the current denoised image; calculating the normalized Euclidean distance of each intermediate layer feature map pair, and weighting and summing each normalized Euclidean distance to obtain the learning-aware image segment similarity loss term.

[0010] The image denoising method proposed in this application involves acquiring image data to be denoised and preprocessing it to obtain preprocessed image data. The preprocessed image data is then input into a preset image denoising model to obtain the final denoised image. The preset image denoising model is based on a gated texture convolutional neural network architecture and is trained using a preset joint loss function. This preset joint loss function includes a weighted sum of a mean squared error loss term and a loss term for learning perceptual image segment similarity. Therefore, by employing a lightweight gated texture convolutional neural network as the core architecture for denoising and combining it with a joint loss function that fuses pixel-level fidelity and perceptual similarity for model optimization, the problem of excessively smoothed image textures and decreased perceptual quality caused by denoising methods based on mean squared error loss is solved. This allows for the generation of denoised images with rich details and excellent visual perceptual quality within an efficient computational framework.

[0011] To achieve the above objectives, a second aspect of this application provides an image denoising apparatus, comprising: a processing module for acquiring image data to be denoised and preprocessing the image data to be denoised to obtain preprocessed image data; and a first obtaining module for inputting the preprocessed image data into a preset image denoising model to obtain a final denoised image, wherein the preset image denoising model is based on a gated texture convolutional neural network architecture and is trained using a preset joint loss function, the preset joint loss function comprising a weighted sum of a mean squared error loss term and a learned perceptual image segment similarity loss term.

[0012] According to one embodiment of this application, before inputting the preprocessed image data into the preset image denoising model to obtain the final denoised image, the first obtaining module is further configured to: obtain an original image dataset and a noisy image dataset corresponding to the original image data, and construct a training dataset based on the original image dataset and the noisy image dataset; preprocess the training dataset to obtain a preprocessed training dataset; divide the preprocessed training dataset into a training set, a validation set, and a test set based on a preset partitioning ratio; construct a target neural network based on the gated texture convolutional neural network architecture, and input the training set into the target neural network for training to obtain initial model parameters; input the validation set into the target neural network for performance evaluation based on the initial model parameters, and adjust the initial model parameters according to the performance evaluation results until the preset joint loss function of the validation set converges to obtain optimal model parameters; input the test set into the target neural network for model testing based on the optimal model parameters, and obtain the preset image denoising model if the test results meet preset requirements.

[0013] According to one embodiment of this application, the first obtaining module is specifically used for: performing multi-level downsampling and upsampling processing on the preprocessed image data using the multi-scale context extraction module of the preset image denoising model to extract the contextual structure feature information of the preprocessed image data; performing convolution processing on the preprocessed image data using the single-scale noise estimation module of the preset image denoising model to obtain a preliminary noise distribution estimation result of the preprocessed image data; converting the contextual structure feature information into modulation coefficients through the gated feature fusion module of the preset image denoising model; and performing weighted modulation processing on the preliminary noise distribution estimation result based on the modulation coefficients to generate the final denoised image.

[0014] According to one embodiment of this application, the above-described image denoising device further includes: a calculation module, configured to calculate the pixel-level difference between the current denoised image output by the preset image denoising model and the original image corresponding to the current denoised image, to obtain the mean square error loss term; and a second obtaining module, configured to obtain the learning-perceptual image segment similarity loss term based on the distance between the current denoised image output by the preset image denoising model and the original image corresponding to the current denoised image in a multi-layer deep feature space.

[0015] According to one embodiment of this application, the second obtaining module is specifically used for: inputting the current denoised image and the original image corresponding to the current denoised image into a preset deep convolutional network to obtain multiple feature map pairs of intermediate layers, wherein the feature map pair includes a first feature map corresponding to the current denoised image and a second feature map corresponding to the original image corresponding to the current denoised image; calculating the normalized Euclidean distance of each feature map pair of intermediate layers, and weighting and summing each normalized Euclidean distance to obtain the learning-aware image segment similarity loss term.

[0016] The image denoising apparatus proposed in this application acquires image data to be denoised and preprocesses it to obtain preprocessed image data. The preprocessed image data is then input into a preset image denoising model to obtain the final denoised image. The preset image denoising model is based on a gated texture convolutional neural network architecture and is trained using a preset joint loss function. This preset joint loss function includes a weighted sum of a mean squared error loss term and a loss term for learning perceptual image segment similarity. Therefore, by employing a lightweight gated texture convolutional neural network as the core architecture for denoising and combining it with a joint loss function that fuses pixel-level fidelity and perceptual similarity for model optimization, the problem of excessively smoothed image textures and decreased perceptual quality caused by denoising methods based on mean squared error loss is solved. This allows for the generation of denoised images with rich details and excellent visual perceptual quality within an efficient computing framework.

[0017] To achieve the above objectives, a third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image noise reduction method as described in the above embodiments.

[0018] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the image noise reduction method as described in the above embodiments.

[0019] To achieve the above objectives, a fifth aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, is used to implement the image noise reduction method as described in the above embodiments.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which: FIG1 is a flowchart of an image denoising method provided according to an embodiment of this application; FIG2 is a visual comparison diagram of the denoising effect of a texture image according to an embodiment of this application, wherein FIG2(a) is a schematic diagram of the original image, FIG2(b) is a schematic diagram of the denoising effect of the U-net method on the texture image, and FIG2(c) is a schematic diagram of the denoising effect of the method of this application on the texture image.

[0022] Figure 3 is a block diagram of an image noise reduction device provided according to an embodiment of this application; Figure 4 is a structural diagram of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0023] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0024] The image denoising method, apparatus, electronic device, and storage medium according to embodiments of this application will now be described with reference to the accompanying drawings. First, the image denoising method according to embodiments of this application will be described with reference to the accompanying drawings.

[0025] Figure 1 is a flowchart of an image noise reduction method according to an embodiment of this application.

[0026] As shown in Figure 1, the image denoising method includes the following steps: In step S101, the image data to be denoised is acquired, and the image data to be denoised is preprocessed to obtain the preprocessed image data.

[0027] It is understandable that the image data to be denoised refers to image information represented in the form of a digital matrix. For a grayscale image, it can be a two-dimensional matrix, where the value of each matrix element (pixel) represents the brightness (grayscale value) of that point; for a color image, it can be composed of three two-dimensional matrices representing the red, green, and blue color channels respectively, or it can use other color space representation methods.

[0028] Specifically, the process begins by reading the original image to be processed (i.e., the image data to be denoised) from the storage unit or real-time data stream of the image acquisition device (such as a digital camera, medical imaging device, surveillance camera, etc.). This image typically contains various types of noise (such as Gaussian noise, Poisson noise, etc.) due to imaging limitations. Subsequently, a series of standardized preprocessing operations can be performed on the acquired image data to be denoised, aiming to transform the image data into a standardized format that meets the input requirements of the subsequent neural network model. The preprocessing process includes, but is not limited to: image size normalization (e.g., scaling or cropping the image to a fixed resolution set during model training, such as 256×256 pixels); pixel value normalization (e.g., linearly mapping the original 8-bit integer pixel value range [0,255] to the floating-point range [0.0,1.0] or performing zero-centering); color space conversion (e.g., converting an RGB (Red Green Blue) color image to a YUV (Luminance Bandwidth Chrominance) color space and processing only the luminance channel Y, or directly processing a grayscale image). The purpose of preprocessing is to eliminate scale differences and format inconsistencies in the input data, thereby laying the foundation for efficient and accurate noise reduction processing in the future.

[0029] In step S102, the preprocessed image data is input into a preset image denoising model to obtain the final denoised image. The preset image denoising model is based on a gated texture convolutional neural network architecture and is trained using a preset joint loss function. The preset joint loss function includes a weighted sum of a mean squared error loss term and a learning-aware image segment similarity loss term.

[0030] Understandably, a gated texture convolutional neural network (GNN) is a neural network architecture specifically designed for image restoration tasks. Its core lies in the introduction of a gating mechanism, which can be understood as an adaptive, data-dependent weight adjuster. It utilizes one branch network (a multi-scale network) to analyze the contextual structure information of the input image (such as edges, texture direction, and region categories) and generates a control signal (gating map) to dynamically modulate the feature map or activation values ​​of another branch network (a single-scale denoising network), thereby achieving content-aware denoising—that is, stronger denoising in smooth regions and more conservative denoising in detailed regions to preserve texture. Mean squared error loss refers to a regression task loss function that calculates the mean of the squares of the differences between the model's predicted and true values. It focuses on pixel-level absolute accuracy, and optimizing this loss can directly improve objective metrics such as peak signal-to-noise ratio (PSNR). Learning-aware image fragment similarity loss refers to a deep learning-based image-aware similarity metric that can be used to calculate the distance between two images in the feature space of a deep convolutional neural network. It does not directly compare pixel values ​​but rather compares the feature representations of the images after activation in multiple convolutional layers. Optimizing this loss means forcing the denoised image to approach a real, clear image at the level of "looks like" (i.e., high-level semantic features such as texture and structure), thereby effectively preserving visual details and realism and avoiding over-smoothing.

[0031] Specifically, the image data matrix standardized in step S101 (i.e., the preprocessed image data) is used as the input tensor and fed into a pre-trained image denoising model with fixed parameters (i.e., the preset image denoising model). This model is built on a gated texture convolutional neural network architecture, which internally uses a specific gating mechanism to deeply fuse and coordinate the multi-scale sub-network responsible for extracting global contextual structure information of the image with the single-scale sub-network responsible for local noise estimation and suppression. When processing the input data, the preset image denoising model can dynamically and adaptively adjust the denoising intensity according to the structural complexity of each region of the image, thereby effectively removing noise in smooth areas while preserving more details in texture-rich edge areas. After the model is processed, it can directly output an image data matrix of the same size as the input, i.e., the final denoised image. The superior performance of this preset image denoising model stems from its unique training method, namely, optimization by minimizing a preset joint loss function. This preset joint loss function is not a single index, but a weighted linear combination of the mean squared error loss term and the learned perceptual image segment similarity loss term. The former can force the model to pursue pixel-level numerical accuracy restoration to ensure the overall signal-to-noise ratio; the latter can constrain the model in the deep feature space from the perspective of human visual perception, so that the output image is highly similar to the real clear image in terms of texture, structure and other semantic levels. The two work together to effectively solve the common problem of over-smoothing and achieve high-perceptual-quality image denoising.

[0032] Next, we will explain in detail how to obtain the final denoised image.

[0033] As one possible implementation, in some embodiments, the preprocessed image data is input into a preset image denoising model to obtain the final denoised image, including: using the multi-scale context extraction module of the preset image denoising model to perform multi-level downsampling and upsampling processing on the preprocessed image data to extract the contextual structure feature information of the preprocessed image data; using the single-scale noise estimation module of the preset image denoising model to perform convolution processing on the preprocessed image data to obtain a preliminary noise distribution estimation result of the preprocessed image data; converting the contextual structure feature information into modulation coefficients through the gated feature fusion module of the preset image denoising model; and performing weighted modulation processing on the preliminary noise distribution estimation result based on the modulation coefficients to generate the final denoised image.

[0034] Specifically, the pre-defined image denoising model mainly consists of three core sub-modules: a multi-scale context extraction module, a single-scale noise estimation module, and a gated feature fusion module. The multi-scale context extraction module can be a symmetrical encoder-decoder structure, responsible for capturing the global semantic and structural context information of the input image. The encoder can contain three levels of downsampling, each level consisting of two convolutional layers (e.g., 3×3 kernel, stride 1, padding 1) and a max-pooling layer (e.g., 2×2 pooling kernel, stride 2) connected sequentially. The number of channels in the output feature maps of the three levels can be 32, 64, and 128, respectively, with the spatial size halved sequentially. The decoder can contain three levels of upsampling, symmetrical to the encoder. Each level first performs upsampling through a transposed convolutional layer (e.g., 2×2 kernel, stride 2), then concatenates the upsampled feature map with the corresponding level's feature map from the encoder, and finally processes it through two convolutional layers (e.g., 3×3 kernel, stride 1, padding 1). The final output context feature map (i.e., contextual structure feature information) of the decoder has the same size as the input image and 32 channels. The single-scale noise estimation module can be a deep residual network responsible for estimating the noise distribution pixel by pixel. This module can be composed of 15 identical residual blocks cascaded together. Each residual block can contain two convolutional layers (e.g., 3×3 kernel, stride 1, padding 1), with ReLU (Rectified Linear Unit) activation functions used between layers and a skip connection with an identity mapping. The number of input and output channels for all residual blocks can be fixed at 64, so the spatial resolution of the feature map remains unchanged throughout the single-scale noise estimation module. The intermediate layer feature map of this module (i.e., the preliminary noise distribution estimation result), for example, the output of the 8th residual block, can be fed into the gated feature fusion module for modulation. The gated feature fusion module can achieve adaptive guidance of the denoising process by contextual information. First, the feature map output by the multi-scale context extraction module (e.g., size H×W×32) can be fed into a gated generation unit. This unit first uses a 1×1 convolutional layer to compress the number of channels to 1, obtaining a single-channel feature map. Then, it normalizes this map to the [0,1] interval using a sigmoid activation function, generating a spatially gated map (e.g., H×W×1 in size). Regions in the spatially gated map with values ​​close to 1 indicate complex textures or structures, requiring weakened noise suppression to preserve details; regions with values ​​close to 0 represent flat areas, which can be strongly denoised. Next, the generated spatially gated map is multiplied element-wise with the intermediate feature map (e.g., H×W×64 in size) from the single-scale noise estimation module, using a channel-wise broadcast multiplication (i.e., weighted modulation processing). That is, the spatially gated map is copied 64 times along the channel dimension and then multiplied element-wise with the corresponding elements of the intermediate feature map. This process yields the modulated feature map. The modulated feature map can then be fed back into the subsequent residual blocks of the single-scale noise estimation module for further processing. Finally, the last convolutional layer of this module (e.g., 3×3 kernel, stride 1, padding 1, output channels equal to the number of channels in the input image) outputs the predicted noise residual image. Subtracting the input noisy image from the predicted noise residual image yields the final denoised image.

[0035] Optionally, in some embodiments, before inputting the preprocessed image data into a preset image denoising model to obtain the final denoised image, the method further includes: acquiring an original image dataset and a noisy image dataset corresponding to the original image data, constructing a training dataset based on the original image dataset and the noisy image dataset, preprocessing the training dataset to obtain a preprocessed training dataset; dividing the preprocessed training dataset into a training set, a validation set, and a test set based on a preset partitioning ratio; constructing a target neural network based on a gated texture convolutional neural network architecture, and inputting the training set into the target neural network for training to obtain initial model parameters; based on the initial model parameters, inputting the validation set into the target neural network for performance evaluation, and adjusting the initial model parameters according to the performance evaluation results until the preset joint loss function of the validation set converges to obtain optimal model parameters; based on the optimal model parameters, inputting the test set into the target neural network for model testing, and obtaining a preset image denoising model if the test results meet preset requirements.

[0036] Specifically, in training the pre-defined image denoising model, a high-quality training dataset can be prepared first. This dataset can consist of pairs of original, clear images and their corresponding noisy images obtained by simulating the real noise generation process (such as adding Gaussian white noise with a specific variance). This pairing ensures that the model's learning objective is clear. Subsequently, the dataset undergoes preprocessing operations consistent with the inference stage (such as size normalization and pixel value standardization) to ensure consistency in the input distribution between training and application. Next is the data strategy stage, where, following statistical best practices, the entire dataset is divided into training, validation, and test sets according to a pre-defined ratio (e.g., 70%, 15%, 15%). The training set can be directly used to drive model learning; the validation set can be used to monitor the model's generalization ability during training and as a means to adjust hyperparameters and prevent overfitting; and the test set can ultimately evaluate the model's performance, and its evaluation results can be used to determine whether the model meets application standards.

[0037] Next, we can proceed to the model optimization phase. First, we can instantiate a target neural network based on a gated texture convolutional neural network architecture and initialize its weights. The training loop can be carried out on the training set: the predicted output is calculated via forward propagation; the difference between the predicted output and the real, clear image is quantified using the pre-defined joint loss function (a weighted sum of MSE (Mean Squared Error) and LPIPS (Learned Perceptual Image Patch Similarity)); then, the network parameters are updated via backpropagation and an optimizer (such as Adam (Adaptive Moment Estimation)). This process is iterated repeatedly. However, pursuing loss reduction solely on the training set may lead to overfitting, meaning the model only remembers the noisy patterns of the training samples. Therefore, after each training cycle, the current model can be evaluated using a validation set. The changing trends of validation set loss or related metrics (such as PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index Measure)) are the core signals guiding training: when validation set metrics stop improving or even begin to deteriorate, it means that the model performance has reached a bottleneck or begun to overfit, at which point an early stopping mechanism can be triggered or the learning rate adjusted. Through this training-validation feedback loop, the model parameters that perform best on the validation set are ultimately selected, i.e., the optimal model parameters.

[0038] Finally, in the model validation phase, to ensure the model's objective performance and generalization ability, a test set that has never participated in any parameter optimization process can be input into the model with optimal parameters for final testing. The model must meet preset performance requirements on the test set (e.g., average PSNR above 30dB, or LPIPS value below 0.15) to be confirmed as a qualified preset image denoising model, and then the parameters are solidified for subsequent deployment and inference applications.

[0039] Optionally, in some embodiments, the above-described image denoising method further includes: calculating the pixel-level difference between the current denoised image output by the preset image denoising model and the original image corresponding to the current denoised image to obtain a mean squared error loss term; and obtaining a learning-aware image segment similarity loss term based on the distance between the current denoised image output by the preset image denoising model and the original image corresponding to the current denoised image in a multi-layer deep feature space.

[0040] Specifically, the pre-defined joint loss function is not a single metric, but rather integrates two different dimensions of image similarity measurement (mean squared error loss and learned perceptual image segment similarity loss), aiming to simultaneously guide the model to pursue pixel-level accurate restoration and high-quality recovery at the human visual perception level. Its calculation process can be performed synchronously in each training iteration. For the calculation of the mean squared error loss term, firstly, the current model's predicted output for the noisy input image (i.e., the current denoised image) and its paired original clear image are obtained. Then, these two images are compared point-by-point in the pixel space; that is, the square of the difference between corresponding pixel values ​​is calculated, and the average of the squared differences of all pixels is taken. This loss term quantifies the overall deviation between the predicted image and the real image in terms of global brightness, contrast, and the grayscale value of each pixel. Minimizing the mean squared error loss can directly drive the model to improve traditional objective metrics such as peak signal-to-noise ratio, ensuring that the denoised image is highly faithful to the original signal at the signal level.

[0041] For calculating the loss term for learning to perceive image fragment similarity, the same pair of current denoised images and original sharp images can be input into a deep convolutional neural network (such as VGG (Visual Geometry Group)) with fixed parameters, pre-trained on a large-scale image classification dataset (such as ImageNet). In LPIPS computation, instead of using the network's final classification output, activation feature maps from multiple intermediate convolutional layers are extracted. Then, at each layer, the normalized Euclidean distance (or a similar distance metric) between the denoised image feature map and the sharp image feature map is calculated. Finally, the distances calculated from all these intermediate layers are weighted and summed. The LPIPS loss term thus measures the differences between two images in high-level visual attributes such as texture, structure, and semantic content. Optimizing this loss means forcing the denoised image to approximate the real sharp image at the level of "looking like" and "feeling like," thereby effectively preserving rich visual details and a natural feel, and avoiding overly smoothed, texture-deficient artificial artifacts.

[0042] Finally, by linearly combining the two losses using preset weighting coefficients, the total loss used for backpropagation can be formed. During the optimization process, the model is simultaneously constrained by both the pixel space and the perceptual space, thus learning a balance strategy: effectively suppressing noise while carefully protecting the image structure and texture details that are crucial to visual perception.

[0043] As one possible implementation, in some embodiments, the learning-aware image segment similarity loss term is obtained based on the distance between the current denoised image output by a preset image denoising model and the original image corresponding to the current denoised image in a multi-layer deep feature space. This includes: inputting the current denoised image and the original image corresponding to the current denoised image into a preset deep convolutional network to obtain multiple intermediate layer feature map pairs, wherein the feature map pair includes a first feature map corresponding to the current denoised image and a second feature map corresponding to the original image corresponding to the current denoised image; calculating the normalized Euclidean distance of each intermediate layer feature map pair, and weighting and summing each normalized Euclidean distance to obtain the learning-aware image segment similarity loss term.

[0044] Specifically, the core of learning to calculate the image fragment similarity loss term lies in utilizing a mature, pre-trained deep convolutional network to measure the difference in high-level visual features between the denoised image and the original sharp image. First, the current denoised image and its paired original image are simultaneously input into a pre-defined deep convolutional network (such as VGG). The network performs forward propagation, and for each selected intermediate layer, it outputs two feature maps: one from the denoised image (i.e., the first feature map) and the other from the original image (i.e., the second feature map). This pair of feature maps together constitutes a feature map pair, capturing visual patterns at a similar abstract level from their respective input images within the same network layer. For each feature map pair generated by each layer, the differences between them can be quantified. Directly calculating the Euclidean distance of the original feature maps is affected by the scale and magnitude of the feature vectors. Therefore, L2 normalization can be performed on each position of the feature map in the spatial dimension (i.e., each feature vector), transforming it into a unit vector. This eliminates the influence of the absolute value of the feature activations, allowing the distance metric to focus on differences in feature orientation, i.e., the similarities and differences in pattern information such as texture and structure. Subsequently, the mean of the squared Euclidean distances between the normalized first and second feature maps at all spatial locations is calculated, which is the normalized Euclidean distance for this layer. The smaller this value, the more similar the visual patterns of the two images are at this abstraction level.

[0045] Finally, because different intermediate layers capture different information (shallow layers capture more edges, deeper layers capture more semantics), the distance of a single layer is insufficient to comprehensively evaluate perceptual similarity. Therefore, the calculated normalized Euclidean distances of each layer can be assigned different layer weights (these weights can be learned by the LPIPS method on a large human perceptual dataset, reflecting the contribution of each layer's features to human visual judgment). The weighted distances of all layers are summed to obtain a scalar value, which is the loss term for learning perceptual image fragment similarity. This composite distance integrates multi-scale perceptual differences from low-level details to high-level semantics, providing a powerful and human-visually-aligned perceptual quality guidance signal for model optimization.

[0046] It should be noted that Table 1 shows a quantitative comparison of the image denoising method proposed in this application and the U-net method in terms of model complexity and computational efficiency. The number of model parameters reflects the spatial complexity of the model, mainly determined by the number of input / output channels and the kernel size in the convolutional layer; floating-point operations reflect the time complexity of the model, usually measured in GFLOPs (Giga Floating-point Operations Per Second), representing the computational overhead required during actual inference. To facilitate a fair comparison, the input image resolution can be uniformly set to 256×256 pixels when calculating floating-point operations, and statistics are performed based on the actual forward propagation process of each network structure. As shown in Table 1, the method proposed in this application significantly reduces the model's storage requirements and computational burden while maintaining performance, making it more suitable for real-time and resource-constrained applications.

[0047]

[0048] Figure 2 shows a direct comparison of the image denoising effect between the image denoising method of this application embodiment and the method represented by U-net. Figure 2(a) is the input noisy original image, Figure 2(b) is the denoising output of the U-net method, and Figure 2(c) is the denoising output of the image denoising method of this application embodiment.

[0049] Visual comparison clearly shows that the output image of the U-net method exhibits obvious oversmoothing in textured areas (such as hair and fabric details in the image), resulting in the loss of rich texture details and making the image appear stiff and unnatural. In contrast, the method in this application effectively suppresses noise while significantly better reproducing and preserving the subtle texture structure in the original image, making the denoised image visually closer to the real scene, with richer and more natural details.

[0050] The above comparison results show that, compared with methods relying on a single mean squared error loss, the gated texture convolutional neural network architecture and the perceptual-pixel joint loss function used in this application can guide the model to pay more attention to and reconstruct the texture information of the image during the denoising process. Therefore, this application can still achieve better perceptual quality while having significantly fewer model parameters and less computational cost than the U-net method (see Table 1), achieving a good balance between denoising effect and computational efficiency.

[0051] The image denoising method proposed in this application involves acquiring image data to be denoised and preprocessing it to obtain preprocessed image data. The preprocessed image data is then input into a preset image denoising model to obtain the final denoised image. The preset image denoising model is based on a gated texture convolutional neural network architecture and is trained using a preset joint loss function. This preset joint loss function includes a weighted sum of a mean squared error loss term and a loss term for learning perceptual image segment similarity. Therefore, by employing a lightweight gated texture convolutional neural network as the core architecture for denoising and combining it with a joint loss function that fuses pixel-level fidelity and perceptual similarity for model optimization, the problem of excessively smoothed image textures and decreased perceptual quality caused by denoising methods based on mean squared error loss is solved. This allows for the generation of denoised images with rich details and excellent visual perceptual quality within an efficient computational framework.

[0052] Next, the image noise reduction apparatus according to the embodiments of this application is described with reference to the accompanying drawings.

[0053] Figure 3 is a block diagram of an image noise reduction apparatus according to an embodiment of this application.

[0054] As shown in Figure 3, the image noise reduction device 10 includes a processing module 100 and a first acquisition module 200.

[0055] The processing module 100 is used to acquire the image data to be denoised and preprocess the image data to be denoised to obtain the preprocessed image data; the first obtaining module 200 is used to input the preprocessed image data into a preset image denoising model to obtain the final denoised image. The preset image denoising model is based on a gated texture convolutional neural network architecture and is trained using a preset joint loss function. The preset joint loss function includes a weighted sum of a mean squared error loss term and a learning-aware image segment similarity loss term.

[0056] Optionally, in some embodiments, before inputting the preprocessed image data into a preset image denoising model to obtain the final denoised image, the first obtaining module 200 is further configured to: obtain the original image dataset and the noisy image dataset corresponding to the original image data, and construct a training dataset based on the original image dataset and the noisy image dataset; preprocess the training dataset to obtain a preprocessed training dataset; divide the preprocessed training dataset into a training set, a validation set, and a test set based on a preset partitioning ratio; construct a target neural network based on a gated texture convolutional neural network architecture, and input the training set into the target neural network for training to obtain initial model parameters; based on the initial model parameters, input the validation set into the target neural network for performance evaluation, and adjust the initial model parameters according to the performance evaluation results until the preset joint loss function of the validation set converges to obtain the optimal model parameters; based on the optimal model parameters, input the test set into the target neural network for model testing, and obtain the preset image denoising model if the test results meet the preset requirements.

[0057] Optionally, in some embodiments, the first obtaining module 200 is specifically used to: perform multi-level downsampling and upsampling processing on the preprocessed image data using the multi-scale context extraction module of the preset image denoising model to extract the contextual structure feature information of the preprocessed image data; perform convolution processing on the preprocessed image data using the single-scale noise estimation module of the preset image denoising model to obtain the preliminary noise distribution estimation result of the preprocessed image data; convert the contextual structure feature information into modulation coefficients through the gated feature fusion module of the preset image denoising model; and perform weighted modulation processing on the preliminary noise distribution estimation result based on the modulation coefficients to generate the final denoised image.

[0058] Optionally, in some embodiments, the image denoising device 10 described above further includes: a calculation module, used to calculate the pixel-level difference between the current denoised image output by the preset image denoising model and the original image corresponding to the current denoised image, to obtain a mean squared error loss term; and a second obtaining module, used to obtain a learning-aware image segment similarity loss term based on the distance between the current denoised image output by the preset image denoising model and the original image corresponding to the current denoised image in a multi-layer deep feature space.

[0059] Optionally, in some embodiments, the second obtaining module is specifically used for: inputting the current denoised image and the original image corresponding to the current denoised image into a preset deep convolutional network to obtain multiple feature map pairs of intermediate layers, wherein the feature map pair includes a first feature map corresponding to the current denoised image and a second feature map corresponding to the original image corresponding to the current denoised image; calculating the normalized Euclidean distance of each feature map pair of intermediate layers, and weighting and summing each normalized Euclidean distance to obtain a loss term for learning the similarity of image segments.

[0060] It should be noted that the foregoing explanation of the image denoising method embodiment also applies to the image denoising device of this embodiment, and will not be repeated here.

[0061] The image denoising apparatus proposed in this application acquires image data to be denoised and preprocesses it to obtain preprocessed image data. The preprocessed image data is then input into a preset image denoising model to obtain the final denoised image. The preset image denoising model is based on a gated texture convolutional neural network architecture and is trained using a preset joint loss function. This preset joint loss function includes a weighted sum of a mean squared error loss term and a loss term for learning perceptual image segment similarity. Therefore, by employing a lightweight gated texture convolutional neural network as the core architecture for denoising and combining it with a joint loss function that fuses pixel-level fidelity and perceptual similarity for model optimization, the problem of excessively smoothed image textures and decreased perceptual quality caused by denoising methods based on mean squared error loss is solved. This allows for the generation of denoised images with rich details and excellent visual perceptual quality within an efficient computing framework.

[0062] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: a memory 401, a processor 402, and a computer program stored in the memory 401 and executable on the processor 402.

[0063] When the processor 402 executes the program, it implements the image noise reduction method provided in the above embodiments.

[0064] Furthermore, the electronic device also includes a communication interface 403 for communication between the memory 401 and the processor 402.

[0065] The memory 401 is used to store computer programs that can run on the processor 402.

[0066] The memory 401 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0067] If the memory 401, processor 402, and communication interface 403 are implemented independently, they can be interconnected via a bus to communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one thick line is used in Figure 4, but this does not indicate that there is only one bus or one type of bus.

[0068] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0069] Processor 402 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0070] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image noise reduction method described above.

[0071] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the image noise reduction method described above.

[0072] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0073] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0074] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. An image denoising method, characterized in that, Includes the following steps: Acquire the image data to be denoised, and preprocess the image data to be denoised to obtain the preprocessed image data; The preprocessed image data is input into a preset image denoising model to obtain the final denoised image. The preset image denoising model is based on a gated texture convolutional neural network architecture and is trained using a preset joint loss function. The preset joint loss function includes a weighted sum of a mean squared error loss term and a learning-aware image segment similarity loss term.

2. The method according to claim 1, characterized in that, Before inputting the preprocessed image data into the preset image denoising model to obtain the final denoised image, the method further includes: acquiring an original image dataset and a noisy image dataset corresponding to the original image data; constructing a training dataset based on the original image dataset and the noisy image dataset; preprocessing the training dataset to obtain a preprocessed training dataset; dividing the preprocessed training dataset into a training set, a validation set, and a test set based on a preset partitioning ratio; constructing a target neural network based on the gated texture convolutional neural network architecture; inputting the training set into the target neural network for training to obtain initial model parameters; inputting the validation set into the target neural network for performance evaluation based on the initial model parameters; adjusting the initial model parameters according to the performance evaluation results until the preset joint loss function of the validation set converges to obtain optimal model parameters; inputting the test set into the target neural network for model testing based on the optimal model parameters; and obtaining the preset image denoising model if the test results meet preset requirements.

3. The method according to claim 1, characterized in that, The step of inputting the preprocessed image data into a preset image denoising model to obtain the final denoised image includes: using the multi-scale context extraction module of the preset image denoising model to perform multi-level downsampling and upsampling processing on the preprocessed image data to extract the contextual structure feature information of the preprocessed image data; using the single-scale noise estimation module of the preset image denoising model to perform convolution processing on the preprocessed image data to obtain a preliminary noise distribution estimation result of the preprocessed image data; converting the contextual structure feature information into modulation coefficients through the gated feature fusion module of the preset image denoising model; and performing weighted modulation processing on the preliminary noise distribution estimation result based on the modulation coefficients to generate the final denoised image.

4. The method according to claim 1, characterized in that, Also includes: The pixel-level difference between the current denoised image output by the preset image denoising model and the original image corresponding to the current denoised image is calculated to obtain the mean square error loss term; Based on the distance between the current denoised image and the original image corresponding to the current denoised image in the multi-layer deep feature space output by the preset image denoising model, the learning-aware image segment similarity loss term is obtained.

5. The method according to claim 4, characterized in that, The method of obtaining the learning-aware image segment similarity loss term by measuring the distance between the current denoised image and the original image corresponding to the current denoised image in a multi-layer deep feature space based on the preset image denoising model includes: inputting the current denoised image and the original image corresponding to the current denoised image into a preset deep convolutional network to obtain multiple intermediate layer feature map pairs, wherein the feature map pair includes a first feature map corresponding to the current denoised image and a second feature map corresponding to the original image corresponding to the current denoised image; calculating the normalized Euclidean distance of each intermediate layer feature map pair, and weighting and summing each normalized Euclidean distance to obtain the learning-aware image segment similarity loss term.

6. An image noise reduction device, characterized in that, include: The processing module is used to acquire the image data to be denoised and to preprocess the image data to be denoised to obtain the preprocessed image data. The first obtaining module is used to input the preprocessed image data into a preset image denoising model to obtain the final denoised image. The preset image denoising model is based on a gated texture convolutional neural network architecture and is trained using a preset joint loss function. The preset joint loss function includes a weighted sum of a mean squared error loss term and a learning-aware image segment similarity loss term.

7. The apparatus according to claim 6, characterized in that, Before inputting the preprocessed image data into the preset image denoising model to obtain the final denoised image, the first obtaining module is further configured to: obtain the original image dataset and the noisy image dataset corresponding to the original image data, construct a training dataset based on the original image dataset and the noisy image dataset, preprocess the training dataset, and obtain the preprocessed training dataset. Based on a preset partitioning ratio, the preprocessed training dataset is divided into a training set, a validation set, and a test set. A target neural network based on the gated texture convolutional neural network architecture is constructed, and the training set is input into the target neural network for training to obtain initial model parameters. Based on the initial model parameters, the validation set is input into the target neural network for performance evaluation, and the initial model parameters are adjusted according to the performance evaluation results until the preset joint loss function of the validation set converges to obtain the optimal model parameters. Based on the optimal model parameters, the test set is input into the target neural network for model testing, and if the test results meet the preset requirements, the preset image denoising model is obtained.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the image noise reduction method as described in any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the image denoising method as described in any one of claims 1-5.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, is used to implement the image noise reduction method according to any one of claims 1-5.

Images