Image denoising method, device, computer equipment and storage medium

Abstract

The present application relates to the field of image processing technology, and discloses an image denoising method, device, computer equipment and storage medium. A preliminary denoising image is obtained by performing preliminary denoising processing on a noisy image based on a diffusion coefficient function model. A denoising convolutional neural network is used to perform a second denoising processing on the preliminary denoised image, and a denoised image is output. According to the characteristic information of the image, the threshold parameter and the number of iterations of the diffusion coefficient function are adjusted to adapt to the noise characteristics of different images, and finally the optimized target denoised image is output. The image denoising method of the present application can effectively remove noise from an image while accurately retaining the details and edge information of the image. By adaptively adjusting the threshold parameters and the number of iterations, the optimal balance between noise removal and detail protection is achieved, and the denoising effect is significantly improved. The resulting image has higher quality, richer details and clearer edges.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image denoising method, apparatus, computer equipment and storage medium. Background Art

[0002] In the field of image processing, especially image denoising, with the development of digital imaging technology, the improvement of image quality has received more and more attention. However, during the image acquisition and transmission process, due to the imperfection of sensors, the limitations of optical equipment and compressed transmission, noise will inevitably be introduced into the image, which not only reduces the image quality, but also increases the difficulty of subsequent analysis and processing. Therefore, image denoising has always been an important research direction in the field of computer vision.

[0003] At present, image denoising technology can be roughly divided into three categories: traditional filtering methods, methods based on total variation (TV) regularization, and methods based on deep learning. Traditional filtering methods use linear or nonlinear filters to denoise, however, these methods often lose image details and edges during denoising. For this reason, adaptive filtering methods have emerged, which can adjust the filtering strength according to the local features of the image, but still face the problems of excessive denoising and loss of details. Another method is based on the total variation (TV) model, which uses the local features of the image, especially the edge information, for denoising. Although this method can better preserve the image edges, when the noise is strong, the balance between denoising effect and detail retention is still challenging, which may cause blurred details.

[0004] With the development of deep learning, image denoising methods based on convolutional neural networks (CNNs) have gradually become mainstream. These methods can remove noise while retaining more details through training. Despite the progress made in existing technologies, how to accurately protect the edges and details of images remains an important issue. In the diffusion coefficient model based on partial differential equations, the diffusion coefficient function is used to denoise and protect the edges of images. However, the key parameter of the diffusion coefficient function, the threshold parameter K, is usually set through manual experience or prior estimation, which limits the effectiveness of the method, especially in edge protection.

[0005] In existing methods, the choice of threshold parameter K will affect the denoising effect. If the K value is set improperly, it may cause image edges to be blurred, details to be lost, or staircase artifacts to be generated, especially when the noise is strong. In addition, traditional methods cannot adaptively adjust parameters according to the local features of the image, resulting in unsatisfactory denoising effects.

[0006] Therefore, how to intelligently and accurately adjust the threshold parameter K to remove noise while protecting image details and edges has become an important issue that needs to be solved urgently. Summary of the invention

[0007] In view of this, the embodiments of the present application provide an image denoising method, apparatus, computer device and storage medium, which can effectively solve the problems in the prior art.

[0008] In a first aspect, an embodiment of the present application provides an image denoising method, comprising:

[0009] Get a noisy image;

[0010] Performing preliminary denoising on the noisy image based on a diffusion coefficient function model to obtain a preliminary denoised image, wherein the diffusion coefficient function model suppresses noise and protects image edge information by calculating gradient information of the noisy image and adjusting a threshold parameter of the diffusion coefficient function;

[0011] A denoising convolutional neural network is used to perform a second denoising process on the preliminary denoised image, wherein the denoising convolutional neural network predicts the noise residual signal of the noisy image by a residual learning method and outputs a denoised image;

[0012] According to the characteristic information of the denoised image output by the denoising convolutional neural network, the threshold parameter and the number of iterations of the diffusion coefficient function are dynamically adjusted to adapt to the noise characteristics of different images, and an optimized target denoised image is output.

[0013] In some embodiments, performing preliminary denoising on the noisy image based on the diffusion coefficient function model to obtain a preliminary denoised image includes:

[0014] Downsampling the noisy image, and using a convolution operation to extract image feature information of the noisy image at different scales;

[0015] Based on the image feature information, calculating the gradient information of the noisy image in different directions to obtain a gradient map of the noisy image;

[0016] Designing a diffusion coefficient function according to the gradient information and initializing a threshold parameter in the diffusion coefficient function;

[0017] The diffusion coefficient function model is used to perform iterative processing to generate the preliminary denoised image.

[0018] In some embodiments, designing a diffusion coefficient function according to the gradient information and initializing a threshold parameter in the diffusion coefficient function includes:

[0019] The gradient information is used to measure the intensity of changes in each area of ​​the image, and dynamically adjust the diffusion coefficient of the diffusion coefficient function according to the gradient size of different areas; the threshold parameter is used to control the diffusion intensity, reduce the diffusion intensity in the edge area of ​​the image, and enhance the diffusion intensity in the flat area.

[0020] In some embodiments, the second denoising process of the preliminary denoised image using a denoising convolutional neural network includes:

[0021] Inputting the noisy image and the preliminary denoised image into the denoising convolutional neural network for end-to-end training, wherein the denoising convolutional neural network includes a plurality of convolutional layers and deconvolutional layers for extracting feature information of the preliminary denoised image and restoring details and edge information of the image layer by layer;

[0022] Using the residual learning method, learning the noise residual between the preliminary denoised image and the noisy image;

[0023] Optimizing the weights of the neural network by back propagation so that details and edge information of the preliminary denoised image are retained while removing noise;

[0024] Through repeated convolution operations, the details and edge information in the preliminary denoised image are restored layer by layer, and the denoised image is output.

[0025] In some embodiments, the dynamically adjusting the threshold parameter and the number of iterations of the diffusion coefficient function to meet the denoising requirements of different images and noise types, and outputting an optimized target denoised image, includes:

[0026] Calculating local gradient information of the denoised image according to noise feature information in the denoised image, and dynamically adjusting a threshold parameter and the number of iterations of the diffusion coefficient function according to the local gradient information to meet denoising requirements of different noise types;

[0027] According to the adjusted threshold parameter and the number of iterations, the parameters of the diffusion coefficient function model are updated, and the updated parameters are used to perform denoising processing in each iteration process until the denoising requirements of the different images are met, and the target denoised image is generated and output.

[0028] In some embodiments, dynamically adjusting the threshold parameter and the number of iterations of the diffusion coefficient function according to the characteristics of the denoised image further includes:

[0029] Increase the threshold parameter and the number of iterations in areas with stronger noise, and decrease the threshold parameter and the number of iterations in edge areas with weaker noise.

[0030] In some embodiments, the outputting of the optimized target denoised image includes:

[0031] Upsampling the target denoised image to increase the image resolution to a preset target resolution by an interpolation method;

[0032] Performing feature fusion on the upsampled target denoised image to improve detail information of the target denoised image;

[0033] The target denoised image after feature fusion is reconstructed, the visual parameters in the image are restored, and a standard image is output.

[0034] In a second aspect, an embodiment of the present application provides an image denoising device, comprising:

[0035] An image acquisition module, used for acquiring a noisy image;

[0036] An image processing module, configured to perform preliminary denoising on the noisy image based on a diffusion coefficient function model to obtain a preliminary denoised image, wherein the diffusion coefficient function model suppresses noise and protects image edge information by calculating gradient information of the noisy image and adjusting a threshold parameter of the diffusion coefficient function;

[0037] A denoising processing module, configured to perform a second denoising process on the preliminary denoised image using a denoising convolutional neural network, wherein the denoising convolutional neural network predicts a noise residual signal of the noisy image by a residual learning method and outputs a denoised image;

[0038] The iterative optimization module is used to dynamically adjust the threshold parameter and the number of iterations of the diffusion coefficient function according to the characteristic information of the denoised image output by the denoising convolutional neural network to adapt to the noise characteristics of different images, and output the optimized target denoised image.

[0039] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the image denoising method of the first aspect.

[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, and when the computer program is executed on a processor, the image denoising method of the first aspect is implemented.

[0041] The embodiments of the present application have the following beneficial effects: The image denoising method of the present application significantly improves the denoising effect of the image by combining the advantages of the diffusion coefficient function model and the denoising convolutional neural network (DNCNN), through multi-stage denoising processing and dynamic adjustment mechanism, while effectively protecting the edge and detail information of the image. First, the preliminary denoising can effectively remove the noise in the image through the use of the diffusion coefficient function model. The diffusion coefficient function model calculates the gradient information of the image, enhances diffusion in the flat area of ​​the image to remove noise, and suppresses diffusion in the edge area, thereby protecting the edge and details of the image. The initialization and dynamic adjustment of the threshold parameter K ensure the adaptability of the diffusion coefficient function in different areas, making the image denoising processing more accurate.

[0042] Secondly, the denoising convolutional neural network (DNCNN) further optimizes the denoising effect. Through the residual learning strategy, DNCNN can accurately predict the noise residual signal in the noisy image and compare it with the preliminary denoised image, and then continuously remove the noise through the training of the neural network, while restoring the details and edges of the image. This denoising method based on deep learning can better handle complex noise types and accurately restore the image structure through multiple layers of convolutional and deconvolutional layers, so that the final output image retains more detailed information while denoising.

[0043] Finally, by analyzing the image feature information output by the denoising convolutional neural network, the threshold parameter K and the number of iterations in the diffusion coefficient function model are dynamically adjusted. This adjustment mechanism is based on the feature information output by the network and can dynamically optimize the diffusion processing according to the different noise characteristics of the image. In areas with strong noise, the threshold parameter K and the number of iterations are appropriately increased to enhance the denoising effect; while in edge areas, the threshold parameter K is reduced and the number of iterations is reduced to avoid excessive diffusion and edge blur. The dynamic adjustment of the threshold parameter K ensures that the image denoising process can effectively remove noise while accurately protecting image details and edges.

[0044] This adaptive adjustment strategy makes the entire denoising process more flexible and efficient, and can automatically optimize the denoising strength and smoothing effect according to the image characteristics, ensuring the optimization of the final denoised image quality. Ultimately, the present invention can not only retain important details and edge information during the image denoising process, but also effectively remove various types of noise, thereby generating images with clear visual effects, rich details and clear edges. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A working schematic diagram of an image denoising model based on a deep learning algorithm in an image denoising method in an embodiment of the present application is shown;

[0047] Figure 2 A schematic diagram of a process of an image denoising method according to an embodiment of the present application is shown;

[0048] Figure 3 Another working schematic diagram of an image denoising method in an embodiment of the present application is shown;

[0049] Figure 4 The working principle of the DNCNN model in an image denoising method in an embodiment of the present application is shown;

[0050] Figure 5 A schematic diagram of experimental data in an image denoising method according to an embodiment of the present application is shown;

[0051] Figure 6 A schematic structural diagram of an image denoising device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0053] The components of the embodiments of the present application generally described and shown in the drawings herein may be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0054] Hereinafter, the terms "including", "having" and their cognates that can be used in various embodiments of the present application are intended only to indicate specific features, numbers, steps, operations, elements, components or a combination of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or a combination of the foregoing items or increasing the possibility of one or more features, numbers, steps, operations, elements, components or a combination of the foregoing items. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions and cannot be understood as indicating or implying relative importance.

[0055] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present application.

[0056] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0057] Considering that the existing image denoising methods are difficult to effectively protect the details and edge information of the image while removing noise, an image denoising method is proposed, such as Figure 1 As shown in the figure, by combining the diffusion coefficient function model and the denoising convolutional neural network (DNCNN), a multi-stage denoising process and adaptive adjustment mechanism are realized. This method can dynamically adjust the threshold parameter K in the diffusion coefficient function according to the characteristics of different regions of the image, thereby effectively removing noise while accurately retaining the details and edge information of the image.

[0058] In the following embodiments, an anisotropic diffusion coefficient function model is selected to optimize the threshold parameter K. Specifically, the estimation method of the threshold parameter K provided by the diffusion coefficient function model in four directions is used, and calculations are performed in each direction respectively. By analyzing the gradient information in each direction and combining the denoising convolutional neural network model (DNCNN) to optimize the threshold parameter K, the performance of image denoising is effectively improved. The specific calculation steps are as follows:

[0059] Figure 2 A flow chart of an image denoising method according to an embodiment of the present application is shown. Exemplarily, the method includes steps S100 to S400.

[0060] Step S100, obtaining a noisy image.

[0061] Exemplarily, unprocessed image data is obtained from various image acquisition devices or image data sources. These images are usually affected by various factors, such as noise of the device during the acquisition process, compression of data during the transmission process, imperfections of the sensor, etc., resulting in noise in the image. Such noise can be manifested in the form of uneven brightness in the image, randomly scattered granular textures, or color deviation.

[0062] In practical applications, the process of acquiring noisy images may include acquiring data from different sources. For example, when taking an image with a camera, the image may be affected by factors such as low light and high ISO settings, resulting in Gaussian noise, salt and pepper noise, etc. In medical imaging, satellite imaging, or industrial inspection, the image may also be affected by different types of noise due to factors such as sensor accuracy, environmental conditions, transmission errors, etc.

[0063] Step S200, performing preliminary denoising on the noisy image based on the diffusion coefficient function model to obtain a preliminary denoised image. The diffusion coefficient function model suppresses noise and protects image edge information by calculating gradient information of the noisy image and adjusting a threshold parameter of the diffusion coefficient function.

[0064] Exemplarily, the diffusion coefficient function model is a mathematical model that processes image noise by simulating the diffusion phenomenon in the physical process. The basic principle of this process is to regard the pixels of the image as a substance in the physical system (for example, temperature or concentration), and then describe the propagation process of the substance in space through the diffusion equation. The diffusion coefficient function is used to control this diffusion process and determines the smoothness of different areas in the image.

[0065] In the preliminary denoising process, the diffusion coefficient function model first calculates the gradient information of the image to understand the local structure of the image. The gradient of the image represents the degree of change of the pixel value, which can effectively reflect the edges and details of the image. When the gradient is large, it usually means that the area is an edge or detail area and should remain unchanged, so diffusion needs to be suppressed; while in a flat area with a small gradient, it means that there is a lot of noise in the area and diffusion processing is required to remove the noise. In order to achieve this effect, the threshold parameter in the diffusion coefficient function will be dynamically adjusted according to the gradient of the image. When the gradient of the image is large, the threshold parameter will be small, thereby suppressing diffusion and maintaining edge details; while in a flat area, the threshold parameter will be large, enhancing diffusion and removing noise. In this way, the diffusion coefficient function model can effectively remove noise from the image while retaining the edge of the image to generate a preliminary denoised image.

[0066] The gradient information is used to adjust the denoising strength of the image. The mathematical formula of the diffusion coefficient function model is as follows:

[0067]

[0068] in, Represents the rate of change of the image over time t; : Gradient operator of the image, which indicates the rate of change of image pixels; G(||∇I||): Diffusion coefficient function, which adjusts the diffusion intensity according to the gradient size of the image to protect the edge of the image; || I||: The gradient modulus of the image, which indicates the degree of change in image intensity.

[0069] That is, in each iteration, the pixel value is updated according to the gradient and diffusion coefficient of the signal, thereby removing noise and smoothing the image.

[0070] In an optional embodiment, if Figure 3 As shown, in step S200, a preliminary denoising process is performed on the noisy image based on the diffusion coefficient function model, including:

[0071] First, the noisy image needs to be downsampled. Downsampling is to extract the feature information of the image at different scales by reducing the resolution of the image. In this way, the amount of calculation can be reduced while retaining the main structural features of the image. In the downsampled image, a convolution operation is usually performed to extract the multi-scale feature information of the image. The convolution operation can help capture the detailed information such as texture, shape and edge in the image. These feature information provides the necessary data support for the subsequent gradient calculation.

[0072] Based on the extracted multi-scale image features, the gradient information of the image in different directions (for example, horizontal, vertical and diagonal directions) is calculated. Gradient information can reflect the degree of change of pixel values ​​in the image and is an important basis for judging the edge area and flat area of ​​the image. Gradient information can be calculated using the Sobel operator, Laplace operator or other edge detection algorithms. For the pixel point I (x, y), its gradient value can be calculated by comparing the grayscale difference of adjacent pixels. In the diffusion coefficient function model, typical gradient directions include four directions: up, down, left and right. The corresponding gradient calculation formula is as follows:

[0073]

[0074]

[0075]

[0076]

[0077] The gradient calculation formula is used to obtain the gradient of the image in different directions. , , , To represent the gradient, M and N represent the image, which reflects how to calculate the gradient of the image in the horizontal and vertical directions so as to facilitate the gradient weighting in the subsequent diffusion process.

[0078] Furthermore, the design of the diffusion coefficient function is based on the gradient information of the image, and the diffusion coefficient is dynamically adjusted according to the local features of the image to meet the denoising requirements of different areas. When initializing the threshold parameters in the diffusion coefficient function, some empirical methods are usually used to ensure that a relatively reasonable denoising effect can be achieved in the initial state. The diffusion coefficient function is used to control the denoising effect of the image and ensure that the image edge is effectively protected. In this scheme, the diffusion coefficient is adaptively adjusted through the following formula, and the intensity of the diffusion is dynamically controlled according to the gradient size of the image:

[0079]

[0080] Where: || I||: The gradient modulus of the image, indicating the local change of the image at a certain position; ω(|| I||): Diffusion coefficient, which controls the diffusion intensity in different areas and avoids excessive diffusion in the edge areas; It is the ratio of the square of the gradient modulus to a constant, which is used to adjust the magnitude of the gradient effect; The exponential function gives a larger diffusion coefficient (smaller gradient) to the flat area of ​​the image, and reduces the diffusion coefficient in the edge area (larger gradient) to protect the edge information.

[0081] The diffusion coefficient formula adjusts the intensity of diffusion through gradient information. In the edge area of ​​the image, the diffusion coefficient is small, thereby preventing the loss of image edge information; in the flat area, the diffusion coefficient is large, which helps to remove noise. The designed diffusion coefficient function model is used for iterative processing. The iterative processing process continuously updates the denoising effect of the image based on the current gradient information and the adjusted threshold parameters. Each iteration can further reduce the noise in the image, while gradually restoring the edges and details of the image until the ideal denoising effect is achieved. After multiple iterations, the optimized preliminary denoising image is finally generated.

[0082] For example, the following iterative formula of the diffusion model can be used to iterate, so that in each iteration, each pixel of the image is updated according to its gradient information, noise is removed and image quality is optimized:

[0083]

[0084] in, The pixel value of the image at time t and position (x, y); It represents the gradient calculation of the image in different directions, and the diffusion intensity is adjusted by the diffusion coefficient function f.

[0085] That is, in each iteration, the pixels of the image are updated according to their local gradients and diffusion coefficients, removing noise and smoothing the image.

[0086] In an optional implementation, designing a diffusion coefficient function according to the gradient information and initializing a threshold parameter in the diffusion coefficient function includes:

[0087] Exemplarily, the threshold parameter K is a key parameter in the diffusion coefficient function, which controls the intensity of the diffusion process, thereby effectively protecting the edge and detail information of the image while denoising. Specifically, the role of the threshold parameter K is to dynamically adjust the diffusion coefficient according to the gradient information of the image to adapt to the noise characteristics and detail protection requirements of different regions.

[0088] In the edge area of ​​the image, since the gradient in this area is large, it represents important structures and details in the image. When denoising, it is necessary to suppress diffusion to avoid over-smoothing and loss of details. Therefore, in these edge areas, the K value is small, thereby reducing the diffusion intensity, limiting the impact of the diffusion process on the edges, and ensuring that the details of the image can be preserved. On the contrary, in the flat area of ​​the image, due to the small gradient, the noise is usually more significant. At this time, the diffusion intensity needs to be enhanced to effectively remove the noise. Therefore, in the flat area, the K value is large, thereby enhancing the diffusion process to remove more noise.

[0089] The diffusion coefficient function model can achieve a more intelligent denoising process by dynamically adjusting the threshold parameter K. Specifically, when the local gradient of the image is large (i.e., the edge or structure area), the K value will become smaller, inhibiting diffusion and keeping the image edge clear; while in the flat area of ​​the image, the K value will be larger to enhance diffusion and effectively remove noise.

[0090] Therefore, the threshold parameter K not only determines the intensity of diffusion in the image denoising process, but also can achieve a balance between noise removal and edge protection in different image regions through its dynamic adjustment, ensuring the image denoising effect while retaining as much detail information as possible.

[0091] Step S300, a denoising convolutional neural network is used to perform a second denoising process on the preliminary denoised image. The denoising convolutional neural network predicts the noise residual signal of the noisy image through a residual learning method and outputs a denoised image.

[0092] As a deep learning model, the denoising convolutional neural network (DNCNN) performs image denoising through end-to-end training. For images that have been initially denoised, the denoising convolutional neural network is used to further perform a second denoising process to improve the image quality, especially the restoration of details and edges. In this process, the main task of the convolutional neural network is to predict the noise residual, that is, the noise part in the image, from the initial denoised image.

[0093] In this step, the network learns the characteristics of noise through residual learning based on the difference between the preliminary denoised image and the original noisy image. The residual learning method can effectively reduce the interference of noise and restore the details in the image. When the network training is completed, it can accurately predict the noise residual in the image and correct the preliminary denoised image based on the residual to obtain a higher quality denoised image. This process ensures that the edge and detail information of the image is better preserved after denoising, improving the visual quality of the image.

[0094] In detail, the working principle of the DNCNN model is as follows: the DNCNN model can predict the residual signal from the noise signal, combined with the discriminator network of the GAN model, to achieve adaptive evaluation of the denoising model, effectively evaluate the denoising effect of the diffusion coefficient function and the threshold parameter K, and the system automatically determines whether the denoising has achieved the optimal effect. The purpose of this model is to sample the noisy image signal x as input and the image signal v that needs to be filtered as the target. The noise signal x and v are decomposed into a training set, a validation set, and a test set respectively. The training set is used to train the network model end-to-end, and the validation set is used to adjust the hyperparameters in the network model. The signal that needs to be denoised, the original collected signal x, is input into the network model. End-to-end training is used to learn the residual signal and obtain the denoised signal through residual connection. The denoised signal and the original clean signal are used in the discriminator to optimize the parameters of the DNCNN denoising network. At the same time, the image itself is denoised using a diffusion coefficient function model based on anisotropy. The image parameters after two-way reconstruction will be classified by the discriminator for signal classification, and continuously fed back to the denoising convolutional neural network model for parameter tuning. At the same time, during the tuning process, the threshold K and the number of denoising times in the anisotropic diffusion coefficient function model will stop increasing the K value after the parameters of the DNCNN model reach the optimal value, thereby automatically achieving the optimal solution of the optimal diffusion function threshold K for the anisotropic diffusion coefficient function model.

[0095] Furthermore, if Figure 4As shown in the figure, using the characteristics of the DNCNN model, the signal x to be denoised is input into the network model. End-to-end training is used to learn the residual signal n, and the denoised signal v=x−n is obtained through residual connection. The denoised signal and the original clean signal are used in the discriminator to optimize the parameters of the DNCNN denoising network. The input-output relationship of the convolution operation in the convolution layer of this model is:

[0096]

[0097] in, ; Represents convolution and deconvolution operations, represents the Lth neuron in the ith input layer. represents the scale of the filter, represents the bias value, and f represents the activation function; assuming that in the convolution operation, we set the step size of the Lth layer to S, then the signal length formula of the (L+1)th layer is defined as: represents rounding up. In the deconvolution process, the signal length formula of the (L+1)th layer is defined as: After batch normalization, the convolutional layer output of the feature signal is , input to the normalization layer to get the output The training parameters are Using Linear Operations To expand or scale. To prevent gradient explosion and gradient disappearance, and to speed up neural network training and convergence. In the convolution layer, the residual block is introduced ,in Indicates the input signal layer, Represents the residual map. The residual block is used to connect the previous residual block, the combination of the convolution block and the deconvolution block to speed up the training process, and finally the residual signal is used To obtain a noise-reduced signal .

[0098] In an optional embodiment, in step S300, a denoising convolutional neural network is used to perform a second denoising process on the preliminary denoised image, including:

[0099] The end-to-end training method is adopted, and the noisy image and the preliminary denoised image are used as input and sent to the denoising convolutional neural network for learning. In this training mode, the goal of the denoising convolutional neural network is to minimize the error between the input image and the output image. During the training process, the weights of the network will be gradually adjusted until the model can better remove noise while retaining the details and edge information of the image.

[0100] The denoising convolutional neural network consists of multiple convolutional layers and deconvolutional layers. The convolutional layer is used to extract local feature information in the image, including texture, edge, color, etc., while the deconvolutional layer is used to restore the details of the image, especially the edge of the image. In each convolutional layer, the features of the image are gradually abstracted and extracted, and the deconvolutional layer gradually reconstructs these features to restore the high-frequency detail information of the image, such as the edge, texture and structure of the image.

[0101] When using residual learning, the neural network further refines the denoising process by learning the difference between the initial denoised image and the noisy image (i.e., the noise residual). Residual learning not only enables the network to focus on removing noise, but also helps it better recover important details in the image. Under this strategy, the goal of the network is to optimize the weights so that it can remove noise while maximizing the preservation of image details and edge information.

[0102] The back propagation algorithm in the training process is used to adjust the weights of the neural network to ensure that the denoised image output by the model is as close to the real clean image as possible. Through back propagation, the network can adjust the parameters of each layer according to the error gradient, so that in subsequent image denoising, the network can output a denoised image with higher quality.

[0103] Finally, after multiple convolution operations and training, the network is able to restore the image details layer by layer and output an optimized denoised image, in which the noise is effectively removed, while the edge and detail information of the image is well preserved, ensuring the clarity and visual effect of the final image.

[0104] Step S400, according to the characteristic information of the denoised image output by the denoising convolutional neural network, dynamically adjust the threshold parameter and the number of iterations of the diffusion coefficient function to adapt to the noise characteristics of different images, and output the optimized target denoised image.

[0105] After denoising the input image, the denoising convolutional neural network (DNCNN) outputs a denoised image. This image contains the details and edge information after the noise is removed. When further optimizing the image quality, a key step is to dynamically adjust the threshold parameters and number of iterations of the diffusion coefficient function based on the feature information of the denoised image, especially the type and characteristics of the noise in the image.

[0106] First, the gradient information of the local area of ​​the denoised image is calculated based on the noise feature information of the denoised image. Then, the threshold parameter K and the number of iterations in the diffusion coefficient function are dynamically adjusted according to the noise characteristics of different areas in the image (for example, noise intensity and noise type). The threshold parameter K directly determines the intensity of image diffusion. In areas with strong noise, the threshold needs to be increased to enhance the denoising process; while in areas with weak noise (for example, edge areas), the threshold needs to be reduced to avoid excessive diffusion and retain image details.

[0107] In addition, the number of iterations will be dynamically adjusted according to the noise characteristics of the image. For images with strong noise, more iterations may be required to further remove the noise; while for images with weak noise, appropriately reducing the number of iterations can avoid over-smoothing and protect image details.

[0108] In this process, after each iteration, the denoising effect is evaluated through the discriminator loss function of DNCNN, and the threshold parameter K in the diffusion coefficient function is adjusted to ensure the optimization of the denoising effect. This process will be iterated continuously until the ideal denoising effect is achieved and an optimized target denoised image is generated. In the end, the denoising effect is significantly improved, and the image details are better preserved.

[0109] In an optional embodiment, in step S400, dynamically adjusting the threshold parameter and the number of iterations of the diffusion coefficient function to meet the denoising requirements of different images and noise types, and outputting an optimized target denoised image, includes:

[0110] After obtaining the denoised image, the noise feature information in the image is further analyzed, which is usually achieved by calculating the local gradient information of the image. The local gradient information can reflect the rate of change of brightness in different areas of the image, especially in the edge area, where the gradient is larger; while in the flat area, the gradient is smaller. By calculating this gradient information, it can be clear which areas contain more details and edge information, and which areas are flat areas with more noise.

[0111] Based on the calculated local gradient information, the threshold parameter and the number of iterations of the diffusion coefficient function are dynamically adjusted. In the edge area with a large gradient, the threshold parameter will be set smaller to reduce the diffusion intensity and ensure that the edges and details of the image are protected; in the flat area with a small gradient, the threshold parameter will be increased to enhance the diffusion intensity and remove more noise. In this way, through the analysis of local gradient information, the denoising process of the image can be adjusted in a targeted manner to make the denoising process more in line with the noise characteristics of the image.

[0112] In each iteration of image denoising, the threshold parameter of the diffusion coefficient function is updated, and the denoising process is continued using the updated threshold parameter. After each iteration, the noise in the image will gradually decrease, and the details and edges of the image will be better preserved. The iterative process will continue until the image achieves the desired denoising effect, that is, the noise in the image is removed, the details are restored, and the target denoised image that meets the requirements of different image denoising is generated.

[0113] This dynamic adjustment mechanism based on image gradient information and noise characteristics enables image denoising to adapt to different noise types more accurately, removing noise while retaining image edge details, achieving high-quality image denoising effects.

[0114] In an optional implementation, dynamically adjusting the threshold parameter and the number of iterations of the diffusion coefficient function according to the characteristics of the denoised image also includes:

[0115] The threshold parameter controls the diffusion intensity during the image denoising process, while the number of iterations determines the depth of the denoising process. Therefore, how to dynamically adjust these parameters according to the noise characteristics of the image, especially according to the noise intensity in different areas of the image, is the core of optimizing the image denoising effect.

[0116] First, in areas with strong noise, the gradient of the image is usually small, which means that these areas may be flat background areas with relatively obvious noise. In order to effectively remove these noises, we need to increase the threshold parameter and the number of iterations. Increasing the threshold parameter can enhance the diffusion intensity, thereby promoting the removal of noise and reducing the interference of noise on the image. Increasing the number of iterations can ensure that the denoising process is carried out in depth, so that the noise can be completely removed after multiple rounds of processing. This strategy ensures that the noise in the image can be effectively suppressed and eliminated by strengthening the diffusion process in areas with strong noise.

[0117] However, in the edge areas with weak noise, the gradient of the image is large, which usually represents the structure and edge information of the image. In these areas, excessive diffusion will cause the details and edges of the image to be blurred, thus losing the quality of the image. To avoid this, the threshold parameter should be reduced to suppress the diffusion intensity and maintain the edges and details of the image. At the same time, reducing the number of iterations also helps prevent over-smoothing of the image in these areas, thereby retaining more image details and edge structures.

[0118] Through this adjustment strategy, the image noise removal process can more accurately adapt to different areas of the image: in areas with strong noise, the denoising process is strengthened; while in edge areas with weak noise, the denoising process is suppressed, thus effectively avoiding edge blurring and detail loss. This strategy can maximize the preservation of image structure and detail information while denoising, improving the visual effect of the denoised image.

[0119] In an optional embodiment, after step S400, the following steps are included:

[0120] After the denoising process is completed, the resolution of the image may be lower than the desired final output requirement. Therefore, the denoised image is first upsampled to increase the resolution of the image. Upsampling is achieved through interpolation methods, and common interpolation methods include bilinear interpolation, cubic interpolation, etc. Through these interpolation methods, the pixel values ​​of the image are smoothly calculated between existing pixels to generate a higher resolution image. Upsampling can not only increase the resolution of the image, but also keep the image smooth and delicate transition effect after enlargement, making it visually clearer.

[0121] After upsampling an image, although the resolution is improved, the details and structures in the image may still be insufficient. Therefore, feature fusion is a key step to improve image details. Through multi-level feature extraction and fusion, high-frequency details in the image can be enhanced, especially in terms of texture, edges, and color. Usually, feature fusion technology uses convolutional neural networks (CNNs) or other deep learning methods to extract feature information at different levels of the image and combine them together to produce richer image features. The fused image details will be clearer, especially the edges and complex texture structures in the image.

[0122] After feature fusion, the image may still lack some important visual characteristics, such as contrast, brightness, saturation and other visual parameters. Therefore, it is necessary to reconstruct the fused target denoised image to restore the important visual parameters in the image. By adjusting the image's brightness, contrast, saturation and other parameters, the visual effect of the image can be further improved to make it more natural and realistic. The reconstruction process is usually completed through image enhancement technology or deep learning-based models to ensure that the image meets the expected standards in terms of visual performance. The final output standard image has high resolution, rich details, and a good balance in color and contrast, thereby providing a clear and realistic image effect.

[0123] Through these three steps, upsampling, feature fusion and image reconstruction, not only the resolution of the image is improved, but also the details and visual effects of the image are effectively restored, ensuring that the denoised image has higher quality and better visual experience. This process is particularly suitable for application scenarios that require high resolution and fine details, such as medical imaging, satellite imaging and high-definition photography.

[0124] It is worth noting that Figure 5As shown, the present invention adopts the industry-standard Set12 data set, and uses PSNR and SSIM indicators to evaluate the quality of smoothed images by different methods. PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index) are two commonly used image quality evaluation indicators, each of which has different definitions, advantages and disadvantages, and applicable scenarios. Compared with other mainstream diffusion coefficient function models, the PSNR and SSIM when different noises are introduced into the image are as follows. The image denoising model based on deep learning algorithm in this application has a certain improvement in image edge quality compared with traditional models. Because this model pays more attention to iterative optimization of the threshold parameter K using a neural network, it is actually possible to consider updating the algorithm model in the future to combine it with a more efficient diffusion coefficient function model to further improve the ability of image edge denoising.

[0125] Figure 6 A schematic diagram of the structure of an image denoising device according to an embodiment of the present application is shown. Exemplarily, the device includes:

[0126] An image acquisition module 61 is used to acquire a noisy image;

[0127] An image processing module 62 is used to perform preliminary denoising on the noisy image based on a diffusion coefficient function model to obtain a preliminary denoised image, wherein the diffusion coefficient function model suppresses noise and protects image edge information by calculating gradient information of the noisy image and adjusting a threshold parameter of the diffusion coefficient function;

[0128] A denoising processing module 63, configured to perform a second denoising process on the preliminary denoised image using a denoising convolutional neural network, wherein the denoising convolutional neural network predicts a noise residual signal of the noisy image by a residual learning method and outputs a denoised image;

[0129] The iterative optimization module 64 is used to dynamically adjust the threshold parameters and the number of iterations of the diffusion coefficient function according to the characteristic information of the denoised image output by the denoising convolutional neural network to adapt to the noise characteristics of different images, optimize the diffusion intensity and smoothing effect, and output the optimized target denoised image.

[0130] It can be understood that the device of this embodiment corresponds to the method of the above embodiment, and the options in the above embodiment are also applicable to this embodiment, so they will not be described repeatedly here.

[0131] The present application also provides a computer device. Exemplarily, the computer device includes a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program, thereby enabling the computer device to execute the functions of each module in the above method or the above apparatus.

[0132] Among them, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or at least one of other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0133] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electric erasable programmable read-only memory (EEPROM), etc. The memory is used to store a computer program, and the processor may execute the computer program accordingly after receiving an execution instruction.

[0134] The present application also provides a computer-readable storage medium for storing the computer program used in the above-mentioned computer device. For example, the computer-readable storage medium may include but is not limited to: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0135] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0136] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0137] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0138] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. An image denoising method, characterized in that: The method comprises: Get a noisy image; Performing preliminary denoising on the noisy image based on a diffusion coefficient function model to obtain a preliminary denoised image, wherein the diffusion coefficient function model suppresses noise and protects image edge information by calculating gradient information of the noisy image and adjusting a threshold parameter of the diffusion coefficient function; A denoising convolutional neural network is used to perform a second denoising process on the preliminary denoised image, wherein the denoising convolutional neural network predicts the noise residual signal of the noisy image by a residual learning method and outputs a denoised image; According to the characteristic information of the denoised image output by the denoising convolutional neural network, the threshold parameter and the number of iterations of the diffusion coefficient function are dynamically adjusted to adapt to the noise characteristics of different images, and an optimized target denoised image is output.

2. The image denoising method according to claim 1, characterized in that: The performing preliminary denoising on the noisy image based on the diffusion coefficient function model to obtain a preliminary denoised image includes: Downsampling the noisy image, and using a convolution operation to extract image feature information of the noisy image at different scales; Based on the image feature information, calculating the gradient information of the noisy image in different directions to obtain a gradient map of the noisy image; Designing a diffusion coefficient function according to the gradient information and initializing a threshold parameter in the diffusion coefficient function; The diffusion coefficient function model is used to perform iterative processing to generate the preliminary denoised image.

3. The image denoising method according to claim 2, characterized in that: The step of designing a diffusion coefficient function according to the gradient information and initializing a threshold parameter in the diffusion coefficient function includes: The gradient information is used to measure the intensity of changes in each area of ​​the image, and dynamically adjust the diffusion coefficient of the diffusion coefficient function according to the gradient size of different areas; the threshold parameter is used to control the diffusion intensity, reduce the diffusion intensity in the edge area of ​​the image, and enhance the diffusion intensity in the flat area.

4. The image denoising method according to claim 1, characterized in that: The second denoising process of the preliminary denoised image using a denoising convolutional neural network comprises: Inputting the noisy image and the preliminary denoised image into the denoising convolutional neural network for end-to-end training, wherein the denoising convolutional neural network includes a plurality of convolutional layers and deconvolutional layers for extracting feature information of the preliminary denoised image and restoring details and edge information of the image layer by layer; Using the residual learning method, learning the noise residual between the preliminary denoised image and the noisy image; Optimizing the weights of the neural network by back propagation so that details and edge information of the preliminary denoised image are retained while removing noise; Through repeated convolution operations, the details and edge information in the preliminary denoised image are restored layer by layer, and the denoised image is output.

5. The image denoising method according to claim 1, characterized in that: The dynamically adjusting the threshold parameter and the number of iterations of the diffusion coefficient function to meet the denoising requirements of different images and noise types, and outputting the optimized target denoised image, includes: Calculating local gradient information of the denoised image according to noise feature information in the denoised image, and dynamically adjusting a threshold parameter and the number of iterations of the diffusion coefficient function according to the local gradient information to meet denoising requirements of different noise types; According to the adjusted threshold parameter and the number of iterations, the parameters of the diffusion coefficient function model are updated, and the updated parameters are used to perform denoising processing in each iteration process until the denoising requirements of the different images are met, and the target denoised image is generated and output.

6. The image denoising method according to claim 1, characterized in that: The dynamically adjusting the threshold parameter and the number of iterations of the diffusion coefficient function also includes: The threshold parameter and the number of iterations are increased in a background area with relatively strong noise, and are decreased in an edge area with relatively weak noise.

7. The image denoising method according to claim 1, characterized in that: After outputting the optimized target denoised image, the method comprises: Upsampling the target denoised image to increase the image resolution to a preset target resolution by an interpolation method; Performing feature fusion on the upsampled target denoised image to improve detail information of the target denoised image; The target denoised image after feature fusion is reconstructed, the visual parameters in the image are restored, and a standard image is output.

8. An image denoising device, characterized in that: The device comprises: An image acquisition module, used for acquiring a noisy image; An image processing module, configured to perform preliminary denoising on the noisy image based on a diffusion coefficient function model to obtain a preliminary denoised image, wherein the diffusion coefficient function model suppresses noise and protects image edge information by calculating gradient information of the noisy image and adjusting a threshold parameter of the diffusion coefficient function; A denoising processing module, configured to perform a second denoising process on the preliminary denoised image using a denoising convolutional neural network, wherein the denoising convolutional neural network predicts a noise residual signal of the noisy image by a residual learning method and outputs a denoised image; The iterative optimization module is used to dynamically adjust the threshold parameter and the number of iterations of the diffusion coefficient function according to the characteristic information of the denoised image output by the denoising convolutional neural network to adapt to the noise characteristics of different images, and output the optimized target denoised image.

9. A computer device, characterized in that: The computer device comprises a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the image denoising method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer program is stored therein, and when the computer program is executed on a processor, the image denoising method according to any one of claims 1 to 7 is implemented.

Images