Image denoising method and device, equipment and storage medium
By using noise sensing network and inverse diffusion sampling in the image denoising method and combining implicit models for gradual noise reduction, the shortcomings of image denoising methods in the prior art in global noise capture and processing speed are solved, and high efficiency is achieved. Fast image denoising effect.
Patent Information
- Application Number
- CN202411677392.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-22
- Publication Date
- 2025-02-18
AI Technical Summary
The existing image noise reduction method has shortcomings in capturing global noise and processing speed. Convolutional neural networks cannot effectively capture long-distance pixel dependence. GAN networks have difficulties in capturing noise distribution, resulting in limited denoising performance and slow inference speed.
An image denoising method is adopted to combine inverse diffusion sampling and implicit model through a noise-sensing network to gradually remove noise and generate noise-free images. The specific steps include inputting the pre-installed noisy image into the noise sensing network, performing downsampling, vector transformation, relational modeling and upsampling to predict the noise of the image to be denoised, and then performing gradual noise reduction through an implicit diffusion acceleration algorithm.
It significantly improves the iterative inference speed of image denoising, and at the same time improves the denoising performance, which can effectively process noise of different complexities and generate high-quality noiseless images.
Smart Images
Figure CN119477743A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to an image denoising method, device, equipment and storage medium. Background Art
[0002] Existing image denoising methods are mainly based on CNN (Convolutional Neural Networks) and GAN (Generative Adversarial Networks). Convolutional neural networks mainly focus on feature extraction in local areas and cannot capture dependencies between distant pixels, resulting in limited modeling performance of noise-aware networks for global noise. They also require hundreds or even thousands of sampling steps, and are therefore slow during inference. GAN networks are good at learning image-to-image distributions, but have difficulty capturing noise distributions and are prone to mode collapse, which causes the network to tend to output a single noise and makes it difficult to model noises of different complexities, resulting in limited denoising performance. Summary of the invention
[0003] In view of this, the purpose of the present invention is to provide an image denoising method, device, equipment and storage medium, which can combine low-quality noisy images as conditional inputs, guide the model to gradually remove noise, generate noise-free images through reverse diffusion sampling iterations, and use implicit models to accelerate the sampling process, so that the iterative reasoning speed is significantly improved. The specific scheme is as follows:
[0004] In a first aspect, the present application discloses an image denoising method, which is applied to an image denoising system, comprising:
[0005] Inputting the preset noisy image and the initial image to be denoised as input images into the noise perception network;
[0006] Downsampling the input image through the encoding module in the noise perception network, performing vector conversion and relationship modeling on the first processed image obtained by encoding through the preset neural network in the noise perception network, and then upsampling the second processed image obtained after processing by the preset neural network through the decoding module to obtain the predicted noise corresponding to the image to be denoised;
[0007] The image to be denoised is gradually denoised based on a preset step size and the predicted noise using an implicit diffusion acceleration algorithm to obtain a target denoised image.
[0008] Optionally, the step of inputting the preset noisy image and the initial image to be denoised as input images into the noise perception network includes:
[0009] Add preset Gaussian noise to the initial noise-free image to obtain a noisy image, and record the number of noise additions;
[0010] The noisy image is used as the initial noise-free image for the next round, and the process is skipped to the step of adding a preset Gaussian noise to the initial noise-free image until the number of noise additions is greater than a preset threshold number of noise additions, so as to obtain a plurality of noisy images;
[0011] The noise-aware network to be trained is trained based on the plurality of noise images to obtain a noise-aware network.
[0012] Optionally, downsampling the input image through an encoding module in the noise-aware network includes:
[0013] Encoding the time step corresponding to the input image through the Time Embedding layer in the encoding module in the noise perception network to obtain encoded time step information;
[0014] The input image is normalized and feature extracted based on the encoded time step information to obtain a first processed image.
[0015] Optionally, the performing normalization processing and feature extraction processing on the input image based on the encoded time step information to obtain a first processed image includes:
[0016] The input image is normalized by the normalization layer in the encoding module to obtain a normalized image, and the normalized image is subjected to feature extraction by a preset activation function and a convolution layer in the encoding module to reduce the resolution of the input image to obtain a processed image;
[0017] The processed image is used as the current input image, and the step of encoding the time step corresponding to the input image through the Time Embedding layer in the encoding module in the noise perception network is jumped to, and the first processing times are recorded until the first processing times is greater than the preset processing times threshold, so as to obtain the first processed image.
[0018] Optionally, upsampling the second processed image obtained after being processed by a preset neural network through a decoding module to obtain a predicted noise currently corresponding to the image to be denoised includes:
[0019] The second processed image is upsampled by a decoding module in the noise perception network to perform noise prediction on the second processed image to obtain a predicted noise currently corresponding to the image to be denoised.
[0020] Optionally, the stepwise denoising the image to be denoised based on a preset step size and the predicted noise by using an implicit diffusion acceleration algorithm to obtain a target denoised image includes:
[0021] Denoising the image to be de-noised by using the implicit acceleration algorithm and according to the predicted noise, so as to obtain a de-noised image corresponding to a previous time step at the time step;
[0022] Noise prediction is performed on the denoised image to obtain the predicted noise of the denoised image at the corresponding time step, and the process jumps to the step of denoising the image to be denoised by using the implicit acceleration algorithm and according to the predicted noise, until the noise in the image to be denoised is eliminated to obtain the target denoised image.
[0023] Optionally, the implicit acceleration algorithm is: ;
[0024] Where s is the sampling sequence step length of the implicit diffusion acceleration algorithm, and The difference is S time steps, is the noise image at time t, For the preset noisy image, is the predicted noise corresponding to the noise image at time t, is the preset variance sequence.
[0025] In a second aspect, the present application discloses an image denoising device, which is applied to an image denoising system, comprising:
[0026] A data input module, used for inputting a preset noisy image and an initial image to be denoised as input images into a noise perception network;
[0027] A feature enhancement module, used to downsample the input image through the encoding module in the noise perception network, perform vector conversion and relationship modeling on the first processed image obtained by encoding through the preset neural network in the noise perception network, and then upsample the second processed image obtained after processing by the preset neural network through the decoding module to obtain the predicted noise corresponding to the image to be denoised;
[0028] The image denoising module is used to gradually denoise the image to be denoised based on a preset step size and the predicted noise by using an implicit diffusion acceleration algorithm to obtain a target denoised image.
[0029] In a third aspect, the present application discloses an electronic device, comprising:
[0030] Memory, used to store computer programs;
[0031] The processor is used to execute the computer program to implement the image denoising method as described above.
[0032] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the image denoising method as described above.
[0033] In the present application, the preset noisy image and the initial image to be denoised can be input as input images into the noise perception network; the input image is downsampled by the encoding module in the noise perception network respectively, and the first processed image obtained by encoding is vectorized and modeled by the preset neural network in the noise perception network, and then the second processed image obtained after processing by the preset neural network is upsampled by the decoding module to obtain the predicted noise corresponding to the image to be denoised; the image to be denoised is gradually denoised based on the preset step size and the predicted noise by the implicit diffusion acceleration algorithm to obtain the target denoised image. It can be seen that through the method of the present application, the preset noisy image and the initial image to be denoised can be input as input images into the noise perception network, so as to use the preset noisy image as a conditional input to guide the model to gradually predict and remove noise. After the input image is input into the noise perception network, the input image can be downsampled, vectorized, modeled and upsampled to predict the current noise corresponding to the initial image to be denoised, and finally the image to be denoised is gradually denoised according to the implicit acceleration algorithm and the predicted noise obtained to obtain a noise-free image corresponding to the initial image to be denoised. In this way, we can combine the preset noisy image as the conditional input and predict the noise distribution of the image, guiding the model to gradually predict and remove the noise. Through inverse diffusion sampling, we can iteratively generate high-quality noise-free images. And the implicit model is used to accelerate the sampling process, which significantly improves the iterative reasoning speed while ensuring the image quality. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0035] Figure 1 A flow chart of an image denoising method disclosed in this application;
[0036] Figure 2 A conditional diffusion model denoising framework diagram disclosed in this application;
[0037] Figure 3A noise estimation network framework diagram disclosed in this application;
[0038] Figure 4 This is a schematic diagram of the structure of an image denoising device disclosed in this application;
[0039] Figure 5 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION
[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0041] In the prior art, existing image denoising techniques mainly focus on feature extraction in local areas and are unable to capture dependencies between distant pixels, resulting in limited modeling performance of the noise-aware network for global noise, and a slow speed during the inference process. In addition, there are difficulties in capturing noise distribution and modeling noises of different complexities, resulting in limited denoising performance.
[0042] In order to overcome the above-mentioned technical problems, the present application discloses an image denoising method, apparatus, device and storage medium, which can combine a low-quality noisy image as a conditional input to guide the model to gradually remove noise, generate a noise-free image through inverse diffusion sampling iteration, and use an implicit model to accelerate the sampling process, so that the iterative reasoning speed is significantly improved.
[0043] See also Figure 1 As shown, an embodiment of the present invention discloses an image denoising method, which is applied to an image denoising system, comprising:
[0044] Step S11: input the preset noisy image and the initial image to be denoised as input images into the noise perception network.
[0045] In this embodiment, the diffusion model is used as the basic framework of image denoising. Figure 2The denoising process of the image shown is divided into a forward process and a reverse process, wherein the forward process is the process of adding noise to the initial high-quality noise-free image. Therefore, in the forward process, noise can be added to the initial noise-free image to obtain a noise image, and then the preset noisy image and the obtained noise image are input into the noise perception network. Specifically, preset Gaussian noise can be added to the initial noise-free image to obtain a noise image, and the number of noise additions is recorded; the noise image is used as the initial noise-free image of the next round, and jumps to the step of adding preset Gaussian noise to the initial noise-free image until the number of noise additions is greater than the preset noise addition threshold to obtain a number of noise images, and then the noise perception network to be trained is trained based on the several noise images to obtain a noise perception network, that is, the forward process gradually adds noise to the high-quality noise-free image, and finally converts it into a complete Gaussian noise image. In the forward process, Gaussian noise is gradually added to the high-quality noise-free image, and noise is added T times to generate a series of noisy images. ,… ,… . Add noise to During the process, the standard deviation / variance of the noise is a fixed value The mean is determined by a fixed value. and the current image data The formula for the noise adding process is as follows:
[0046] ;
[0047] ;
[0048] in, Represents the conditional probability distribution, which is from the previous moment To the current time The transition probability distribution of represents Gaussian distribution (normal distribution), and I is the unit matrix. As t increases, the initial noise-free image will gradually lose its characteristics. hour, Approaching an isotropic Gaussian distribution. According to the properties of Gaussian distribution, in the process of gradual noise addition, Can be and fixed value sequences Directly calculated, through the reparameterization technique, set , , you can get, In practice, the noisy image at time step t is generated as: , is the noise sampled from a standard normal distribution, is the original image. After obtaining the predicted noisy image at time t, a low-quality preset noisy image can be obtained, where low quality means that the image itself contains noise, and then the preset noisy image and the predicted noisy image are input as input images to the noise perception network. In this way, the preset noisy image can be combined as a conditional input to simulate the real environment and enhance the robustness of the algorithm.
[0049] Step S12, down-sampling the input image through the encoding module in the noise perception network, and performing vector conversion and relationship modeling on the first processed image obtained by encoding through the preset neural network in the noise perception network, and then up-sampling the second processed image obtained after processing by the preset neural network through the decoding module to obtain the current predicted noise corresponding to the image to be denoised.
[0050] In this embodiment, in the reverse process of image denoising, a noise-aware network based on Swim Transformer is used, wherein the noise-aware network introduces a hybrid encoder based on the Unet model, combines CNN and Swim Transformer, extracts local features through CNN, aggregates global features through Transformer, and can use a preset noisy image as a conditional input, gradually predict noise from the predicted noisy image, realize local and global modeling of complex noise, and then remove noise to achieve the purpose of image denoising. Among them, the input image can be first downsampled by the encoding module in the noise-aware network, specifically, as Figure 3As shown in the figure, four encoding modules can be used as feature extractors to extract the feature map of the input image. Each encoding module consists of a group normalization layer, a Swish activation function, a convolution layer, and a TimeEmbedding layer. The time step corresponding to the input image can be encoded by the Time Embedding layer in the encoding module in the noise perception network to obtain the encoded time step information. Specifically, in the diffusion model, the noise estimation network needs to accurately estimate the noise for each time step, but since the parameters of the network are shared across all time steps, this requires the network to be able to distinguish different time steps. Therefore, it is necessary to use Time Embedding to encode the time step, and finally embed the time step information into the encoding module after linear changes. Then, the input image is normalized by the normalization layer in the encoding module to obtain a normalized image, and the normalized image is feature extracted by the preset activation function in the encoding module, namely the Swish activation function, and the convolution layer to reduce the resolution of the input image and obtain a processed image; then the processed image is used as the current input image, and the step of encoding the time step corresponding to the input image by the Time Embedding layer in the encoding module in the noise perception network is jumped to, and the first processing times are recorded until the first processing times is greater than the preset processing times threshold, so as to obtain the first processed image. It should be noted that since four encoding modules are required as feature extractors to extract the feature map of the input image, the input image can be downsampled four times to reduce the resolution of the first processed image to 1 / 16 of the input image.
[0051] Furthermore, it is necessary to perform vector conversion and relationship modeling on the first processed image obtained by encoding through the preset neural network in the noise perception network, and upsample the second processed image obtained after processing by the preset neural network through the decoding module to obtain the current predicted noise corresponding to the image to be denoised. Specifically, it is necessary to convert the high-level feature image into a vector and input it into several Swim Transformer modules to obtain the self-attention result, that is, it is necessary to upsample the first processed image four times through the four decoding modules of the decoder part to enhance the features of the first processed image, and restore the resolution of the first processed image, and then perform noise prediction on the image after the current resolution is restored to obtain the current predicted noise corresponding to the image to be denoised. It should be noted that through the processing of the noise estimation network, the predicted noise of the current time step of the image to be denoised can be obtained.
[0052] Step S13: gradually denoise the image to be denoised based on a preset step size and the predicted noise using an implicit diffusion acceleration algorithm to obtain a target denoised image.
[0053] In this embodiment, it is necessary to gradually reduce the noise of the image to be denoised according to the predicted noise obtained by using an implicit diffusion acceleration algorithm to obtain a target denoised image. Specifically, the inverse process of denoising the image satisfies the following formula:
[0054] ;
[0055] ;
[0056] in is from a standard normal distribution A sampled Gaussian noise image. is the conditional probability distribution, which is given the current observation and input In the case of The conditional probability of is the noise variance, is the mean of the noise distribution. During training, the diffusion model fixes the variance , fitting the mean through a neural network :
[0057] ;
[0058] Since t and Known, so only the noise estimation network is needed to predict the noise distribution , first at time t from Prediction noise , and then through the reverse diffusion process Sampling can obtain samples of the target distribution and finally generate a noise-free high-quality image. Among them, the inverse diffusion process is processed by the implicit diffusion acceleration algorithm, and the implicit diffusion acceleration algorithm is publicized as follows:
[0059] ;
[0060] Where s is the sampling sequence step length of the implicit diffusion acceleration algorithm, and The difference is S time steps, is the noise image at time t, For the preset noisy image, The noise predicted by the noise estimation network.
[0061] Therefore, the image to be denoised can be denoised according to the predicted noise based on the preset time step by using an implicit acceleration algorithm to obtain a denoised image corresponding to the previous time step, and then the denoised image is predicted to obtain the predicted noise corresponding to the time step of the denoised image, and then jump to the step of denoising the image to be denoised by using the implicit acceleration algorithm and according to the predicted noise, until the noise in the image to be denoised is gradually eliminated to obtain the target denoised image. For example, if the preset time step is 100, the sampling sequence is 100, 200, 300, ... 900, 1000}, thereby effectively improving the sampling efficiency compared to the traditional DDPM (Denoising Diffusion Probabilistic Model) which needs to sample 1000 times at the same step length. In this way, in the reverse process, a noise estimation network noise perception network combining CNN and Swim Transformer is designed to overcome the limitations of traditional convolutional neural networks in capturing long-distance pixel dependencies and processing full-size images, and generate data with a distribution similar to the original noise. At the same time, in order to significantly increase the sampling speed, an implicit diffusion acceleration algorithm is adopted in the inference stage.
[0062] It can be seen that through the method of the present application, the preset noisy image and the initial image to be denoised can be input as input images into the noise perception network, so that the preset noisy image can be used as a conditional input to guide the model to gradually predict and remove noise. After the input image is input into the noise perception network, the input image can be downsampled, vector converted, relationally modeled, and upsampled to predict the current noise corresponding to the initial image to be denoised. Finally, the image to be denoised is gradually denoised according to the implicit acceleration algorithm and the predicted noise obtained to obtain a noise-free image corresponding to the initial image to be denoised. In this way, the preset noisy image can be combined as a conditional input, and the noise distribution of the image can be predicted to guide the model to gradually predict and remove noise. Through inverse diffusion sampling, high-quality noise-free images can be iteratively generated. In addition, an implicit model is used to accelerate the sampling process, which significantly improves the iterative reasoning speed while ensuring image quality.
[0063] See also Figure 4 As shown, an embodiment of the present invention discloses an image denoising device, which is applied to an image denoising system, comprising:
[0064] A data input module 11 is used to input a preset noisy image and an initial image to be denoised as input images into a noise perception network;
[0065] The feature enhancement module 12 is used to downsample the input image through the encoding module in the noise perception network, perform vector conversion and relationship modeling on the first processed image obtained by encoding through the preset neural network in the noise perception network, and then upsample the second processed image obtained after processing by the preset neural network through the decoding module to obtain the predicted noise corresponding to the image to be denoised;
[0066] The image denoising module 13 is used to gradually denoise the image to be denoised based on a preset step size and the predicted noise by using an implicit diffusion acceleration algorithm to obtain a target denoised image.
[0067] In this embodiment, the preset noisy image and the initial image to be denoised can be input as input images to the noise perception network; the input image is downsampled by the encoding module in the noise perception network respectively, and the first processed image obtained by encoding is vectorized and modeled by the preset neural network in the noise perception network, and then the second processed image obtained after processing by the preset neural network is upsampled by the decoding module to obtain the predicted noise corresponding to the image to be denoised; the image to be denoised is gradually denoised based on the preset step size and the predicted noise by the implicit diffusion acceleration algorithm to obtain the target denoised image. It can be seen that through the method of the present application, the preset noisy image and the initial image to be denoised can be input as input images to the noise perception network, so as to use the preset noisy image as a conditional input to guide the model to gradually predict and remove noise. After the input image is input into the noise perception network, the input image can be downsampled, vectorized, modeled and upsampled to predict the current noise corresponding to the initial image to be denoised, and finally the image to be denoised is gradually denoised according to the implicit acceleration algorithm and the predicted noise obtained to obtain a noise-free image corresponding to the initial image to be denoised. In this way, we can combine the preset noisy image as the conditional input and predict the noise distribution of the image, guiding the model to gradually predict and remove the noise. Through inverse diffusion sampling, we can iteratively generate high-quality noise-free images. And the implicit model is used to accelerate the sampling process, which significantly improves the iterative reasoning speed while ensuring the image quality.
[0068] In some embodiments, the data input module 11 may specifically include:
[0069] A first noise adding unit, used for adding a preset Gaussian noise to the initial noise-free image to obtain a noisy image, and recording the number of noise additions;
[0070] A second noise adding unit is used to use the noise image as the next round of initial noise-free image, and jump to the step of adding preset Gaussian noise to the initial noise-free image until the number of noise additions is greater than a preset noise addition number threshold, so as to obtain a plurality of noise images;
[0071] The image input unit is used to train the noise-aware network to be trained based on the plurality of noise images to obtain the noise-aware network.
[0072] In some embodiments, the feature enhancement module 12 may specifically include:
[0073] An image encoding unit, used for encoding the time step corresponding to the input image through the Time Embedding layer in the encoding module in the noise perception network to obtain encoded time step information;
[0074] The image normalization submodule is used to perform normalization processing and feature extraction processing on the input image based on the encoded time step information to obtain a first processed image.
[0075] In some embodiments, the image normalization submodule may specifically include:
[0076] An image normalization unit, configured to perform normalization processing on the input image through the normalization layer in the encoding module to obtain a normalized image, and perform feature extraction processing on the normalized image through a preset activation function and a convolution layer in the encoding module to reduce the resolution of the input image to obtain a processed image;
[0077] The first loop processing unit is used to use the processed image as the current input image, and jump to the step of encoding the time step corresponding to the input image through the Time Embedding layer in the encoding module in the noise perception network, and record the first processing times until the first processing times is greater than a preset processing times threshold to obtain a first processed image.
[0078] In some embodiments, the feature enhancement module 12 may specifically include:
[0079] The feature enhancement unit is used to upsample the second processed image through the decoding module in the noise perception network to perform noise prediction on the second processed image to obtain the predicted noise currently corresponding to the image to be denoised.
[0080] In some embodiments, the image noise reduction module 13 may specifically include:
[0081] An image denoising unit, configured to denoise the image to be denoised by using the implicit acceleration algorithm and according to the predicted noise, so as to obtain a denoised image corresponding to a previous time step at the time step;
[0082] The second loop processing unit is used to perform noise prediction on the denoised image to obtain the predicted noise of the denoised image at the corresponding time step, and jump to the step of denoising the image to be denoised by using the implicit acceleration algorithm and according to the predicted noise, until the noise in the image to be denoised is eliminated to obtain the target denoised image.
[0083] In some embodiments, the implicit acceleration algorithm is: ;
[0084] Where s is the sampling sequence step length of the implicit diffusion acceleration algorithm, and The difference is S time steps, is the noise image at time t, For the preset noisy image, is the predicted noise corresponding to the noise image at time t, is the preset variance sequence.
[0085] Furthermore, the present application also discloses an electronic device. Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.
[0086] Figure 5 The present invention provides a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the image denoising method disclosed in any of the above embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0087] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.
[0088] In addition, the memory 22 as a carrier for resource storage may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.
[0089] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the image denoising method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.
[0090] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the image denoising method disclosed above. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the above embodiments, and no further description will be given here.
[0091] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0092] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0093] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0094] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0095] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. An image denoising method, characterized in that: Applied to image denoising systems, including: Inputting the preset noisy image and the initial image to be denoised as input images into the noise perception network; Downsampling the input image through the encoding module in the noise perception network, performing vector conversion and relationship modeling on the first processed image obtained through encoding through the preset neural network in the noise perception network, and then upsampling the second processed image obtained after processing by the preset neural network through the decoding module to obtain the predicted noise corresponding to the image to be denoised; The image to be denoised is gradually denoised based on a preset step size and the predicted noise using an implicit diffusion acceleration algorithm to obtain a target denoised image.
2. The image denoising method according to claim 1, characterized in that: Before inputting the preset noisy image and the initial image to be denoised as input images into the noise perception network, the method further includes: Add preset Gaussian noise to the initial noise-free image to obtain a noisy image, and record the number of noise additions; The noisy image is used as the initial noise-free image for the next round, and the process is skipped to the step of adding a preset Gaussian noise to the initial noise-free image until the number of noise additions is greater than a preset threshold number of noise additions, so as to obtain a plurality of noisy images; The noise-aware network to be trained is trained based on the plurality of noise images to obtain a noise-aware network.
3. The image denoising method according to claim 1, characterized in that: The downsampling the input image by the encoding module in the noise perception network includes: Encoding the time step corresponding to the input image through the Time Embedding layer in the encoding module in the noise perception network to obtain encoded time step information; The input image is normalized and feature extracted based on the encoded time step information to obtain a first processed image.
4. The image denoising method according to claim 3, characterized in that: The step of performing normalization and feature extraction on the input image based on the encoded time step information to obtain a first processed image includes: The input image is normalized by the normalization layer in the encoding module to obtain a normalized image, and the normalized image is subjected to feature extraction by a preset activation function and a convolution layer in the encoding module to reduce the resolution of the input image to obtain a processed image; The processed image is used as the current input image, and the step of encoding the time step corresponding to the input image through the Time Embedding layer in the encoding module in the noise perception network is jumped to, and the first processing times are recorded until the first processing times is greater than the preset processing times threshold, so as to obtain the first processed image.
5. The image denoising method according to claim 1, characterized in that: The up-sampling of the second processed image obtained after the preset neural network processing by the decoding module to obtain the predicted noise corresponding to the image to be denoised currently includes: The second processed image is upsampled by a decoding module in the noise perception network to perform noise prediction on the second processed image to obtain a predicted noise currently corresponding to the image to be denoised.
6. The image denoising method according to any one of claims 1 to 5, characterized in that: The stepwise denoising of the image to be denoised based on a preset step size and the predicted noise by using an implicit diffusion acceleration algorithm to obtain a target denoised image includes: Denoising the image to be de-noised by using the implicit acceleration algorithm and according to the predicted noise, so as to obtain a de-noised image corresponding to a previous time step at the time step; Noise prediction is performed on the denoised image to obtain the predicted noise of the denoised image at the corresponding time step, and the process jumps to the step of denoising the image to be denoised by using the implicit acceleration algorithm and according to the predicted noise, until the noise in the image to be denoised is eliminated to obtain the target denoised image.
7. The image denoising method according to claim 6, characterized in that: The implicit acceleration algorithm is: ； Where s is the sampling sequence step length of the implicit diffusion acceleration algorithm, and The difference is S time steps, is the noise image at time t, For the preset noisy image, is the predicted noise corresponding to the noise image at time t, is the preset variance sequence.
8. An image denoising device, characterized in that: Applied to image denoising systems, including: A data input module, used for inputting a preset noisy image and an initial image to be denoised as input images into a noise perception network; A feature enhancement module, used to downsample the input image through the encoding module in the noise perception network, perform vector conversion and relationship modeling on the first processed image obtained by encoding through the preset neural network in the noise perception network, and then upsample the second processed image obtained after processing by the preset neural network through the decoding module to obtain the predicted noise corresponding to the image to be denoised; The image denoising module is used to gradually denoise the image to be denoised based on a preset step size and the predicted noise by using an implicit diffusion acceleration algorithm to obtain a target denoised image.
9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the image denoising method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the image denoising method according to any one of claims 1 to 7 is implemented.