A lightweight single-image denoising method and device based on a learning model

Through the lightweight single-frame graphics noise reduction method based on learning model, the downsampling module and residual neural network are used to perform adaptive training on images, which solves the problem that the image noise reduction method based on deep learning requires a large number of samples to be trained, and realizes the problem of the image denoising method based on deep learning. Efficient image noise reduction effect is especially suitable for scenarios that lack a large amount of training data.

CN118644416BActive Publication Date: 2024-12-13SUZHOU CITY UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411129405.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-16
Publication Date
2024-12-13
Estimated Expiration
2044-08-16

AI Technical Summary

Technical Problem

Existing image denoising methods based on deep learning require a large number of clean and noisy image samples for model training, resulting in high cost and low noise reduction efficiency, especially in application scenarios where a large amount of training data is lacking, it is difficult to achieve efficient image noise reduction.

Method used

Using a lightweight single-frame graph noise reduction method based on learning model, the image noise reduction model is constructed by constructing a downsampling module and a residual neural network, and the downsampling noise image data set and the initial noise reduction image data set are used to construct a loss function to perform the image noise reduction model. Optimize training to implement adaptive training without relying on a large number of marked specific types of training samples.

Benefits of technology

This method effectively reduces the training cost of image noise reduction model, improves the efficiency of image noise reduction, is suitable for deployment in application scenarios that lack a large amount of training data, and has better generalization capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118644416B_ABST
    Figure CN118644416B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of image processing technology, and in particular to a lightweight single-image denoising method and device based on a learning model. The present invention uses a downsampling module to divide an original image into a downsampled noise image data set, and divides the initial denoised image obtained after the original image is denoised by an image denoising model into a downsampled initial denoised image data set, and denoises the downsampled noise image data set by the image denoising model to obtain a sampled denoised image data set; based on the downsampled noise image sub-data set, the downsampled initial denoised image data set and the downsampled denoised image data set, a loss function of the image denoising model is constructed, and the image denoising model is trained, so that the training of the image denoising model does not need to rely on a large number of labeled training samples of a specific type, and only a single original image can be used to achieve adaptive training of the image denoising model, which effectively reduces the training cost of the image denoising model and improves the efficiency of image denoising.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a lightweight single-image denoising method and device based on a learning model. Background Art

[0002] Image denoising is a key technology in the field of computer vision and image processing, and its purpose is to restore high-quality images from noisy images. Since distinguishing fine textures from noisy images is extremely important for target recognition and defect detection, noise is an inevitable factor in the process of image acquisition and transmission, which will seriously affect the visual quality of the image. Therefore, in various image processing tasks, noise removal is very important, and image denoising has always been a research hotspot in the field of image processing.

[0003] Noise in images can come from a variety of different sources, including sensor limitations, environmental conditions, and artifacts generated during transmission. Common types of noise include Gaussian noise and Poisson noise; Gaussian noise is additive and follows a Gaussian distribution; Poisson noise arises from the photon counting process and is typically encountered under low-light imaging conditions. The presence of noise degrades image quality, reduces the peak signal-to-noise ratio (PSNR), and hinders the performance of subsequent image analysis tasks such as object detection, recognition, and segmentation.

[0004] Traditional image denoising methods usually subtract the noise part from the original acquired image to obtain a noise-free image. The main techniques include spatial filters, such as Gaussian smoothing, median filtering, and Wiener filtering, which operate on pixel intensity or local neighborhoods to reduce noise while preserving image details. Since traditional methods are not efficient and applicable in dealing with complex noise patterns or preserving fine details of high-resolution images, with the rapid development of artificial intelligence technology in recent years, neural networks and deep learning technologies have also been introduced into image denoising tasks. Denoising methods based on deep learning are able to leverage the ability of convolutional neural networks (CNNs) to learn hierarchical representations directly from data, and have shown superior performance compared to traditional methods, especially when the noise features are complex and the availability of labeled data is limited. Denoising methods based on deep learning can effectively learn and distinguish between signals and noise in images by training on large datasets, enabling them to generalize well to unseen image data and noise types.

[0005] However, deep learning-based denoising methods require a large number of clean and noisy image samples for model training, which is impractical or costly in some application areas, such as medical imaging, remote sensing, or historical document restoration. Although some researchers have proposed self-supervised learning methods that do not require a large number of clean images as training samples, this method still has shortcomings such as high computational cost and low denoising efficiency. Summary of the invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the problem that the existing deep learning-based denoising method requires a large number of samples to train the deep learning model, resulting in high cost and low denoising efficiency.

[0007] In order to solve the above technical problems, the present invention provides a lightweight single-image denoising method based on a learning model, comprising:

[0008] Construct a downsampling module, which includes three pairs of downsamplers, each pair of downsamplers has the same convolution kernel size, a main diagonal matrix and a sub-diagonal matrix, and satisfies ,in represents the element in the mth row and mth column of the main diagonal matrix, represents the element in the mth row and (n-m+1)th column in the secondary diagonal matrix, m=1,2,…,n, where n is the order of the primary diagonal matrix; each downsampler performs a downsampling operation on the input image, and outputs a downsampled sub-dataset corresponding to each downsampler; all downsampler sub-datasets constitute a downsampled data set;

[0009] Input the original image into the downsampling module and output the downsampled noisy image dataset;

[0010] An image denoising model is constructed based on a residual neural network, a downsampled noisy image dataset is input into the image denoising model to extract noise, and the noise corresponding to the extracted noise of the image denoising model is subtracted from the image in the downsampled noisy image dataset to obtain a downsampled denoised image dataset;

[0011] The original image is input into the image denoising model to extract the noise, and the noise corresponding to the image denoising model is subtracted from the original image to obtain the initial denoised image; the initial denoised image is input into the downsampling module, and the downsampled initial denoised image data set is output;

[0012] A loss function of an image denoising model is constructed based on a downsampled noisy image dataset, a downsampled denoised image dataset and a downsampled initial denoised image dataset, and the network parameters of the image denoising model are optimized to obtain a trained image denoising model;

[0013] The original image is input into the trained image denoising model to obtain the target denoised image.

[0014] Preferably, in the down-sampling module, the convolution kernels of the six down-samplers are:

[0015] ;

[0016] ;

[0017] ;

[0018] ;

[0019] ;

[0020] ;

[0021] in, is the convolution kernel of the first downsampler, is the convolution kernel of the second downsampler, is the convolution kernel of the third downsampler, is the convolution kernel of the fourth downsampler, is the convolution kernel of the fifth downsampler, is the convolution kernel of the sixth downsampler.

[0022] Preferably, each downsampler performs a downsampling operation on the input image, and outputs a downsampled sub-data set corresponding to each downsampler, including:

[0023] Each downsampler starts from the upper left corner of the input image, selects an area with a length and width twice the size of the convolution kernel for non-overlapping downsampling, fills the insufficient edges of the input image with zeros, and obtains several downsampled sub-images after traversing the entire input image, and outputs the corresponding downsampled sub-dataset.

[0024] Preferably, the downsampled noisy image dataset is input into the image denoising model to extract noise, and the noise corresponding to the extracted noise of the image denoising model is subtracted from the image in the downsampled noisy image dataset to obtain the downsampled denoised image dataset, and the formula is:

[0025] ;

[0026] in, is the original image, is the i-th down-sampled denoised image sub-dataset, is the i-th downsampled noise image sub-dataset, i=1,2,…,6, It is an image denoising model;

[0027] All downsampled noisy image sub-datasets constitute the downsampled noisy image dataset, and the formula is:

[0028] ;

[0029] in, It is a downsampled noisy image dataset;

[0030] All downsampled denoised image sub-datasets constitute the downsampled denoised image dataset, and the formula is:

[0031] ;

[0032] in, It is a downsampled denoised image dataset.

[0033] Preferably, the loss function of the image denoising model includes residual loss, regularization loss and guidance loss, and the formula is:

[0034] ;

[0035] in, is the loss function of the image denoising model, is the residual loss, is the regularization loss, To guide the loss, , and are scalar regularization parameters.

[0036] Preferably, the calculation formula of the residual loss is:

[0037] ;

[0038] in, represents the residual loss, and denote the jth and j+1th downsampled noise image sub-datasets respectively, and They represent the j+1th and jth downsampled denoised image sub-datasets respectively.

[0039] Preferably, the calculation formula of the regularization loss is:

[0040] ;

[0041] in, is the regularization loss, is the i-th downsampled initial denoised image sub-dataset, is the i-th downsampled denoised image sub-dataset, i=1,2,…,6;

[0042] All downsampled initial denoised image sub-datasets constitute the downsampled initial denoised image dataset, and the formula is:

[0043] ;

[0044] in, is the downsampled initial denoised image dataset.

[0045] Preferably, the calculation formula of the guidance loss is:

[0046] ;

[0047] in, To guide the loss, is the i-th down-sampled denoised image sub-dataset, is the i-th downsampled initial denoised image sub-dataset The low-frequency image sub-dataset obtained by passing through a Gaussian low-pass filter.

[0048] Preferably, the i-th downsampled initial denoised image sub-dataset The low-frequency image sub-dataset obtained by Gaussian low-pass filter is as follows:

[0049] ;

[0050] in, represents a Gaussian low-pass filter, Represents the parameters of the Gaussian filter function in the Gaussian low-pass filter.

[0051] The present invention also provides a lightweight single-image denoising device based on a learning model, comprising:

[0052] The downsampling module includes three pairs of downsamplers. The convolution kernels of each pair of downsamplers have the same size, which are a main diagonal matrix and a sub-diagonal matrix respectively, and satisfy ,in represents the element in the mth row and mth column of the main diagonal matrix, represents the element in the mth row and (n-m+1)th column in the secondary diagonal matrix, m=1,2,…,n, where n is the order of the primary diagonal matrix; each downsampler performs a downsampling operation on the input image, and outputs a downsampled sub-dataset corresponding to each downsampler; all downsampler sub-datasets constitute a downsampled data set;

[0053] A down-sampled noise image acquisition module is used to input the original image into the down-sampled module and output a down-sampled noise image data set;

[0054] The down-sampled denoised image acquisition module is used to construct an image denoising model based on a residual neural network, input the down-sampled noisy image data set into the image denoising model to extract noise, and subtract the noise corresponding to the image denoising model from the image in the down-sampled noisy image data set to obtain the down-sampled denoised image data set;

[0055] The down-sampled initial denoised image acquisition module is used to input the original image into the image denoising model to extract noise, and subtract the noise corresponding to the image denoising model from the original image to obtain the initial denoised image; the initial denoised image is input into the down-sampling module, and the down-sampled initial denoised image data set is output;

[0056] A loss function construction module is used to construct a loss function of an image denoising model based on a downsampled noisy image sub-dataset, a downsampled initial denoised image data set and a downsampled denoised image data set, optimize the network parameters of the image denoising model, and obtain a trained image denoising model;

[0057] The denoising module is used to input the original image into the trained image denoising model to obtain the target denoised image.

[0058] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0059] The lightweight single-image denoising method based on a learning model described in the present invention downsamples the original image through a downsampling module to obtain a downsampled noise image data set; the downsampled noise image data set is denoised by an image denoising model to obtain a downsampled denoised image data set; after the original image is denoised by the image denoising model, the initial denoised image is downsampled by a downsampling module to obtain a downsampled initial denoised image data set; based on the downsampled noise image subdataset, the downsampled initial denoised image data set and the downsampled denoised image data set, a loss function of the image denoising model is constructed, and the image denoising model is trained, so that the training of the image denoising model does not need to rely on a large number of labeled training samples of a specific type, and only a single original image can be used to achieve adaptive training of the image denoising model. The present invention has strong adaptability and better generalization ability, and is particularly suitable for deployment in application scenarios lacking a large amount of training data, effectively reducing the training cost of the image denoising model and improving the efficiency of image denoising. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0061] Figure 1 It is a flow chart of a lightweight single-image denoising method based on a learning model of the present invention;

[0062] Figure 2 is a schematic diagram of a first downsampler and a second downsampler in a downsampling module generating a downsampled sub-image;

[0063] Figure 3 is a comparison diagram of the noise reduction effect of the method of the present invention, wherein Figure 3(a) in the figure is the first original image. Figure 3 (b) is a first target denoised image obtained after the first original image is denoised by the method of the present invention; Figure 3 (c) in the figure is the second original image. Figure 3 (d) is a second target denoised image obtained after the second original image is denoised by the method of the present invention. DETAILED DESCRIPTION

[0064] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0065] Reference Figure 1 As shown, the present invention provides a lightweight single-image denoising method based on a learning model, the steps comprising:

[0066] S1. Construct a downsampling module, which includes three pairs of downsamplers. The convolution kernels of each pair of downsamplers are of the same size, namely, a main diagonal matrix and a sub-diagonal matrix, and satisfy ,in represents the element in the mth row and mth column of the main diagonal matrix, Represents the element in the mth row and (n-m+1)th column in the secondary diagonal matrix, m=1,2,…,n, n is the order of the main diagonal matrix; each downsampler performs a downsampling operation on the input image and outputs a downsampled sub-dataset corresponding to each downsampler; all downsampler sub-datasets constitute a downsampled dataset.

[0067] This convolution kernel construction method for the downsampler allows each pair of downsamplers to retain the complete noise characteristics in the input image while performing fast convolution.

[0068] Preferably, in the down-sampling module constructed in this embodiment, the convolution kernel of each down-sampler is as follows:

[0069] ;

[0070] ;

[0071] ;

[0072] ;

[0073] ;

[0074] ;

[0075] in, is the convolution kernel of the first downsampler, is the convolution kernel of the second downsampler, is the convolution kernel of the third downsampler, is the convolution kernel of the fourth downsampler, is the convolution kernel of the fifth downsampler, is the convolution kernel of the sixth downsampler. and The size of is 3×3 in order to preserve multi-scale image features.

[0076] Preferably, each of the downsamplers performs a downsampling operation on the input image, and outputs a downsampled sub-data set corresponding to each downsampler, respectively, including:

[0077] Each downsampler starts from the upper left corner of the input image, selects an area with a length and width twice the size of the convolution kernel for non-overlapping downsampling, fills the insufficient edges of the input image with zeros, and obtains several downsampled sub-images after traversing the entire input image, and outputs the corresponding downsampled sub-dataset.

[0078] Figure 2 Schematic diagram of the first downsampler and the second downsampler in the downsampler module generating downsampled sub-images. The downsampling operation of each downsampler is performed on all channels h, w, c of the input image, where h is the height of the input image, w is the width of the input image, and c is the number of channels of the input image.

[0079] S2. Input the original image into the downsampling module and output the downsampled noisy image dataset.

[0080] The original image is input into the downsampling module, and each downsampler outputs a downsampled noise image sub-dataset. The specific formula is:

[0081] ;

[0082] Where i=1,2,…6, is the original image, is the convolution kernel of the i-th downsampler, is the downsampled noise image sub-dataset output by the i-th downsampler, and * is the convolution operator.

[0083] All downsampled noisy image sub-datasets constitute the downsampled noisy image dataset, and the formula is:

[0084] ;

[0085] in, It is a downsampled noisy image dataset.

[0086] Since the noise pattern in the original image is unstructured and operates independently, the downsampled noise sub-images obtained after the original image passes through the downsampling module are multiple images that retain similar signal content but exhibit independent noise patterns.

[0087] S3, build an image denoising model based on the residual neural network, input the downsampled noisy image dataset into the image denoising model to extract the noise, and subtract the corresponding noise extracted by the image denoising model from the image in the downsampled noisy image dataset to obtain the downsampled denoised image dataset. The formula is:

[0088] ;

[0089] in, is the i-th down-sampled denoised image sub-dataset, is the i-th downsampled noise image sub-dataset, It is an image denoising model.

[0090] All downsampled denoised image sub-datasets constitute the downsampled denoised image dataset, and the formula is:

[0091] ;

[0092] in, It is a downsampled denoised image dataset.

[0093] S4. Input the original image into the image denoising model to extract noise, subtract the corresponding noise extracted by the image denoising model from the original image to obtain an initial denoised image; input the initial denoised image into the downsampling module, and output the downsampled initial denoised image data set.

[0094] The original image is input into the image denoising model to extract the noise, and the noise corresponding to the image denoising model is subtracted from the original image to obtain the initial denoised image. The formula is expressed as:

[0095] ;

[0096] in, is the initial denoised image.

[0097] The initial denoised image is input into the downsampling module, and each downsampler performs a downsampling operation on the initial denoised image and outputs a downsampled initial denoised image sub-dataset respectively.

[0098] All downsampled initial denoised image sub-datasets constitute the downsampled initial denoised image dataset, and the formula is:

[0099] ;

[0100] in, is the downsampled initial denoised image dataset, is the i-th downsampled initial denoised image sub-dataset, i=1,2,…,6.

[0101] S5. Based on the downsampled noisy image sub-dataset, the downsampled initial denoised image data set and the downsampled denoised image data set, a loss function of the image denoising model is constructed, and the network parameters of the image denoising model are optimized to obtain a trained image denoising model.

[0102] The loss function of the image denoising model includes residual loss, regularization loss and guidance loss, which are described in detail below.

[0103] The residual loss is calculated as a symmetric loss by evaluating the mean squared error between each downsampled noisy image sub-dataset and the downsampled denoised image sub-dataset image, as follows:

[0104] ;

[0105] in, represents the residual loss, and denote the jth and j+1th downsampled noise image sub-datasets respectively, and They represent the j+1th and jth downsampled denoised image sub-datasets respectively.

[0106] Regularization loss is used to prevent model overfitting and promote the generalization ability of the model. The regularization loss is calculated based on the downsampled initial denoised image dataset and the downsampled denoised image dataset. The formula is:

[0107] ;

[0108] in, is the regularization loss, is the i-th downsampled initial denoised image sub-dataset, is the i-th down-sampled denoised image sub-dataset.

[0109] The guidance loss further reduces the high-frequency noise by retaining the necessary low-frequency information. The calculation method includes:

[0110] The downsampled initial denoised image dataset is input into the Gaussian low-pass filter to obtain a low-frequency image dataset. The low-frequency image sub-dataset obtained by Gaussian low-pass filter is as follows:

[0111] ;

[0112] in, represents a Gaussian low-pass filter, Represents the parameters of the Gaussian filter function in the Gaussian low-pass filter, is the i-th downsampled initial denoised image sub-dataset The low-frequency image sub-dataset obtained by passing through a Gaussian low-pass filter.

[0113] Low-frequency image dataset .

[0114] The guidance loss is calculated based on the low-frequency image dataset and the downsampled denoised image dataset. The formula is:

[0115] ;

[0116] in, To guide the loss, is the i-th down-sampled denoised image sub-dataset.

[0117] The loss function of the image denoising model is:

[0118] ;

[0119] in, is the loss function of the image denoising model, , and are scalar regularization parameters.

[0120] The image denoising model constructed by the method of the present invention involves about 28,000 network parameters and has a light computational burden.

[0121] S6. Input the original image into the trained image denoising model to obtain a target denoised image.

[0122] In summary, the present invention provides a lightweight single-image denoising method based on a learning model, which downsamples the original image through a downsampling module to obtain a downsampled noise image data set; the downsampled noise image data set is denoised by an image denoising model to obtain a downsampled denoised image data set; after the original image is denoised by the image denoising model, the initial denoised image is downsampled by a downsampling module to obtain a downsampled initial denoised image data set; based on the downsampled noise image sub-dataset, the downsampled initial denoised image data set and the downsampled denoised image data set, the loss function of the image denoising model is constructed, and the image denoising model is trained, so that the training of the image denoising model does not need to rely on a large number of labeled training samples of a specific type, and only a single original image can be used to realize the adaptive training of the image denoising model. The present invention has strong adaptability and better generalization ability, and is particularly suitable for deployment in application scenarios that lack a large amount of training data, effectively reducing the training cost of the image denoising model and improving the efficiency of image denoising.

[0123] Figure 3 is a comparison diagram of the noise reduction effect of the method of the present invention, wherein Figure 3 (a) in the figure is the first original image. Figure 3 (b) is a first target denoised image obtained after the first original image is denoised by the method of the present invention; Figure 3 (c) in the figure is the second original image. Figure 3 (d) is a second target denoised image obtained after the second original image is denoised by the method of the present invention.

[0124] Based on the above-mentioned lightweight single-image denoising method based on a learning model, this embodiment further provides a lightweight single-image denoising device based on a learning model, including:

[0125] The downsampling module includes three pairs of downsamplers. The convolution kernels of each pair of downsamplers have the same size, which are a main diagonal matrix and a sub-diagonal matrix respectively, and satisfy ,in represents the element in the mth row and mth column of the main diagonal matrix, represents the element in the mth row and (n-m+1)th column in the secondary diagonal matrix, m=1,2,…,n, where n is the order of the primary diagonal matrix; each downsampler performs a downsampling operation on the input image, and outputs a downsampled sub-dataset corresponding to each downsampler; all downsampler sub-datasets constitute a downsampled data set;

[0126] A down-sampled noise image acquisition module is used to input the original image into the down-sampled module and output a down-sampled noise image data set;

[0127] The down-sampled denoised image acquisition module is used to construct an image denoising model based on a residual neural network, input the down-sampled noisy image data set into the image denoising model to extract noise, and subtract the noise corresponding to the image denoising model from the image in the down-sampled noisy image data set to obtain the down-sampled denoised image data set;

[0128] The down-sampled initial denoised image acquisition module is used to input the original image into the image denoising model to extract noise, and subtract the noise corresponding to the image denoising model from the original image to obtain the initial denoised image; the initial denoised image is input into the down-sampling module, and the down-sampled initial denoised image data set is output;

[0129] A loss function construction module is used to construct a loss function of an image denoising model based on a downsampled noisy image sub-dataset, a downsampled initial denoised image data set and a downsampled denoised image data set, optimize the network parameters of the image denoising model, and obtain a trained image denoising model;

[0130] The denoising module is used to input the original image into the trained image denoising model to obtain the target denoised image.

[0131] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0135] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A lightweight single-image denoising method based on a learning model, characterized in that: include: Construct a downsampling module, which includes three pairs of downsamplers, each pair of downsamplers has the same convolution kernel size, a main diagonal matrix and a sub-diagonal matrix, and satisfies A mm =B m(n-m+1) , where A mm represents the element in the mth row and mth column of the main diagonal matrix, B m(n-m+1) represents the element in the mth row and the (n-m+1)th column in the secondary diagonal matrix, m=1,2,…,n, where n is the order of the primary diagonal matrix; each downsampler performs a downsampling operation on the input image, and outputs a downsampled subdata set corresponding to each downsampler; all downsampler subdata sets constitute a downsampled data set; Input the original image into the downsampling module and output the downsampled noisy image dataset; An image denoising model is constructed based on a residual neural network, a downsampled noisy image dataset is input into the image denoising model to extract noise, and the noise corresponding to the extracted noise of the image denoising model is subtracted from the image in the downsampled noisy image dataset to obtain a downsampled denoised image dataset; The original image is input into the image denoising model to extract the noise, and the noise corresponding to the extracted noise of the image denoising model is subtracted from the original image to obtain the initial denoised image; Input the initial denoised image into the downsampling module, and output the downsampled initial denoised image dataset; A loss function of an image denoising model is constructed based on a downsampled noisy image dataset, a downsampled denoised image dataset and a downsampled initial denoised image dataset, and a network parameter of the image denoising model is optimized to obtain a trained image denoising model; the loss function of the image denoising model includes a residual loss, a regularization loss and a guided loss, The formula is: loss final =αloss res +βloss reg +γloss g Among them, loss final is the loss function of the image denoising model, loss res is the residual loss, loss reg is the regularization loss, loss g is the guided loss, α, β and γ are scalar regularization parameters; The calculation formula of the residual loss is: Among them, loss res represents the residual loss, d j (I) and d j+1 (I) represents the jth and j+1th downsampled noise image sub-datasets, respectively, S Ej+1 and S Ej denote the j+1th and jth downsampled denoised image sub-datasets respectively; all downsampled noisy image sub-datasets constitute a downsampled noisy image dataset, and all downsampled denoised image sub-datasets constitute a downsampled denoised image dataset; The calculation formula of the regularization loss is: Among them, loss reg is the regularization loss, d i (p) is the i-th downsampled initial denoised image sub-dataset, S Ei is the i-th downsampled denoised image sub-dataset, i=1,2,…,6; all downsampled initial denoised image sub-datasets constitute the downsampled initial denoised image dataset; The calculation formula of the guidance loss is: Among them, loss g is the guidance loss, S Ei is the i-th downsampled denoised image sub-dataset, d i (f) is the i-th downsampled initial denoised image sub-dataset d i (p) Low-frequency image sub-dataset obtained by Gaussian low-pass filter; The original image is input into the trained image denoising model to obtain the target denoised image.

2. A lightweight single-image denoising method based on a learning model according to claim 1, characterized in that: In the downsampling module, the convolution kernels of the six downsamplers are: Among them, k1 is the convolution kernel of the first downsampler, k2 is the convolution kernel of the second downsampler, k3 is the convolution kernel of the third down sampler, k4 is the convolution kernel of the fourth down sampler, k5 is the convolution kernel of the fifth down sampler, and k6 is the convolution kernel of the sixth down sampler.

3. A lightweight single-image denoising method based on a learning model according to claim 2, characterized in that: Each downsampler performs downsampling operations on the input image, and outputs downsampled sub-datasets corresponding to each downsampler, including: Each downsampler starts from the upper left corner of the input image, selects an area with a length and width twice the size of the convolution kernel for non-overlapping downsampling, fills the insufficient edges of the input image with zeros, and obtains several downsampled sub-images after traversing the entire input image, and outputs the corresponding downsampled sub-dataset.

4. The lightweight learning model-based single-image denoising method according to claim 1, characterized in that: The downsampled noisy image dataset is input into the image denoising model to extract the noise, and the image in the downsampled noisy image dataset is subtracted from the corresponding noise extracted by the image denoising model to obtain the downsampled denoised image dataset. The formula is: S Ei =d i (I)-N θ (d i (I)) Among them, I is the original image, S Ei is the i-th downsampled denoised image sub-dataset, d i (I) is the i-th downsampled noise image sub-dataset, i = 1, 2, ..., 6, N θ It is an image denoising model; All downsampled noisy image sub-datasets constitute the downsampled noisy image dataset, and the formula is: D I ={(d1(I),d2(I),d3(I),d4(I),d5(I),d6(I))} Among them, D I It is a downsampled noisy image dataset; All downsampled denoised image sub-datasets constitute the downsampled denoised image dataset, and the formula is: S E ={(S E1 ,S E2 ,S E3 ,S E4 ,S E5 ,S E6 )} Among them, S E It is a downsampled denoised image dataset.

5. The lightweight learning model-based single-image denoising method according to claim 1, characterized in that: All downsampled initial denoised image sub-datasets constitute the downsampled initial denoised image dataset, and the formula is: D p ={(d1(p),d2(p),d3(p),d4(p),d5(p),d6(p))} Among them, D p is the downsampled initial denoised image dataset.

6. The lightweight learning model-based single-image denoising method according to claim 1, characterized in that: The i-th downsampled initial denoised image sub-dataset d i (p) The low-frequency image sub-dataset obtained by Gaussian low-pass filter is: d i (f)=GF σ=3,9,15 (d i (p)) Among them, GF σ=3,9,15 represents a Gaussian low-pass filter, and σ represents the parameters of the Gaussian filter function in the Gaussian low-pass filter.

7. A lightweight single-image denoising device based on a learning model, comprising: The downsampling module includes three pairs of downsamplers. The convolution kernels of each pair of downsamplers have the same size, which are a main diagonal matrix and a sub-diagonal matrix respectively, and satisfy A mm =B m(n-m+1) , where A mm represents the element in the mth row and mth column of the main diagonal matrix, B m(n-m+1) represents the element in the mth row and (n-m+1)th column of the off-diagonal matrix, m=1,2,…,n, n is the order of the main diagonal matrix; each downsampler performs downsampling operation on the input image respectively, and outputs downsampled sub-datasets corresponding to each downsampler respectively; all downsampler sub-datasets constitute a downsampled data set; The down-sampling noise image acquisition module is used to input the original image into the down-sampling module. Output downsampled noisy image dataset; The down-sampled denoised image acquisition module is used to construct an image denoising model based on a residual neural network, input the down-sampled noisy image data set into the image denoising model to extract noise, and subtract the noise corresponding to the image denoising model from the image in the down-sampled noisy image data set to obtain the down-sampled denoised image data set; The down-sampled initial denoised image acquisition module is used to input the original image into the image denoising model to extract noise, and subtract the noise corresponding to the image denoising model from the original image to obtain the initial denoised image; the initial denoised image is input into the down-sampling module, and the down-sampled initial denoised image data set is output; The loss function construction module is used to construct the loss function of the image denoising model based on the downsampled noisy image sub-dataset, the downsampled initial denoised image data set and the downsampled denoised image data set, optimize the network parameters of the image denoising model, and obtain the trained image denoising model; the loss function of the image denoising model includes residual loss, regularization loss and guidance loss, and the formula is: loss final =αloss res +βloss reg +γloss g Among them, loss final is the loss function of the image denoising model, loss res is the residual loss, loss reg is the regularization loss, loss g is the guided loss, α, β and γ are scalar regularization parameters; The calculation formula of the residual loss is: Among them, loss res represents the residual loss, d j (I) and d j+1 (I) represents the jth and j+1th downsampled noise image sub-datasets, respectively, S Ej+1 and S Ej denote the j+1th and jth downsampled denoised image sub-datasets respectively; all downsampled noisy image sub-datasets constitute a downsampled noisy image dataset, and all downsampled denoised image sub-datasets constitute a downsampled denoised image dataset; The calculation formula of the regularization loss is: Among them, loss reg is the regularization loss, d i (p) is the i-th downsampled initial denoised image sub-dataset, S Ei is the i-th downsampled denoised image sub-dataset, i=1,2,…,6; all downsampled initial denoised image sub-datasets constitute the downsampled initial denoised image dataset; The calculation formula of the guidance loss is: Among them, loss g is the guidance loss, S Ei is the i-th downsampled denoised image sub-dataset, d i (f) is the i-th downsampled initial denoised image sub-dataset d i (p) Low-frequency image sub-dataset obtained by Gaussian low-pass filter; The denoising module is used to input the original image into the trained image denoising model to obtain the target denoised image.

Citation Information

Patent Citations

  • Training method of noise reduction model and related device

    CN112598597A

  • Convolutional blind denoising method containing noise estimation

    CN112837231A

  • Image noise reduction model training method, processing method, device and medium

    CN117765265A

  • Method and apparatus for training image noise reduction model, electronic device, and storage medium

    WO2021253316A1