An image denoising method, device and electronic equipment
By combining a deep learning-based blind denoising neural network model with multi-level modulation of frequency and brightness parameters, the problem of difficult parameter tuning in traditional methods and the mismatch between noise models in deep learning methods is solved, thus achieving efficient image denoising in complex scenes.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- AXERA SEMICON (SHANGHAI) CO LTD
- Filing Date
- 2023-08-24
- Publication Date
- 2026-06-02
AI Technical Summary
Existing image denoising methods perform poorly in complex scenes. Traditional methods require manual parameter tuning and are complex, while deep learning methods rely on noise priors, model mismatch, and have long processing times, resulting in loss of image details and reduced clarity.
A blind noise reduction neural network model based on deep learning is adopted, which combines preset frequency and brightness parameters for multi-level modulation. The algorithm flexibility is enhanced by frequency division and brightness division characteristics, and noise estimation and image are fused to obtain the noise reduction result.
While preserving the simplicity of the end-to-end neural network model, it improves the noise reduction effect and flexibility, adapts to different noise reduction needs, reduces the loss of image details, and enhances the noise reduction performance in complex scenes.
Smart Images
Figure CN117274074B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image denoising technology, and in particular to an image denoising method, apparatus and electronic device. Background Technology
[0002] Image denoising refers to the process of reducing noise in digital images and is a crucial step in image signal processing by an Image Signal Processor (ISP). Digital images are often affected by noise interference from imaging equipment and the external environment during digitization and transmission, resulting in noise interference information in the image data, such as impulse noise and salt-and-pepper noise. To suppress noise and improve image quality, image denoising methods can be used to process noisy image signals and ultimately restore the original image signal.
[0003] Image denoising methods include traditional methods and deep learning methods. Traditional methods include statistical filters, wavelet denoising, and nonlocal means denoising. Deep learning methods are based on various derivative algorithms of neural networks to achieve image denoising.
[0004] However, traditional methods require users to manually adjust various parameters, such as filter size and threshold, to achieve the best noise reduction effect. This requires considerable experience and repeated experiments, presenting a challenge in parameter tuning. Furthermore, in complex scenes such as low light or motion blur, the difficulty of noise reduction increases exponentially, and traditional methods alone cannot effectively handle noise, potentially leading to artifacts or other unnatural phenomena. Deep learning methods rely on assumed noise priors, such as noise models based on Gaussian or uniform noise assumptions. However, since real-world noise is often complex and diverse, there may be a mismatch between the noise and the assumed model, reducing the image noise reduction effect. Moreover, neural network models typically require long processing times, especially when processing high-resolution or large-scale images, limiting their feasibility in real-time or large-scale applications. Additionally, both traditional and deep learning methods suffer from information loss; while reducing noise, they incur a loss of image details, leading to decreased image resolution, reduced sharpness, and ultimately, a decline in noise reduction effectiveness. Summary of the Invention
[0005] This application provides an image denoising method, apparatus, and electronic device to solve the problem of poor denoising effect in image denoising.
[0006] In a first aspect, this application provides an image denoising method, comprising:
[0007] Obtain the image to be denoised;
[0008] The image to be denoised is input into a preset denoising model to obtain the noise estimate output by the denoising model. The denoising model is a blind denoising neural network model built based on deep learning.
[0009] The noise estimation is modulated at a first level based on preset frequency noise reduction parameters, and at a second level based on preset brightness noise reduction parameters to obtain the modulated noise estimation.
[0010] The noise estimation after modulation is fused with the noise estimation after the denoising model and the image to be denoised to obtain the denoised result image.
[0011] In an optional implementation, the method further includes:
[0012] Acquire video stream data;
[0013] The video stream data is split into multiple single-frame images to obtain the image to be denoised.
[0014] In an optional implementation, the method further includes:
[0015] The noise estimate is divided into multiple frequency bands to obtain the noise estimate.
[0016] Obtain frequency noise reduction parameters for multiple frequency bands;
[0017] Based on the frequency noise reduction parameters of the multiple frequency bands, the noise estimation of the multiple frequency bands is modulated in a first-level manner;
[0018] Based on the brightness noise reduction parameters, the noise estimation of the multiple frequency bands is performed using two-stage modulation.
[0019] The noise estimates of the modulated multiple frequency bands are weighted and summed to obtain the modulation noise estimate.
[0020] In an optional implementation, the method further includes:
[0021] Obtain brightness noise reduction parameters for multiple frequency bands;
[0022] Based on the luminance noise reduction parameters of the multiple frequency bands, the noise estimation of the multiple frequency bands is performed in two stages.
[0023] In an optional implementation, the method further includes:
[0024] The noise estimation is divided into frequency ranges to obtain high-frequency noise estimation, mid-frequency noise estimation, and low-frequency noise estimation. The high-frequency noise estimation is the noise signal whose frequency is located in the first frequency band in the noise estimation. The mid-frequency noise estimation is the noise signal whose frequency is located in the second frequency band in the noise estimation. The low-frequency noise estimation is the noise signal whose frequency is located in the third frequency band in the noise estimation.
[0025] Obtain high-frequency noise reduction parameters, mid-frequency noise reduction parameters, and low-frequency noise reduction parameters;
[0026] The high-frequency noise estimation is modulated at the first stage based on the high-frequency noise reduction parameters, the intermediate-frequency noise estimation is modulated at the first stage based on the intermediate-frequency noise reduction parameters, and the low-frequency noise estimation is modulated at the first stage based on the low-frequency noise reduction parameters.
[0027] In an optional implementation, the modulation noise estimate and the image to be denoised are fused based on the objective function of the denoising model, wherein the objective function is:
[0028] Args min ((I+N)-GT);
[0029] Among them, Args min The variable values represent the minimum values of the objective function, I represents the image to be denoised, GT represents the denoised image, and N represents the modulation noise estimate.
[0030] Secondly, this application provides an image denoising device, including an image acquisition module, a model denoising module, a multi-level modulation module, and a denoising output module, wherein:
[0031] Image acquisition module, used to acquire the image to be denoised;
[0032] The model denoising module is used to input the image to be denoised into a preset denoising model to obtain the noise estimate output by the denoising model. The denoising model is a blind denoising neural network model built based on deep learning.
[0033] A multi-level modulation module is used to perform first-level modulation on the noise estimation based on preset frequency noise reduction parameters, and to perform second-level modulation on the noise estimation based on preset brightness noise reduction parameters, so as to obtain the modulation noise estimation.
[0034] The noise reduction output module is used to fuse the modulation noise estimate with the image to be denoised based on the noise reduction model to obtain the noise reduction result image.
[0035] In an optional implementation, the multi-level modulation module is further configured to perform frequency division processing on the noise estimation to obtain noise estimates for multiple frequency bands.
[0036] Obtain frequency noise reduction parameters for multiple frequency bands;
[0037] Based on the frequency noise reduction parameters of the multiple frequency bands, the noise estimation of the multiple frequency bands is modulated in a first-level manner;
[0038] Based on the brightness noise reduction parameters, the noise estimation of the multiple frequency bands is performed using two-stage modulation.
[0039] The noise estimates of the modulated multiple frequency bands are weighted and summed to obtain the modulation noise estimate.
[0040] In an optional implementation, the multi-level modulation module is further configured to perform frequency division processing on the noise estimation to obtain high-frequency noise estimation, intermediate-frequency noise estimation and low-frequency noise estimation, wherein the high-frequency noise estimation is the noise signal whose frequency is located in the first frequency band in the noise estimation, the intermediate-frequency noise estimation is the noise signal whose frequency is located in the second frequency band in the noise estimation, and the low-frequency noise estimation is the noise signal whose frequency is located in the third frequency band in the noise estimation.
[0041] Obtain high-frequency noise reduction parameters, mid-frequency noise reduction parameters, and low-frequency noise reduction parameters;
[0042] The high-frequency noise estimation is modulated at the first stage based on the high-frequency noise reduction parameters, the intermediate-frequency noise estimation is modulated at the first stage based on the intermediate-frequency noise reduction parameters, and the low-frequency noise estimation is modulated at the first stage based on the low-frequency noise reduction parameters.
[0043] Thirdly, this application provides an electronic device, including: a processor, a memory, and a bus. The processor and the memory communicate with each other via the bus; the memory stores computer program instructions executable by the processor, and the processor is configured to:
[0044] Obtain the image to be denoised;
[0045] The image to be denoised is input into a preset denoising model to obtain the noise estimate output by the denoising model. The denoising model is a blind denoising neural network model built based on deep learning.
[0046] The noise estimation is modulated at a first level based on preset frequency noise reduction parameters, and at a second level based on preset brightness noise reduction parameters to obtain the modulated noise estimation.
[0047] The modulation noise estimate is fused with the image to be denoised based on the denoising model to obtain the denoised result image.
[0048] As can be seen from the above technical solutions, this application provides an image denoising method, apparatus, and electronic device. The method, after acquiring the image to be denoised, inputs the image to be denoised into a preset denoising model to obtain a noise estimate output by the denoising model. Then, based on preset frequency denoising parameters, a first-level modulation is applied to the noise estimate, and based on preset brightness denoising parameters, a second-level modulation is applied to the noise estimate to obtain a modulation noise estimate. Finally, the modulation noise estimate and the image to be denoised are fused based on the denoising model to obtain the denoised result image. While retaining the advantages of the simplicity and ease of use of end-to-end neural network models, this method incorporates frequency division and brightness division characteristics into the denoising network through an external dynamic parameter adjustment interface to enhance algorithm flexibility, improve denoising effect, and meet different denoising needs. Attached Figure Description
[0049] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0050] Figure 1 A schematic flowchart of an image noise reduction method provided in an embodiment of this application;
[0051] Figure 2 A modulation schematic diagram for noise estimation provided in an embodiment of this application;
[0052] Figure 3 A modulation schematic diagram for another noise estimation provided in an embodiment of this application;
[0053] Figure 4 A flowchart illustrating image noise reduction provided in this application embodiment;
[0054] Figure 5 This is a structural block diagram of the image noise reduction device provided in the embodiments of this application;
[0055] Figure 6 A structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0056] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.
[0057] Image denoising refers to the process of reducing noise in digital images and is a crucial step in image signal processing by an Image Signal Processor (ISP). In some embodiments, image denoising can be performed based on traditional methods, such as statistical filters, wavelet denoising, and nonlocal mean denoising. However, traditional methods require users to manually adjust various parameters, such as filter size and threshold, to achieve the best denoising effect. This requires considerable experience and repeated experiments, presenting a challenge in parameter tuning. Furthermore, in complex scenes such as low light or motion blur, the difficulty of denoising increases exponentially, and traditional methods alone cannot effectively handle noise, potentially leading to artifacts or other unnatural phenomena.
[0058] In some embodiments, image denoising can also be performed using deep learning methods, i.e., denoising based on various derivative algorithms of neural networks. However, deep learning methods rely on assumed noise priors, such as noise models based on assumptions like Gaussian noise or uniform noise. But since real noise is often complex and diverse, there may be a mismatch between the noise and the assumed model, reducing the effectiveness of image denoising. Furthermore, neural network models typically require long processing times, especially when processing high-resolution or large-scale images, which limits their feasibility in real-time or large-scale applications.
[0059] In addition, both traditional and deep learning methods suffer from information loss. While reducing noise, they also cause a loss of image details, which in turn leads to a decrease in image resolution, clarity, and noise reduction effect.
[0060] To improve the denoising effect of image denoising, some embodiments of this application also provide an image denoising method, such as... Figure 1 As shown, Figure 1 This is a flowchart illustrating an image denoising method provided in an embodiment of this application. The image denoising method includes the following steps:
[0061] S100: Acquire the image to be denoised.
[0062] The image to be denoised refers to the image that needs to undergo noise reduction processing. This application does not specifically limit the implementation method for acquiring the image to be denoised; those skilled in the art can adjust it according to actual circumstances. For example, a RAW image format image to be denoised sent by a camera can be received.
[0063] In some embodiments, the image sequence can also be denoised to obtain video stream data, which includes multiple frames of images. The video stream data is split into multiple single-frame images to obtain multiple frames of images to be denoised. Image denoising processing is performed on each frame of the image to be denoised.
[0064] S200: Input the image to be denoised into the preset denoising model to obtain the noise estimate output by the denoising model.
[0065] The denoising model is a pre-trained blind denoising neural network model based on deep learning. The image to be denoised is input into the denoising model, and the model predicts the image noise to obtain the noise spectrum of the image to be denoised, i.e., noise estimation.
[0066] In some embodiments, the objective function of the noise reduction model can be summarized as follows:
[0067] Args min ((I+N0)-GT0);
[0068] Among them, Args min The variable values represent the minimum values of the objective function, I represents the image to be denoised, GT0 represents the target image, i.e. the ideal image after denoising, and N0 represents the noise estimate of the model output.
[0069] S300: The noise estimation is modulated at the first level based on the preset frequency noise reduction parameters, and the noise estimation is modulated at the second level based on the preset brightness noise reduction parameters to obtain the modulated noise estimation.
[0070] Among them, the frequency denoising parameter is used to characterize the denoising strength of the frequency component, and the luminance denoising parameter is used to characterize the denoising strength of the luminance component. Noise estimation is modulated according to the frequency denoising parameter and the luminance denoising parameter to meet diverse noise estimation needs.
[0071] It is understood that a larger noise reduction parameter results in greater noise reduction for the noise estimation, while a smaller noise reduction parameter results in less noise reduction for the noise estimation. This application does not impose specific limitations on the frequency noise reduction parameter or the brightness noise reduction parameter; those skilled in the art can adjust them according to actual circumstances.
[0072] S400: Based on the denoising model, the modulation noise estimate and the image to be denoised are fused to obtain the denoised result image.
[0073] After obtaining the modulation noise estimate, the modulation noise estimate can be fused with the image to be denoised based on the denoising model to obtain the denoised result image.
[0074] In some embodiments, the modulation noise estimate and the image to be denoised can be fused based on the objective function of the denoising model to obtain the denoised result image corresponding to the image to be denoised. The objective function of the denoising model is:
[0075] Args min ((I+N)-GT);
[0076] Among them, Args minThe variable values represent the minimum values of the objective function, I represents the image to be denoised, GT represents the denoised image, and N represents the modulation noise estimate.
[0077] In some embodiments, to meet the noise reduction requirements of different frequency bands, frequency noise reduction parameters for different frequency bands can be set, and primary modulation can be performed on the noise estimates of different frequency bands. That is, after obtaining the noise estimate output by the noise reduction model, the noise estimate is frequency-divided to obtain noise estimates for multiple frequency bands. Then, preset frequency noise reduction parameters for multiple frequency bands are obtained, and primary modulation is performed on the noise estimates of multiple frequency bands based on these parameters. Similarly, preset luminance noise reduction parameters are obtained, and secondary modulation is performed on the noise estimates of multiple frequency bands based on these parameters.
[0078] After modulation, the noise estimates of multiple frequency bands after modulation are weighted and summed to obtain the modulation noise estimate. Finally, the modulation noise estimate and the image to be denoised are fused based on the denoising model to obtain the denoised result image.
[0079] In some embodiments, such as Figure 2 As shown, frequency division processing of the noise estimation yields high-frequency noise estimation, mid-frequency noise estimation, and low-frequency noise estimation. Specifically, the high-frequency noise estimation refers to the noise signal with frequencies within the first frequency band, the mid-frequency noise estimation refers to the noise signal with frequencies within the second frequency band, and the low-frequency noise estimation refers to the noise signal with frequencies within the third frequency band.
[0080] Accordingly, high-frequency noise reduction parameters, mid-frequency noise reduction parameters, and low-frequency noise reduction parameters corresponding to different frequency bands are preset. During modulation, the preset high-frequency noise reduction parameters, mid-frequency noise reduction parameters, and low-frequency noise reduction parameters are acquired. First-level modulation is performed on the high-frequency noise estimation based on the high-frequency noise reduction parameters, first-level modulation is performed on the mid-frequency noise estimation based on the mid-frequency noise reduction parameters, and first-level modulation is performed on the low-frequency noise estimation based on the low-frequency noise reduction parameters.
[0081] It should be noted that the specific implementation methods of frequency division processing in this application are not limited, and those skilled in the art can make adjustments according to the actual situation. For example, frequency division processing can be performed on noise estimation based on a pyramid frequency division structure.
[0082] In some embodiments, brightness noise reduction parameters for different frequency bands can be set, and the noise estimation of the corresponding frequency band can be subjected to secondary modulation according to the brightness noise reduction strength of different frequency bands. That is, based on the frequency noise reduction parameters of multiple frequency bands, the noise estimation of multiple frequency bands is first-level modulated separately. Then, preset brightness noise reduction parameters for multiple frequency bands are obtained, and based on the brightness noise reduction parameters of multiple frequency bands, secondary modulation is performed on the noise estimation of multiple frequency bands separately. For example, as... Figure 2As shown, after frequency division processing, the noise estimation yields high-frequency noise estimation, intermediate-frequency noise estimation, and low-frequency noise estimation. Secondary modulation can be applied to the high-frequency noise estimation based on preset high-frequency brightness denoising parameters. Secondary modulation can also be applied to the intermediate-frequency noise estimation based on preset intermediate-frequency brightness denoising parameters. Finally, secondary modulation can be applied to the low-frequency noise estimation based on preset low-frequency brightness denoising parameters.
[0083] In some embodiments, luminance denoising parameters for different luminance components can be set to perform secondary modulation on the noise estimation. For example, three luminance component luminance denoising parameters can be set: a first luminance denoising parameter, a second luminance denoising parameter, and a third luminance denoising parameter. The first luminance denoising parameter is used to perform secondary modulation on the noise signal whose luminance component in the noise estimation is located in a first luminance range. The second luminance denoising parameter is used to perform secondary modulation on the noise signal whose luminance component in the noise estimation is located in a second luminance range. The third luminance denoising parameter is used to perform secondary modulation on the noise signal whose luminance component in the noise estimation is located in a third luminance range.
[0084] Of course, luminance denoising parameters based on different luminance components can also be used for secondary modulation of noise estimation across multiple frequency bands. For example, ... Figure 3 As shown, high-frequency noise estimation, mid-frequency noise estimation and low-frequency noise estimation are modulated in two stages based on the first brightness noise reduction parameter, the second brightness noise reduction parameter and the third brightness noise reduction parameter, respectively.
[0085] For example, such as Figure 4 The diagram shown is a flowchart of image denoising provided in an embodiment of this application. The user inputs an image to be denoised with dimensions H×W×C, where H, W, and C represent the image height, width, and number of channels, respectively.
[0086] The image to be denoised is input into the denoising model to obtain the noise estimate output by the denoising model. The objective function of the denoising model can be summarized as Args. min ((I+N0)-GT0), where I represents the input image to be denoised, GT0 represents the target image, and N0 represents the noise estimate of the network output.
[0087] The obtained noise estimates are processed by a pyramid structure to obtain noise estimates for three frequency bands: high frequency, mid frequency, and low frequency, which are respectively high frequency noise estimates, mid frequency noise estimates, and low frequency noise estimates.
[0088] Based on user-preset frequency noise reduction parameters for different frequency bands—namely, high-frequency noise reduction parameters, mid-frequency noise reduction parameters, and low-frequency noise reduction parameters—first-level modulation is performed on the high-frequency noise estimation, mid-frequency noise estimation, and low-frequency noise estimation, respectively. Then, based on user-preset luminance noise reduction parameters for different luminance components, second-level modulation is performed on the noise estimation of each frequency band to achieve greater precision, thereby adapting to diverse noise reduction requirements in practical applications.
[0089] The noise estimates modulated in each frequency band are weighted and summed to obtain the modulation noise estimate for the input image to be denoised. Based on the objective function of the denoising model, the modulation noise estimate and the input image to be denoised are summed to obtain the network's prediction result for the image to be denoised, i.e., the denoised result image with size H×W×C.
[0090] This application provides a frequency-division-based adjustable denoising algorithm to achieve multi-band adjustable neural network denoising. It also extends the brightness adjustable characteristic under the frequency division structure to achieve spatial-frequency domain fusion denoising. By combining traditional methods with deep learning methods, while retaining the advantages of the simplicity and accessibility of end-to-end neural network models, the algorithm's flexibility is enhanced through an external dynamic parameter adjustment interface, enabling it to meet different denoising needs faster and better. Integrating frequency division and brightness-division characteristics into the denoising network expands the algorithm's capability coverage without additional data augmentation or pre-training, achieving low-cost deep learning model fine-tuning training. This addresses the problems of traditional methods being highly experimental, relying on the parameter tuning experience of technicians, and performing poorly in complex scenes, as well as the data-driven nature of deep learning methods, requiring adaptive training for different scenes and exhibiting weak scene adaptability. This results in denoised images performing more robustly in complex and extreme scenes, improving the flexibility and denoising effect of the algorithm.
[0091] Based on the above image denoising method, some embodiments of this application also provide an image denoising apparatus, such as... Figure 5 As shown, it includes an image acquisition module, a model denoising module, a multi-level modulation module, and a denoising output module.
[0092] The image acquisition module is used to acquire the image to be denoised.
[0093] The model denoising module is used to input the image to be denoised into a preset denoising model to obtain the noise estimate output by the denoising model. The denoising model is a blind denoising neural network model built based on deep learning.
[0094] A multi-level modulation module is used to perform first-level modulation on the noise estimate based on preset frequency noise reduction parameters, and to perform second-level modulation on the noise estimate based on preset brightness noise reduction parameters, so as to obtain a modulated noise estimate.
[0095] The noise reduction output module is used to fuse the modulation noise estimate with the image to be denoised based on the noise reduction model to obtain the noise reduction result image.
[0096] In some embodiments, the multi-level modulation module is further configured to perform frequency division processing on the noise estimation to obtain noise estimates for multiple frequency bands.
[0097] Obtain frequency noise reduction parameters for multiple frequency bands.
[0098] Based on the frequency noise reduction parameters of the multiple frequency bands, the noise estimation of the multiple frequency bands is modulated in a first-level manner.
[0099] Based on the brightness noise reduction parameters, the noise estimation of the multiple frequency bands is performed using two-stage modulation.
[0100] The noise estimates of the modulated multiple frequency bands are weighted and summed to obtain the modulation noise estimate.
[0101] In some embodiments, the multi-level modulation module is further configured to perform frequency division processing on the noise estimation to obtain high-frequency noise estimation, intermediate-frequency noise estimation, and low-frequency noise estimation. Specifically, the high-frequency noise estimation is the noise signal whose frequency falls within a first frequency band, the intermediate-frequency noise estimation is the noise signal whose frequency falls within a second frequency band, and the low-frequency noise estimation is the noise signal whose frequency falls within a third frequency band.
[0102] Obtain high-frequency noise reduction parameters, mid-frequency noise reduction parameters, and low-frequency noise reduction parameters.
[0103] The high-frequency noise estimation is modulated at the first stage based on the high-frequency noise reduction parameters, the intermediate-frequency noise estimation is modulated at the first stage based on the intermediate-frequency noise reduction parameters, and the low-frequency noise estimation is modulated at the first stage based on the low-frequency noise reduction parameters.
[0104] Based on the above image denoising method, some embodiments of this application also provide an electronic device, such as... Figure 6 As shown, Figure 6 This application provides a structural block diagram of an electronic device 800, which includes at least one processor 801, at least one communication interface 802, at least one memory 803, and at least one bus 804. The bus 804 enables direct communication between these components, the communication interface 802 facilitates signaling or data communication with other node devices, and the memory 803 stores computer program instructions executable by the processor 801. When the electronic device 800 is running, the processor 801 communicates with the memory 803 via the bus 804. The processor 801 calls and executes the computer program stored in the memory 803 to implement the image noise reduction method provided in this application.
[0105] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.
Claims
1. An image denoising method, characterized in that, include: Obtain the image to be denoised; The image to be denoised is input into a preset denoising model to obtain the noise estimate output by the denoising model. The denoising model is a blind denoising neural network model built based on deep learning. The noise estimation is modulated at a first level based on preset frequency noise reduction parameters and at a second level based on preset luminance noise reduction parameters to obtain modulated noise estimation. The first level modulation represents the noise estimation being divided into multiple frequency bands to obtain noise estimation for multiple frequency bands. Then, the noise estimation for each of the multiple frequency bands is modulated according to the frequency noise reduction parameters of the multiple frequency bands. The second level modulation represents the noise estimation for each of the multiple frequency bands being modulated according to the luminance noise reduction parameters of the multiple frequency bands. The noise estimates of the modulated multiple frequency bands are weighted and summed to obtain the modulation noise estimate; The modulation noise estimate is fused with the image to be denoised based on the denoising model to obtain the denoised result image.
2. The image denoising method according to claim 1, characterized in that, The method further includes: Acquire video stream data; The video stream data is split into multiple single-frame images to obtain the image to be denoised.
3. The image denoising method according to claim 1, characterized in that, The method further includes: The noise estimation is divided into frequency ranges to obtain high-frequency noise estimation, mid-frequency noise estimation, and low-frequency noise estimation. The high-frequency noise estimation is the noise signal whose frequency is located in the first frequency band in the noise estimation. The mid-frequency noise estimation is the noise signal whose frequency is located in the second frequency band in the noise estimation. The low-frequency noise estimation is the noise signal whose frequency is located in the third frequency band in the noise estimation. Obtain high-frequency noise reduction parameters, mid-frequency noise reduction parameters, and low-frequency noise reduction parameters; The high-frequency noise estimation is modulated at the first stage based on the high-frequency noise reduction parameters, the intermediate-frequency noise estimation is modulated at the first stage based on the intermediate-frequency noise reduction parameters, and the low-frequency noise estimation is modulated at the first stage based on the low-frequency noise reduction parameters.
4. The image denoising method according to claim 1, characterized in that, The objective function of the denoising model is used to fuse the modulation noise estimate with the image to be denoised. The objective function is: ； in, The variable values that minimize the objective function, where I represents the image to be denoised, GT represents the denoised image, and N represents the modulation noise estimate.
5. An image noise reduction device, characterized in that, include: Image acquisition module, used to acquire the image to be denoised; The model denoising module is used to input the image to be denoised into a preset denoising model to obtain the noise estimate output by the denoising model. The denoising model is a blind denoising neural network model built based on deep learning. A multi-level modulation module is used to perform primary modulation on the noise estimation based on preset frequency denoising parameters and secondary modulation on the noise estimation based on preset luminance denoising parameters to obtain modulated noise estimation. The primary modulation represents the noise estimation being frequency-divided to obtain noise estimation for multiple frequency bands, and then the noise estimation for each of the multiple frequency bands is modulated according to the frequency denoising parameters of the multiple frequency bands. The secondary modulation represents the noise estimation for each of the multiple frequency bands being modulated according to the luminance denoising parameters of the multiple frequency bands. The noise estimates of the modulated multiple frequency bands are weighted and summed to obtain the modulation noise estimate; The noise reduction output module is used to fuse the modulation noise estimate with the image to be denoised based on the noise reduction model to obtain the noise reduction result image.
6. The image noise reduction apparatus according to claim 5, characterized in that, The multi-level modulation module is further configured to perform frequency division processing on the noise estimation to obtain high-frequency noise estimation, intermediate-frequency noise estimation and low-frequency noise estimation. The high-frequency noise estimation is the noise signal whose frequency is located in the first frequency band in the noise estimation. The intermediate-frequency noise estimation is the noise signal whose frequency is located in the second frequency band in the noise estimation. The low-frequency noise estimation is the noise signal whose frequency is located in the third frequency band in the noise estimation. Obtain high-frequency noise reduction parameters, mid-frequency noise reduction parameters, and low-frequency noise reduction parameters; The high-frequency noise estimation is modulated at the first stage based on the high-frequency noise reduction parameters, the intermediate-frequency noise estimation is modulated at the first stage based on the intermediate-frequency noise reduction parameters, and the low-frequency noise estimation is modulated at the first stage based on the low-frequency noise reduction parameters.
7. An electronic device, characterized in that, include: Processor, memory, and bus; The processor and the memory communicate with each other via the bus; The memory stores computer program instructions that can be executed by the processor, which is configured to: Obtain the image to be denoised; The image to be denoised is input into a preset denoising model to obtain the noise estimate output by the denoising model. The denoising model is a blind denoising neural network model built based on deep learning. The noise estimation is modulated at a first level based on preset frequency noise reduction parameters and at a second level based on preset luminance noise reduction parameters to obtain modulated noise estimation. The first level modulation represents the noise estimation being divided into multiple frequency bands to obtain noise estimation for multiple frequency bands. Then, the noise estimation for each of the multiple frequency bands is modulated according to the frequency noise reduction parameters of the multiple frequency bands. The second level modulation represents the noise estimation for each of the multiple frequency bands being modulated according to the luminance noise reduction parameters of the multiple frequency bands. The noise estimates of the modulated multiple frequency bands are weighted and summed to obtain the modulation noise estimate; The modulation noise estimate is fused with the image to be denoised based on the denoising model to obtain the denoised result image.