Image noise reduction method, medium and electronic device thereof
By applying BM3D or other image block noise reduction algorithms on low-frequency wavelet coefficients, and combining edge-keeping noise reduction algorithms to process high-frequency coefficients and adding image details, the problem of BM3D high memory footprint and insufficient clarity in low-end devices is solved. , realizes efficient image noise reduction processing.
Patent Information
- Application Number
- CN201911294071.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-12-16
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2039-12-16
AI Technical Summary
Existing image noise reduction algorithms such as BM3D have high memory usage and high computing performance requirements, making them difficult to effectively apply in low-end terminal devices, and the image clarity is insufficient after noise reduction.
After wavelet transformation, only the low-frequency wavelet coefficients are processed by BM3D or other image block-based noise reduction algorithm, combined with the edge-keeping noise reduction algorithm, the high-frequency wavelet coefficients are reduced, and image details are added to the low-frequency wavelet coefficients after noise reduction Information, inverse wavelet transform is performed to generate noise-reduced image data.
It effectively reduces the memory usage of BM3D noise reduction, improves computing performance, maintains image edge information, and improves image clarity after noise reduction, and is suitable for low-end terminal devices.
Smart Images

Figure CN112991190B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing, and in particular to an image noise reduction method and a medium and electronic device thereof. Background Art
[0002] In today's era of rapid information development, mobile phones have become an indispensable part of human life, especially mobile phone photography. When taking photos with mobile phones, image acquisition is easily affected by the environment and the internal circuit of the acquisition instrument. In addition to the image information people want, the captured images often contain a lot of noise information. Especially for low-end and mid-range mobile phones, the imaging performance is greatly affected by the relatively cheap photosensitive devices used. Therefore, the use of software noise reduction methods has important application value for improving the photography effect of mobile phones. At present, the noise reduction algorithms used in the industry are mainly divided into two categories: spatial noise reduction algorithms and temporal noise reduction algorithms. Among them, the spatial noise reduction algorithm with better effect is the block matching based three-dimensional filtering algorithm (BM3D), and the most widely used temporal noise reduction algorithm is the multi-frame noise reduction algorithm (MFNR). Summary of the invention
[0003] The embodiments of the present application provide an image denoising method, a medium thereof and an electronic device, which can effectively reduce the memory usage of an image block-based denoising algorithm, improve the algorithm computing performance, and be more suitable for use by low-end terminal devices.
[0004] In a first aspect, an embodiment of the present application provides an image noise reduction method, comprising:
[0005] Acquire image data to be denoised; perform wavelet transform on the image data to obtain low-frequency wavelet coefficients and high-frequency wavelet coefficients; use an image block-based denoising algorithm to perform denoising on at least one of the low-frequency wavelet coefficients; add image detail information to the denoised low-frequency wavelet coefficients, wherein the image detail information is obtained based on edge information of the denoised low-frequency wavelet coefficients; perform inverse wavelet transform on the denoised low-frequency wavelet coefficients to which the image detail information has been added to obtain denoised low-frequency sub-image data; generate denoised image data based on the denoised low-frequency sub-image data.
[0006] Since the memory usage is high when BM3D is used to perform noise reduction on images, and the computing performance requirements of the device are high, only the low-frequency wavelet coefficients are subjected to BM3D (or other noise reduction algorithms based on image blocks) noise reduction after wavelet transform, while other high-frequency wavelet coefficients use edge-preserving noise reduction algorithms with significantly less computational complexity than BM3D, which greatly reduces the computational complexity of noise reduction using the BM3D algorithm, effectively reduces the memory usage of BM3D noise reduction, improves the computing performance of the BM3D algorithm, and is more suitable for image noise reduction processing on low-end terminal devices. In addition, by adding detail information to the low-frequency wavelet coefficients after BM3D noise reduction, the information of the noise-reduced image can be effectively improved. Compared with BM3D noise reduction processing on the entire image, the edge information is effectively maintained, and the clarity of the noise-reduced image is improved.
[0007] In an implementation of the first aspect above, the image detail information is obtained in the following manner:
[0008] The residual between the low-frequency wavelet coefficients before denoising and the low-frequency wavelet coefficients after denoising is calculated, and edge information in the low-frequency wavelet coefficients after denoising is obtained; and the image information shared by the residual and the edge information is used as the image detail information.
[0009] In an implementation of the first aspect above, performing wavelet transform on the image data to obtain low-frequency wavelet coefficients and high-frequency wavelet coefficients includes:
[0010] Performing an n-th order wavelet transform on the image data to obtain an n-th order low-frequency wavelet coefficient, wherein n is an integer greater than 1; and adopting an image block-based denoising algorithm to perform denoising on at least one of the low-frequency wavelet coefficients includes: adopting an image block-based denoising algorithm to perform denoising on the n-th order low-frequency wavelet coefficient.
[0011] In an implementation of the first aspect above, before generating the denoised image data based on the denoised low-frequency sub-image data, the method further includes:
[0012] The high frequency wavelet coefficients are subjected to denoising processing by using an edge preserving denoising algorithm.
[0013] In an implementation of the first aspect, generating the denoised image data based on the denoised low-frequency sub-image data includes:
[0014] Performing inverse wavelet transform on the n-th order low-frequency wavelet coefficient after noise reduction processing and the high-frequency wavelet coefficient after noise reduction processing to obtain the noise-reduced image data.
[0015] In an implementation of the first aspect above, the image block-based denoising algorithm is a three-dimensional block matching algorithm.
[0016] In a second aspect, an embodiment of the present application provides an electronic device for image noise reduction, including:
[0017] An acquisition unit is used to acquire image data to be denoised; a wavelet transform unit is used to perform wavelet transform on the image data to obtain low-frequency wavelet coefficients and high-frequency wavelet coefficients; a denoising unit is used to perform denoising on at least one of the low-frequency wavelet coefficients using an image block-based denoising algorithm; a detail unit is used to add image detail information to the denoised low-frequency wavelet coefficients, wherein the image detail information is obtained based on edge information of the denoised low-frequency wavelet coefficients; an inverse wavelet transform unit is used to perform inverse wavelet transform on the denoised low-frequency wavelet coefficients to which the image detail information has been added to obtain denoised low-frequency sub-image data; and a generation unit is used to generate denoised image data based on the denoised low-frequency sub-image data.
[0018] In an implementation of the second aspect above, the image block-based denoising algorithm is a three-dimensional block matching algorithm.
[0019] In a third aspect, the present application provides an image noise reduction device, which has the function of implementing the above-mentioned image noise reduction method. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the above-mentioned function.
[0020] In a fourth aspect, the present application provides a computer-readable medium on which instructions are stored, and when the instructions are executed on a computer, the computer executes the image denoising method described in the first aspect.
[0021] In a fifth aspect, the present application provides an electronic device, including:
[0022] a memory for storing instructions to be executed by one or more processors of the electronic device, and
[0023] The processor is one of the processors of the electronic device, and is used to execute the image noise reduction method described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 According to some embodiments of the present application, a schematic structural diagram of an electronic device 100 is shown.
[0025] Figure 2 According to some embodiments of the present application, a schematic diagram of a process of the electronic device 100 executing an image noise reduction technology is shown.
[0026] Figure 3According to some embodiments of the present application, wavelet coefficients after first-order wavelet transform is performed on image data are shown.
[0027] Figure 4 According to some embodiments of the present application, wavelet coefficients after performing a second-order wavelet transform on image data are shown.
[0028] Figure 5 (a) shows the image A to be denoised. Figure 5 (b) shows the image A' after BM3D denoising is performed on only the second-order low-frequency wavelet coefficients of image A. Figure 5 (c) shows the final image A'' obtained after adding detail information to image A'.
[0029] Figure 6 (a) shows Figure 5 (b) shows the edge information map of image A'. Figure 6 (b) shows the Figure 6 The edge information map obtained after simple denoising of the edge information in (a).
[0030] Figure 7 According to some embodiments of the present application, a schematic flow chart of an image noise reduction method is shown.
[0031] Figure 8 According to some embodiments of the present application, a schematic structural diagram of an electronic device 800 is shown.
[0032] Fig. 9 According to some embodiments of the present application, a schematic structural diagram of the relationship between the noise reduction unit 803 and the detail unit 804 of the electronic device 800 is shown.
[0033] Fig.10 According to some embodiments of the present application, a schematic structural diagram of an electronic device 800 is shown. Specific embodiments
[0034] Illustrative embodiments of the present application include, but are not limited to, image noise reduction methods and media and devices thereof.
[0035] It will be understood that, as used herein, the term "module or unit" may refer to or include an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and / or memory that executes one or more software or firmware programs, a combinational logic circuit, and / or other appropriate hardware components that provide the described functionality, or may be part of these hardware components.
[0036] It can be understood that the image noise reduction technology provided by the present application can be applied to any electronic device, including but not limited to mobile phones, tablet computers, laptop computers, desktop computers, servers, wearable devices, head-mounted displays, mobile email devices, portable game consoles, portable music players, reader devices, personal digital assistants, virtual reality or augmented reality devices, televisions in which one or more processors are embedded or coupled, and other electronic devices.
[0037] In addition, it can be understood that the image denoising method of the present application can process various images, including images captured in real time by electronic devices and images in videos, images stored by electronic devices and images in videos, and the like.
[0038] The embodiments of the present application will be described in further detail below in conjunction with the accompanying drawings.
[0039] Figure 1 A schematic structural diagram of an electronic device 100 is shown, and the electronic device 100 can perform noise reduction processing on an image using the image noise reduction technology of the present application.
[0040] Specifically, Figure 1 As shown, the electronic device 100 is based on some embodiments of the present application, and the technical solution of the present application is now explained by taking the mobile phone 10 as an example.
[0041] Figure 1 A schematic structural diagram of an electronic device 100 is shown.
[0042] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
[0043] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0044] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units can be independent devices or integrated in one or more processors. The image denoising method of the present application can be implemented by the processor 110, for example, it can be implemented by the GPU in the processor 110 or the ISP.
[0045] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.
[0046] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved. For example, instructions for implementing the image denoising method of the present application and intermediate data generated when performing image denoising may be stored in the cache memory of the processor 110.
[0047] In some embodiments, processor 110 may include one or more interfaces.
[0048] The charging management module 140 is used to receive charging input from the charger. While the charging management module 140 is charging the battery 142, it can also power the electronic device through the power management module 141. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and powers the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160.
[0049] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
[0050] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
[0051] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel.
[0052] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor, etc. In some embodiments, the captured image can be subjected to noise reduction processing using the image noise reduction technology of the present application.
[0053] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193. In some embodiments, ISP can use the image noise reduction method of the present application to perform noise reduction processing on the image.
[0054] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor. For example, in some embodiments, the instructions and intermediate results of the image denoising method executed in the present application may also be stored in the internal memory 121.
[0055] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.
[0056] The buttons 190 include a power button, a volume button, etc. The motor 191 can generate a vibration prompt. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power change, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.
[0057] Figure 2 The process of the electronic device 100 executing the image noise reduction technology of the present application is shown. The second-order wavelet transform is used as an example for explanation, specifically including:
[0058] 201: Obtain image data to be denoised of image A to be denoised.
[0059] 202: Perform a first-order wavelet transform on the denoised image data to obtain Figure 2 Shown are the first-order low-frequency wavelet coefficients LL1 and the first-order high-frequency wavelet coefficients HL1, LH1 and HH1.
[0060] 203: Perform a second-order wavelet transform on the first-order low-frequency wavelet coefficient LL1 to obtain Figure 4 The second-order low-frequency wavelet coefficients LL2 and the second-order high-frequency wavelet coefficients HL2, LH2 and HH2 are shown.
[0061] 204: Using BM3D (Block-matching and 3D filtering) algorithm to perform denoising on the second-order low-frequency wavelet coefficient LL2 to obtain the denoised second-order low-frequency wavelet coefficient LL2'.
[0062] It is understandable that in other embodiments, other image block-based denoising algorithms may be used to perform denoising on the second-order low-frequency wavelet coefficients, for example, NLM (Non-Local Means) algorithm.
[0063] 205: Add the detail information to the denoised second-order low-frequency wavelet coefficient LL2' to obtain the denoised second-order low-frequency wavelet coefficient LL2'' with the detail information added.
[0064] In some embodiments, the detailed information is obtained by:
[0065] 1) Obtaining edge information E from the denoised second-order low-frequency wavelet coefficients LL2'. For example, the edge information E in the denoised second-order low-frequency wavelet coefficients LL2' can be extracted by using a canny operator, a Sobel operator, a Laplace operator, or the like.
[0066] 2) Calculate the residual R between the second-order low-frequency wavelet coefficient LL2 and the denoised second-order low-frequency wavelet coefficient LL2';
[0067] 3) The image information shared between the edge information E and the residual R is used as detail information.
[0068] In some other embodiments, the detail information may also be the residual between the second-order low-frequency wavelet coefficient LL2 and the denoised second-order low-frequency wavelet coefficient LL2'.
[0069] 206: Use an edge-preserving noise reduction algorithm to perform noise reduction processing on the second-order high-frequency wavelet coefficients HL2, LH2, and HH2 to obtain noise-reduced second-order high-frequency wavelet coefficients HL2', LH2', and HH2'. For example, the edge-preserving noise reduction algorithm may be a bilateral filtering algorithm, a guided filtering algorithm, and the like.
[0070] In other embodiments, the edge-preserving denoising process may be performed on only a portion of the second-order high-frequency wavelet coefficients, which is not limited here.
[0071] 207: Using an edge-preserving denoising algorithm to perform denoising on the first-order high-frequency wavelet coefficients HL1, LH2 and HH1, and obtaining denoised first-order high-frequency wavelet coefficients HL1', LH2' and HH1'.
[0072] In other embodiments, the edge-preserving denoising process may be performed on only a portion of the first-order high-frequency wavelet coefficients, which is not limited here.
[0073] 208: Perform inverse wavelet transform on the above-mentioned denoised wavelet coefficients LL2'', HL2', LH2', HH2'HL1', LH2' and HH1' to obtain corresponding sub-image data.
[0074] 209: Combine the sub-image data to obtain image data A' after noise reduction processing.
[0075] In the above embodiment, after the second-order wavelet transform, only the second-order low-frequency wavelet coefficients are subjected to BM3D (or other noise reduction algorithms based on image blocks) noise reduction processing, while other high-frequency wavelet coefficients are subjected to edge-preserving noise reduction algorithms with significantly less computational complexity than BM3D, which reduces the computational complexity of noise reduction using the BM3D algorithm to 1 / 16 of the original, effectively reducing the memory usage of BM3D noise reduction, improving the computational performance of the BM3D algorithm, and making it more suitable for image noise reduction processing on low-end terminal devices. In addition, by adding detail information to the low-frequency wavelet coefficients after BM3D noise reduction, the information of the noise-reduced image can be effectively improved. Compared with the BM3D noise reduction processing of the entire image, the edge information is effectively maintained, and the clarity of the noise-reduced image is improved.
[0076] Figure 5 (a) shows the image A to be denoised. Figure 5 (b) shows the image A' after BM3D denoising is performed on only the second-order low-frequency wavelet coefficients of image A. Figure 5 (c) shows the final image A'' obtained after adding detail information to image A'. Figure 6 (a) shows Figure 5 (b) shows the edge information map of image A'. Figure 6 (b) shows the Figure 6 The edge information graph obtained by performing a simple denoising process on the edge information in (a). For example, a simple denoising process can be performed on Figure 6 The pixel data in (a) is filtered to filter out sudden pixel values. For example, if most of the pixel values in a plurality of (e.g., 20) adjacent pixels are 0, but only one or two pixels have a value of 200, then the values of these one or two pixels are changed to 0.
[0077] It can be seen that after adding detail information, the denoised image B has higher clarity.
[0078] In addition, it can be understood that in other embodiments, it is not necessary to perform BM3D denoising on the second-order low-frequency wavelet coefficients of image A, and edge-preserving denoising on other high-frequency wavelet coefficients. It is also possible to perform third-order and fourth-order wavelet transform on image A and then perform BM3D denoising on the third-order and fourth-order low-frequency wavelet coefficients.
[0079] For example, in some embodiments, a third-order wavelet transform is performed on image A to obtain a third-order low-frequency wavelet coefficient LL3, and then BM3D algorithm is used to perform noise reduction processing on LL3 to obtain LL3', and then detail information is added to LL3' to obtain LL3''. Then, an edge-preserving noise reduction algorithm is used to perform noise reduction processing on other high-order wavelet coefficients. The detail information may be the edge information E of the third-order low-frequency wavelet coefficient LL3' after noise reduction, and the common image information between the residual R between the third-order low-frequency wavelet coefficient LL3 and LL3' before and after noise reduction.
[0080] In this way, the computational complexity of denoising using the BM3D algorithm is reduced to 1 / 16 of the original, effectively reducing the memory usage of BM3D denoising and improving the computational performance of the BM3D algorithm, making it more suitable for image denoising processing on low-end terminal devices.
[0081] In addition, in other embodiments, it is not necessary to use the BM3D algorithm to reduce noise only for the highest-order low-frequency wavelet coefficients. For example, the noise reduction method for the image A to be reduced noise may include:
[0082] 1) Perform a second-order wavelet transform on the image data of image A to obtain second-order low-frequency wavelet coefficients LL2, second-order high-frequency wavelet coefficients HL2', LH2', HH2', and first-order high-frequency wavelet coefficients HL1', LH2', and HH1'.
[0083] 2) BM3D algorithm is used to perform denoising on LL2 to obtain LL2'. At the same time, edge-preserving denoising algorithm is used to perform denoising on the second-order high-frequency wavelet coefficients HL2', LH2', HH2' and the first-order high-frequency wavelet coefficients HL1', LH2' and HH1' to obtain HL2', LH2', HH2', HL1', LH2' and HH1'.
[0084] In addition, in some embodiments, noise reduction processing may be performed on only a portion of the high-frequency wavelet coefficients.
[0085] 3) Add the detail information D1 to the low-frequency wavelet coefficient LL2' after denoising to obtain LL2'. The detail information D1 is the image information shared by the edge information E1 of LL2' and the residual R1 of the second-order low-frequency wavelet coefficient LL2 before and after denoising.
[0086] 4) Performing a third-order wavelet transform on the low-frequency wavelet coefficient LL2'' after the denoising process with the detail information D1 added thereto, to obtain a third-order low-frequency wavelet coefficient LL3 and third-order high-frequency wavelet coefficients HL3, LH3 and HH3.
[0087] 5) BM3D algorithm is used to perform denoising on LL3 to obtain LL3'. At the same time, edge-preserving denoising algorithm is used to perform denoising on the third-order high-frequency wavelet coefficients HL3, LH3 and HH3 to obtain HL3', LH3' and HH3'.
[0088] In addition, in some embodiments, noise reduction processing may be performed on only a portion of the high-frequency wavelet coefficients.
[0089] 6) Add the detail information D2 to the low-frequency wavelet coefficient LL3' after denoising to obtain LL3''. The detail information D2 is the image information shared by the edge information E1 of LL3' and the residual R2 of the second-order low-frequency wavelet coefficient LL3 before and after denoising.
[0090] 7) Perform inverse wavelet transform on the low-frequency wavelet coefficient LL3'' and the high-frequency wavelet coefficients after each noise reduction process, and combine them to obtain the noise reduction processed image A'.
[0091] It can be seen that in the above embodiment, compared with the BM3D algorithm used to perform noise reduction on the entire image, the amount of calculation of the BM3D algorithm is reduced to 5 / 64 (1 / 16+1 / 64) of the original. In addition, the second-order low-frequency wavelet coefficients and the third-order low-frequency wavelet coefficients are backfilled with details, retaining the characteristics of the image edge.
[0092] According to some embodiments of the present application, combined with the description of the above electronic device 100 performing noise reduction processing on an image, the main process of an image noise reduction method of the present application is described. The technical details described above are still applicable in this process. To avoid repetition, some of them will not be repeated this time. Figure 7 Specifically, it includes:
[0093] 701: Obtain image data to be denoised.
[0094] 702: Perform wavelet transform on the image data to be denoised. For example, second-order, third-order, or fourth-order wavelet transform may be performed.
[0095] 703: Using an image block-based noise reduction algorithm to perform noise reduction processing on at least one low-frequency wavelet coefficient.
[0096] For example, in the above embodiment, the BM3D algorithm is used to perform noise reduction processing on the second-order wavelet coefficient LL2, or the BM3D algorithm is used to perform noise reduction processing on both the second-order wavelet coefficient LL2 and the third-order wavelet coefficient LL3.
[0097] 704: Add detail information to the low-frequency wavelet coefficients after noise reduction. For example, in the above embodiment, detail information is added to LL2', or detail information D1 and D2 are added to LL2' and LL3' respectively. The acquisition of detail information can refer to the above Figure 2 Description.
[0098] 705: Perform noise reduction processing on at least one of the high-frequency coefficients using an edge-preserving noise reduction algorithm.
[0099] 706: Perform inverse wavelet transform on each wavelet coefficient that has or has not been subjected to noise reduction. For example, if noise reduction is performed on all high-frequency wavelet coefficients, then inverse wavelet transform is performed on all noise reduction wavelet coefficients. If noise reduction is performed on only some high-frequency wavelet coefficients, then inverse wavelet transform is performed on the wavelet coefficients that have been subjected to noise reduction and the wavelet coefficients that have not been subjected to noise reduction to obtain each sub-image data.
[0100] 707: Combining the sub-image data after the wavelet inverse transformation to obtain the image data after the noise reduction processing.
[0101] Figure 8 FIG. 8 is a schematic diagram showing the structure of another electronic device 800 that implements the image noise reduction technology of the present application. It can be understood that the detailed description of image noise reduction involved in the above electronic device 100 is also applicable to the electronic device 800. Specifically, Figure 8 As shown, the electronic device 800 includes:
[0102] The acquisition unit 801 is used to acquire image data to be denoised.
[0103] The wavelet transform unit 802 is used to perform wavelet transform on the image data to obtain low-frequency wavelet coefficients and high-frequency wavelet coefficients.
[0104] The denoising unit 803 is used to perform denoising on at least one of the low-frequency wavelet coefficients by using an image block-based denoising algorithm, and to perform denoising on at least one of the high-frequency wavelet coefficients by using an edge-preserving denoising algorithm.
[0105] The detail unit 804 is used to add image detail information to the low-frequency wavelet coefficients after denoising, wherein the image detail information is obtained based on edge information of the low-frequency wavelet coefficients after denoising.
[0106] The wavelet inverse transform unit 805 is used to perform an inverse wavelet transform on the denoised low-frequency wavelet coefficients with image detail information added thereto to obtain denoised low-frequency sub-image data, and to perform an inverse wavelet transform on the denoised and non-denoised high-frequency wavelet coefficients to obtain each high-frequency sub-image data.
[0107] The generating unit 806 is used to generate the image data after noise reduction based on the low-frequency sub-image data, that is, to combine the low-frequency sub-image data with the high-frequency sub-image data to obtain the image data after noise reduction.
[0108] An acquisition unit 101 , a wavelet transform unit 102 , a noise reduction unit 103 , a detail unit 104 , an inverse wavelet transform unit 105 , and a generation unit 106 .
[0109] Fig. 9 According to some embodiments, a schematic diagram of the relationship between the noise reduction unit 803 and the detail unit 804 is shown. Fig. 9 As shown, the noise reduction unit 803 includes a low-frequency noise reduction unit 8031 and a high-frequency noise reduction unit 8032. The low-frequency noise reduction unit 8031 is used to use an image block-based noise reduction algorithm to perform noise reduction processing on at least one of the low-frequency wavelet coefficients. In addition, the low-frequency noise reduction unit 8031 is connected to the detail unit 804 to obtain detail information. The high-frequency noise reduction unit 8032 is used to use an edge-preserving noise reduction algorithm to perform noise reduction processing on at least one of the high-frequency wavelet coefficients.
[0110] Fig.10 According to an embodiment of the present application, a schematic diagram of the structure of another electronic device 1000 capable of implementing the image noise reduction technology of the present application is shown. In one embodiment, the electronic device 1000 may include one or more processors 1004, a system control logic 1008 connected to at least one of the processors 1004, a system memory 1012 connected to the system control logic 1008, a non-volatile memory (NVM) 1016 connected to the system control logic 1008, and a network interface 1020 connected to the system control logic 1008.
[0111] In some embodiments, the processor 1004 may include one or more single-core or multi-core processors. In some embodiments, the processor 1004 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In an embodiment where the electronic device 1000 uses an eNB (Evolved Node B, enhanced base station) 101 or a RAN (Radio Access Network, radio access network) controller 102, the processor 1004 may be configured to execute various embodiments, such as Figure 2 Or one or more of the multiple embodiments shown in 7.
[0112] In some embodiments, system control logic 1008 may include any suitable interface controller to provide any suitable interface to at least one of processors 1004 and / or any suitable device or component in communication with system control logic 1008 .
[0113] In some embodiments, the system control logic 1008 may include one or more memory controllers to provide an interface to the system memory 1012. The system memory 1012 may be used to load and store data and / or instructions. In some embodiments, the memory 1012 of the electronic device 1000 may include any suitable volatile memory, such as a suitable dynamic random access memory (DRAM).
[0114] NVM / memory 1016 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, NVM / memory 1016 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of a HDD (Hard Disk Drive), a CD (Compact Disc) drive, and a DVD (Digital Versatile Disc) drive.
[0115] NVM / storage 1016 may include a portion of storage resources on a device on which electronic device 1000 is installed, or it may be accessible to the device but not necessarily a portion of the device. For example, NVM / storage 1016 may be accessed over a network via network interface 1020.
[0116] In particular, system memory 1012 and NVM / storage 1016 may include, respectively, a temporary copy and a permanent copy of instructions 1024. Instructions 1024 may include instructions that, when executed by at least one of processors 1004, cause electronic device 1000 to implement the following: Figure 2 or Figure 7 In some embodiments, instructions 1024, hardware, firmware, and / or software components thereof may additionally / alternatively be placed in system control logic 1008, network interface 1020, and / or processor 1004.
[0117] The network interface 1020 may include a transceiver for providing a radio interface for the electronic device 1000, thereby communicating with any other suitable device (such as a front-end module, an antenna, etc.) through one or more networks. In some embodiments, the network interface 1020 may be integrated with other components of the electronic device 1000. For example, the network interface 1020 may be integrated with at least one of the processor 1004, the system memory 1012, the NVM / storage 1016, and a firmware device (not shown) having instructions. When at least one of the processors 1004 executes the instructions, the electronic device 1000 implements the following. Figure 2 Or the image denoising method shown in 7.
[0118] The network interface 1020 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 1020 may be a network adapter, a wireless network adapter, a telephone modem and / or a wireless modem.
[0119] In one embodiment, at least one of the processors 1004 may be packaged together with logic for one or more controllers of the system control logic 1008 to form a system in package (SiP). In one embodiment, at least one of the processors 1004 may be integrated on the same die with logic for one or more controllers of the system control logic 1008 to form a system on chip (SoC).
[0120] The electronic device 1000 may further include an input / output (I / O) device 1032. The I / O device 1032 may include a user interface to enable a user to interact with the electronic device 1000; the design of the peripheral component interface enables the peripheral components to interact with the electronic device 1000. In some embodiments, the electronic device 1000 further includes a sensor for determining at least one of an environmental condition and location information related to the electronic device 1000.
[0121] In some embodiments, the user interface may include, but is not limited to, a display (e.g., an LCD display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., an LED flash), and a keyboard.
[0122] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.
[0123] In some embodiments, the sensors may include, but are not limited to, gyroscope sensors, accelerometers, proximity sensors, ambient light sensors, and positioning units. The positioning unit may also be part of or interact with the network interface 1020 to communicate with components of a positioning network (e.g., global positioning system (GPS) satellites).
[0124] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.
[0125] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.
[0126] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.
[0127] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Therefore, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a machine (e.g., computer) readable form.
[0128] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.
[0129] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.
[0130] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0131] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.
Claims
1. An image denoising method, characterized in that: include: Acquire image data to be denoised; Performing wavelet transform on the image data to obtain low-frequency wavelet coefficients and high-frequency wavelet coefficients; Using an image block-based noise reduction algorithm to perform noise reduction processing on at least one of the low-frequency wavelet coefficients; Adding image detail information to the low-frequency wavelet coefficients after denoising, wherein the image detail information is obtained based on edge information of the low-frequency wavelet coefficients after denoising; wherein the image detail information is obtained by: Calculate the residual between the low-frequency wavelet coefficients before denoising and the low-frequency wavelet coefficients after denoising, and obtain edge information in the low-frequency wavelet coefficients after denoising; Using the image information common to the residual and the edge information as the image detail information; Performing inverse wavelet transform on the denoised low-frequency wavelet coefficients after adding the image detail information to obtain denoised low-frequency sub-image data; The denoised image data is generated based on the denoised low-frequency sub-image data.
2. The image denoising method according to claim 1, wherein: The performing wavelet transform on the image data to obtain low-frequency wavelet coefficients and high-frequency wavelet coefficients comprises: Performing an n-th order wavelet transform on the image data to obtain an n-th order low-frequency wavelet coefficient, wherein n is an integer greater than 1; and The adopting the image block-based denoising algorithm to perform denoising processing on at least one of the low-frequency wavelet coefficients comprises: The image block-based denoising algorithm is used to denoise the n-th order low-frequency wavelet coefficients.
3. The image denoising method according to claim 2, wherein: Before generating the denoised image data based on the denoised low-frequency sub-image data, the method further includes: The high frequency wavelet coefficients are subjected to denoising processing by using an edge preserving denoising algorithm.
4. The image denoising method according to claim 3, wherein: The generating the denoised image data based on the denoised low-frequency sub-image data comprises: Performing inverse wavelet transform on the n-th order low-frequency wavelet coefficient after noise reduction processing and the high-frequency wavelet coefficient after noise reduction processing to obtain the noise-reduced image data.
5. The image denoising method according to any one of claims 1 to 4, characterized in that: The image block-based noise reduction algorithm is a three-dimensional block matching algorithm.
6. An electronic device for image noise reduction, characterized in that: include: An acquisition unit, used for acquiring image data to be denoised; A wavelet transform unit, used for performing wavelet transform on the image data to obtain low-frequency wavelet coefficients and high-frequency wavelet coefficients; A denoising unit, configured to perform denoising processing on at least one of the low-frequency wavelet coefficients by using an image block-based denoising algorithm; A detail unit is used to add image detail information to the low-frequency wavelet coefficients after denoising, wherein the image detail information is obtained based on the edge information of the low-frequency wavelet coefficients after denoising; wherein the image detail information is obtained by: Calculate the residual between the low-frequency wavelet coefficients before denoising and the low-frequency wavelet coefficients after denoising, and obtain edge information in the low-frequency wavelet coefficients after denoising; The image information common to the residual and the edge information is used as the image detail information An inverse wavelet transform unit is used to perform an inverse wavelet transform on the denoised low-frequency wavelet coefficients after adding the image detail information to obtain denoised low-frequency sub-image data; A generating unit is used to generate denoised image data based on the denoised low-frequency sub-image data.
7. The electronic device for image noise reduction according to claim 6, characterized in that: The image block-based noise reduction algorithm is a three-dimensional block matching algorithm.
8. A computer-readable medium, characterized in that The computer-readable medium stores instructions, which, when executed on a computer, enable the computer to execute the image denoising method according to any one of claims 1 to 5.
9. An electronic device, characterized in that: include: a memory for storing instructions to be executed by one or more processors of the electronic device, and The processor is one of the processors of the electronic device, and is used to execute the image noise reduction method according to any one of claims 1 to 5.
