A raw domain non-local mean image denoising method
By using a nonlocal mean filtering algorithm in the RAW domain to process images channel by channel, the problems of patterned noise and channel imbalance in RAW domain image denoising are solved, resulting in better denoising effect and image quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HEFEI JUNZHENG TECH CO LTD
- Filing Date
- 2025-01-24
- Publication Date
- 2026-07-24
AI Technical Summary
In existing technologies for RAW domain image denoising, the noise is structural and the channel filtering results are unbalanced, leading to color errors and color casts. Furthermore, traditional methods have complex noise models when processing non-RAW domain images, making it difficult to effectively suppress the noise.
In the RAW domain, a nonlocal mean filtering algorithm is used. The image is divided into multiple channels for filtering. The similarity between each pixel and its neighboring window is calculated and weighted accordingly. This allows for reasonable channel processing and avoids noise from forming patterns and channel imbalance.
It effectively reduces the difficulty of noise suppression, reduces the problem of incorrect colors and color casts caused by patterned noise after filtering and channel imbalance, and improves image quality.
Smart Images
Figure CN122453646A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, and specifically relates to a method for denoising RAW domain nonlocal means images. Background Technology
[0002] With the development of digital image processing, various noises often affect image signals during the process of sensors acquiring them, leading to a decrease in image quality. Therefore, in various fields such as medical imaging and security monitoring, it is essential to perform noise reduction processing on image signals to obtain reliable results.
[0003] Traditional image spatial domain denoising processes suppress image noise in the luminance domain using various filters, including Gaussian filtering, bilateral filtering, and median filtering. The filtering methods involved in this application include:
[0004] Neighborhood filtering: By designing window filters with different weight relationships, spatial filtering is performed on the original noisy image. The spatial or gray-level correlation between pixels in the neighborhood is used to suppress noise.
[0005] Non-Local Means (NLM) filtering is a classic image filtering algorithm. This algorithm uses information from similar regions in the image to remove noise, rather than just considering pixel values in the neighborhood. The estimated value of the pixel at the current position is obtained by weighted averaging of pixels in the image that have similar neighborhood structures to it.
[0006] However, the shortcomings of existing technologies are:
[0007] 1. In the interpolated brightness domain image, noise is often structural; and after a series of linear and nonlinear processing by back-end modules, the noise-related characteristics are destroyed, the noise model becomes more complex, and the difficulty of noise suppression increases.
[0008] Although the noise characteristics of the RAW domain image are not destroyed, traditional neighborhood filtering methods produce noise in a patterned form, and the unbalanced channel filtering results can easily cause incorrect colors and color casts.
[0009] Furthermore, the terminology used in this art includes:
[0010] 1. RAW domain: This usually refers to the RAW format used in digital photography. RAW format is an uncompressed and unprocessed image file format that records the raw image data captured from the digital camera sensor;
[0011] 2. Image denoising: An image processing technique designed to reduce or eliminate noise in digital images. Noise is caused by random or unwanted variations in pixel values in an image due to various factors, which may lead to image distortion or a decrease in visual quality. The goal of image denoising is to minimize noise while preserving as much detail and sharpness as possible.
[0012] 3. Pattern-like: This refers to images containing recurring structures or patterns, such as mazes or other pseudo-textures;
[0013] 4. Bayer format: a color filter arrangement widely used in digital cameras and webcam sensors. The core purpose of this format is to enable monochrome sensors to capture color information. The most common Bayer format is RGGB, and there are also BGGR, etc. Summary of the Invention
[0014] To address the aforementioned problems, the purpose of this application is to provide a method and apparatus for denoising based on nonlocal means in the RAW domain. This method utilizes a nonlocal means filtering algorithm in the RAW domain to overcome the shortcomings of the prior art. It mainly includes:
[0015] 1. Image filtering is performed in the RAW domain using the original characteristics of noise, reducing the difficulty of noise suppression;
[0016] 2. Non-local mean filtering is employed to avoid patterned noise after filtering and color errors and casts caused by imbalances in channel filtering results. Existing techniques use non-local mean filtering in the non-RAW domain but not in the RAW domain. This is because the RAW domain lacks interpolation, resulting in weak correlation between pixels. Therefore, this method overcomes this deficiency by appropriately dividing the channels, achieving better results.
[0017] Specifically, the present invention provides a method for denoising RAW domain nonlocal means images, the method comprising the following steps:
[0018] The method includes the following steps:
[0019] S1: The sensor acquires the RAW image;
[0020] S2: Based on the actual Bayer format, divide the image into N channels, where the Bayer format has 4 channels and is greater than or equal to 1, assuming it includes R, Gr, Gb and B channels;
[0021] S3: Perform nonlocal mean filtering on each channel; including:
[0022] S301: Calculate the similarity between each channel pixel and the center pixel of the neighboring window;
[0023] S302: Calculate the weights based on similarity information;
[0024] S303: Calculate the filtering result for each pixel in each channel based on the weights;
[0025] S4: After filtering each channel (i.e., after step S3), merge them into a RAW image and output the result.
[0026] In step S2, the channel distribution of the RAW image corresponding to different Bayer formats is different and needs to be determined according to the actual situation, without any restrictions.
[0027] Assuming the Bayer format is RGGB, then the four channels are R, Gr, Gb, and B;
[0028] One practical arrangement is as follows: Figure 3 As shown.
[0029] Step S3 further includes the following steps:
[0030] S301: For each pixel, first define two fixed-size search windows and neighborhood windows. The neighborhood window slides within the search window; that is, within the search window, the neighborhood window slides horizontally and vertically with a certain step size. After each slide, the number of image pixels contained in the neighborhood window changes, leading to the subsequent similarity calculation process, which calculates the similarity between pixels within the neighborhood window and pixels in the current pixel's neighborhood. Neighborhood window regions with high similarity are considered to have high weight for the current pixel; the weight is the weight w mentioned later. i,j High similarity corresponds to high weight, low similarity corresponds to low weight, and finally, each piece of information is multiplied by its corresponding weight. If the weight is very low, its effect is minimal. The similarity S between the current pixel i and the center pixel j of the neighboring window... i,j for:
[0031]
[0032] Where v(N) i ) represents the sequence of neighboring pixel values centered at the current pixel i in the original image, v(N) j S302: Based on the similarity calculated above, calculate the weight w of the center pixel j of each neighboring window within the search window for the current pixel i. i,j for:
[0033]
[0034] Where I represents the pixel sequence within the search window, h is a parameter controlling the filtering strength; Z is the sum of the weights of the pixel sequence within the search window, i.e., the center pixels of all neighboring windows, the purpose of which is normalization, i.e., to ensure that the sum of the weights calculated for each point is 1.
[0035] S303: Based on the weights of the center pixels j of each neighboring window within the search window calculated above, calculate the filtering result for the current pixel i. for:
[0036]
[0037] Where v j This represents the original image pixel value of the center pixel j of each neighborhood window.
[0038] In step S301, the two fixed-size search windows and neighborhood windows are actually two adjustable parameters that need to be adjusted according to the specific image noise level and requirements. The search window is 21*21 and the neighborhood window is 7*7.
[0039] In step S4, the images are merged into a RAW image. Assuming the original RAW image has four channels R, Gr, Gb and B, after filtering each channel, the images are merged into a single RAW image according to the previous pixel positions. This can be referenced from the properties of RAW images in digital images, which is a well-known technique in the industry.
[0040] Assuming the Bayer format is RGGB, the RAW plot is arranged as follows: Figure 4 As shown;
[0041] After extracting the R, Gr, Gb, and B channels from a single RAW image, four images are obtained. These four filtered images are then mapped to the R, Gr, Gb, and B channels, and finally, the RAW image is reconstructed based on the previous positional relationships.
[0042] Therefore, the advantage of this application is:
[0043] 1. Image filtering is performed in the RAW domain using the original characteristics of noise, which reduces the difficulty of noise suppression and has fewer side effects while achieving the same filtering effect;
[0044] Non-local mean filtering is used to avoid the noise becoming patterned after filtering, as well as the problem of incorrect colors and color cast caused by the imbalance of channel filtering results. Attached Figure Description
[0045] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.
[0046] Figure 1 This is a schematic diagram of the method flow of the present invention.
[0047] Figure 2 This is a schematic diagram of extracting four channel data from the RAW image to form four images in step S2 of the method embodiment of the present invention.
[0048] Figure 3 This is a schematic diagram of an actual arrangement in step S2 of the method of the present invention.
[0049] Figure 4 This is a schematic diagram of RAW format assuming Bayer format is RGGB in step S4 of the method of the present invention. Detailed Implementation
[0050] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.
[0051] To achieve the above objectives, the present invention provides the following technical solutions, such as... Figure 1 As shown, the steps include: In practice, referring to the idea of this invention, specific Bayer formats exist in applications, and specific dimensions are selected. Furthermore, combined with formulas and textual explanations, the meaning can be understood by those skilled in the art with background knowledge in image processing; including:
[0052] S1: The sensor acquires the RAW image;
[0053] S2: Based on the actual Bayer format RGGB, the image is divided into 4 channels, namely R, Gr, Gb and B channels; extract the data of the four channels from the RAW image to form four images;
[0054] like Figure 2 As shown, the R, Gr, Gb, and B channel data are extracted sequentially while maintaining the positional relationship of each pixel. This yields four images, whose size is easily calculated to be one-quarter of the original RAW image. Similarly, to restore the RAW image, each pixel in the four filtered images is placed back into its original position in the RAW image, thus forming a complete processed RAW image. The subscript numbers indicate the position in the image.
[0055] S3: Perform nonlocal mean filtering on the four images corresponding to the four channels;
[0056] Step S3 is nonlocal mean filtering, which mainly includes the following steps:
[0057] Further steps include the following:
[0058] S301: For each pixel, a 21*21 search window and a 7*7 neighborhood window are first defined. The neighborhood window slides within the search window, that is, within the search window, the neighborhood window slides horizontally and vertically with a step size of 1. After each slide, the number of image pixels contained in the neighborhood window changes, leading to the subsequent similarity calculation process, which calculates the similarity between pixels within the neighborhood window and pixels in the neighborhood of the current pixel. Neighborhood window regions with high similarity are considered to have high weight for the current pixel. The level of similarity determines the weight of the center pixel of each neighborhood window. In this technical description, "high similarity" does not have a specific numerical value or range; it needs to be combined with actual values. The weight is the weight w mentioned later. i,j High similarity corresponds to high weight, low similarity corresponds to low weight, and finally, each piece of information is multiplied by its corresponding weight. If the weight is very low, its effect is minimal. The similarity S between the current pixel i and the center pixel j of the neighboring window... i,j for:
[0059]
[0060] Where v(N) i ) represents the sequence of neighboring pixel values of the original image centered at the current pixel i. The current 7x7 neighborhood window contains this sequence of 49 pixel values. j ) represents the sequence of pixel values within the neighborhood window centered at pixel j. In the current 7*7 neighborhood window, this sequence contains 49 pixel values.
[0061] S302: Based on the similarity calculated above, calculate the weight w of the center pixel j of each neighboring window within the search window for the current pixel i. i,j for:
[0062]
[0063] Where I represents the pixel sequence within the search window, h is a parameter that controls the filtering strength, and appropriate values can be set in specific embodiments according to the required filtering strength, such as 0.5, 2, etc.; Z is the pixel sequence within the search window, that is, the sum of the weights of the center pixels of all neighboring windows. Its purpose is to normalize, that is, to ensure that the sum of the weights calculated for each point is 1.
[0064] S303: Based on the weights of the center pixels j of each neighboring window within the search window calculated above, calculate the filtering result for the current pixel i. for:
[0065]
[0066] Where v jThis represents the original image pixel value of the center pixel j of each neighborhood window;
[0067] S4: After filtering each channel, obtain the four filtered channel images: R, Gr, Gb, and B. Then merge them into a single RAW image according to the original positional relationship when the four channels were extracted, and output the final filtered RAW image result.
[0068] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for denoising RAW domain nonlocal means images, characterized in that, The method includes the following steps: S1: The sensor acquires the RAW image; S2: Based on the actual Bayer format, divide the image into N channels, where the Bayer format has 4 channels and is greater than or equal to 1, assuming it includes R, Gr, Gb and B channels; S3: Perform nonlocal mean filtering on each channel; including: S301: Calculate the similarity between each channel pixel and the center pixel of the neighboring window; S302: Calculate the weights based on similarity information; S303: Calculate the filtering result for each pixel in each channel based on the weights; S4: After filtering each channel (i.e., after step S3), merge them into a RAW image and output the result.
2. The RAW domain nonlocal mean image denoising method according to claim 1, characterized in that, In step S2, the channel distribution of the RAW image corresponding to different Bayer formats is different and needs to be determined according to the actual situation, without any restrictions. Assuming the Bayer format is RGGB, then the four channels are R, Gr, Gb, and B; One practical arrangement is shown in Figure 3.
3. The RAW domain nonlocal mean image denoising method according to claim 1, characterized in that, Step S3 further includes the following steps: S301: For each pixel, first define two fixed-size search windows and neighborhood windows; the neighborhood window slides within the search window, that is, within the search window, the neighborhood window slides horizontally and vertically with a certain step size. After each slide, the image pixels contained in the neighborhood window will change, which leads to the subsequent similarity calculation process, calculating the similarity between pixels in the neighborhood window and pixels in the neighborhood of the current pixel; neighborhood window regions with high similarity are considered to have high weight for the current pixel; the weight is the weight w mentioned later. i, j; High similarity corresponds to high weight, low similarity corresponds to low weight, and finally, each piece of information is multiplied by its corresponding weight. If the weight is very low, its effect is very small; the similarity S between the current pixel i and the center pixel j of the neighboring window. i,j for: Where v(N) i ) represents the sequence of neighboring pixel values centered at the current pixel i in the original image, v(N) j () represents the sequence of pixel values within a neighborhood window centered at pixel j; S302: Based on the similarity calculated above, calculate the weight w of the center pixel j of each neighboring window within the search window for the current pixel i. i,j for: Where I represents the pixel sequence within the search window, h is a parameter controlling the filtering strength; Z is the sum of the weights of the pixel sequence within the search window, i.e., the center pixels of all neighboring windows, the purpose of which is normalization, i.e., to ensure that the sum of the weights calculated for each point is 1. S303: Based on the weights of the center pixels j of each neighboring window within the search window calculated above, calculate the filtering result for the current pixel i. for: Where v j This represents the original image pixel value of the center pixel j of each neighborhood window.
4. The RAW domain nonlocal means image denoising method according to claim 1, characterized in that, In step S301, the two fixed-size search windows and neighborhood windows are actually two adjustable parameters that need to be adjusted according to the specific image noise level and requirements. The search window is 21*21 and the neighborhood window is 7*7.
5. A method for denoising RAW domain nonlocal means images according to claim 1, characterized in that, In step S4, the images are merged into a RAW image. Assuming the original RAW image has four channels R, Gr, Gb and B, after filtering each channel, the images are merged into a single RAW image according to the previous pixel positions. The properties of RAW images in digital images can be referenced. Assuming the Bayer format is RGGB, the RAW plots are arranged as shown in Figure 4; After extracting the R, Gr, Gb, and B channels from a single RAW image, four images are obtained. These four filtered images are then mapped to the R, Gr, Gb, and B channels, and finally, the RAW image is reconstructed based on the previous positional relationships.