Image noise estimation method and system based on image edge characteristics
By using a second-order difference filter kernel based on image edge characteristics and gradient information to filter effective sub-images, the problem of inaccurate noise estimation in existing technologies is solved, achieving more accurate and stable noise estimation and improving the performance of the denoising algorithm.
Patent Information
- Application Number
- CN202310880451.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-18
- Publication Date
- 2026-05-22
- Estimated Expiration
- 2043-07-18
Smart Images

Figure CN117078532B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of digital image processing, and more specifically to an image noise estimation method and system based on image edge characteristics. Background Technology
[0002] As is well known, images acquired through hardware will contain a certain degree of noise. Accurate estimation of the noise level in such images is of great reference value for subsequent image processing and analysis. For example, in the image processing flow of the ISP (Image Signal Processor) in a camera, there is usually a denoising stage after the de-mosaic algorithm. Noise is generally generated during the process of the CMOS or CCD image sensor (Charge-coupled Device) receiving photons and converting them into RAW images (i.e., raw image files). When ambient light shines on the image sensor through the lens module, the photoelectric effect excites many electrons. These electrons are converted into current, amplified by an analog amplifier, and then converted by an ADC (Analog-to-Digital Converter) to finally output discrete illuminance values.
[0003] Noise level is a crucial parameter in many image processing applications. Poor estimation of an image's noise level can negatively impact the performance of denoising algorithms. Most existing denoising algorithms assume the noise level in an image is well-known and perform denoising based on this level, effectively suppressing noise and meeting the demands of high-precision measurement and detection. However, even with a given true noise level, these algorithms still fall short of optimal performance, especially for images with rich textures. Current noise estimation methods primarily rely on local grayscale information. These methods fail to accurately reflect the true noise level of images with rich textures. To improve this approach, some improved image noise level estimation methods employ a block-based approach, specifically targeting images with rich texture information to address noise level estimation in complex scenes. The improved method described above first selects image blocks with few or no high-frequency components from multiple image blocks obtained by dividing the image into blocks for analysis (note: this is because the noise distribution is uniform), that is, selecting flat image regions for analysis. Then, based on the statistical gradient information of the flat image regions and principal component analysis, the noise level of the image is estimated. However, this improved method cannot stably estimate the noise level for images with severe noise pollution. Therefore, it is necessary to improve upon the existing technology to address its shortcomings. Summary of the Invention
[0004] The main technical problem solved by this invention is to provide an image noise estimation method and system based on image edge characteristics, which is used to more accurately estimate the distribution of noise in the original image, thereby improving the accuracy and stability of image noise estimation.
[0005] According to a first aspect, one embodiment provides an image noise estimation method based on image edge characteristics. The image noise estimation method includes:
[0006] Obtain the original image;
[0007] A second-order difference image is obtained by performing convolution or correlation operations on the original image using a second-order difference filter kernel; wherein, the second-order difference image contains the second-order difference information of the original image;
[0008] Based on the second-order difference image, noise in the original image is estimated to obtain a noise estimate of the original image; wherein the noise estimate is used to characterize the noise level in the image.
[0009] According to the second aspect, one embodiment also provides another image noise estimation method based on image edge characteristics. This image noise estimation method includes:
[0010] The original image is acquired and divided into multiple sub-images to be estimated; wherein there is no overlapping region between two adjacent sub-images to be estimated; based on the gradient image of the sub-images to be estimated, each sub-image to be estimated is judged to determine whether the sub-image to be estimated is a valid sub-image to be estimated;
[0011] A second-order difference image is obtained by performing convolution or correlation operations on the effective sub-image to be estimated using a second-order difference filter kernel; wherein, the second-order difference image contains the second-order difference information of the original image;
[0012] Based on the second-order difference image, noise in the original image is estimated to obtain a noise estimate of the original image; wherein the noise estimate is used to characterize the noise level in the image.
[0013] In one embodiment, the step of judging each of the sub-images to be estimated based on the gradient image of the sub-image to be estimated, to determine whether the sub-image to be estimated is a valid sub-image to be estimated, includes:
[0014] Obtain the gradient image of the sub-image to be estimated, determine the effective noise estimation region in the sub-image to be estimated based on the gradient image, determine whether the sub-image to be estimated is an invalid sub-image to be estimated based on the area of the effective noise estimation region, and remove the invalid sub-image to be estimated from the sub-image to be estimated to obtain the effective sub-image to be estimated.
[0015] In one embodiment, the process includes obtaining a gradient image of the sub-image to be estimated, determining an effective noise estimation region in the sub-image to be estimated based on the gradient image, determining whether the sub-image to be estimated is an invalid sub-image based on the area of the effective noise estimation region, and removing invalid sub-images from the sub-image to be estimated to obtain an effective sub-image to be estimated.
[0016] Calculate the gradient magnitude of each pixel in the sub-image to be estimated to obtain the gradient image of the sub-image to be estimated; calculate the relative gray-level histogram of the gradient image.
[0017] Starting from the minimum gray value of the relative gray histogram, the probability values of each gray value are accumulated until the accumulated probability value reaches or exceeds the preset cumulative probability threshold. The last accumulated gray value is then used as the binarization threshold.
[0018] The effective noise estimation region in the sub-image to be estimated is obtained according to the binarization threshold, wherein the effective noise estimation region is the region in the sub-image to be estimated whose gradient magnitude is less than the binarization threshold;
[0019] Calculate the area of the effective noise estimation region; wherein, the area of the effective noise estimation region represents the number of pixels in the sub-image to be estimated whose gradient magnitude is less than the binarization threshold;
[0020] Based on the area of the effective noise estimation region, determine whether the sub-image to be estimated is an invalid sub-image to be estimated, and remove the invalid sub-images to be estimated from the sub-images to be estimated to obtain the effective sub-images to be estimated.
[0021] In one embodiment, determining whether the sub-image to be estimated is an invalid sub-image to be estimated based on the area of the effective noise estimation region, and removing invalid sub-images from the sub-images to be estimated to obtain the effective sub-image to be estimated, includes:
[0022] The area ratio of the effective noise estimation region is calculated based on the area of the effective noise estimation region; wherein, the area ratio of the effective noise estimation region represents the percentage of the area of the effective noise estimation region to the area of the sub-image to be estimated corresponding to the effective noise estimation region;
[0023] If the area ratio of the effective noise estimation region is less than a preset area ratio threshold, the sub-image to be estimated corresponding to the effective noise estimation region is regarded as an invalid sub-image to be estimated and is discarded; if the area ratio of the effective noise estimation region is greater than or equal to a preset binarization gradient area ratio threshold, the sub-image to be estimated corresponding to the effective noise estimation region is regarded as the effective sub-image to be estimated.
[0024] In one embodiment, estimating the noise in the original image based on the second-order difference image to obtain a noise estimate of the original image includes:
[0025] Acquire each of the second-order difference images and calculate the global average value of each of the second-order difference images;
[0026] Multiply the global average value of each of the second-order difference images by The noise estimate for each of the second-order difference images is then obtained;
[0027] Based on the noise estimate of each of the second-order difference images, the noise estimate of the original image is obtained.
[0028] In one embodiment, the noise estimate of the original image is determined by the following expression:
[0029]
[0030] Where, σ i β is the noise estimate for the i-th second-order difference image. i Let L be the ratio of the area of the effective noise estimation region of the sub-image to be estimated corresponding to the i-th second-order difference image to the area of the i-th second-order difference image, where L is the total number of second-order difference images.
[0031] According to a third aspect, one embodiment provides an image noise estimation system based on image edge characteristics. The image noise estimation system includes:
[0032] The preprocessing module is configured to acquire the raw image;
[0033] The second-order difference information extraction module is configured to perform convolution or correlation operations on the original image using a second-order difference filter kernel to obtain a second-order difference image; wherein, the second-order difference image contains the second-order difference information of the original image;
[0034] The analysis and processing module is configured to estimate the noise in the original image based on the second-order difference image to obtain a noise estimate of the original image; wherein the noise estimate is used to characterize the noise level in the image.
[0035] According to a fourth aspect, one embodiment provides an image noise estimation system based on image edge characteristics. The image noise estimation system includes:
[0036] The preprocessing module is configured to acquire the original image, divide the original image into multiple sub-images to be estimated, wherein there is no overlapping region between two adjacent sub-images to be estimated; and judge each sub-image to be estimated based on the gradient image of the original image to determine whether the sub-image to be estimated is a valid sub-image to be estimated.
[0037] The second-order difference information extraction module is configured to perform convolution or correlation operations on the effective sub-image to be estimated using a second-order difference filter kernel to obtain a second-order difference image; wherein, the second-order difference image contains the second-order difference information of the original image;
[0038] The analysis and processing module is configured to estimate the noise in the original image based on the second-order difference image to obtain a noise estimate of the original image; wherein the noise estimate is used to characterize the noise level in the image.
[0039] According to a fifth aspect, one embodiment provides a computer-readable storage medium, characterized in that it includes a program that can be executed by a processor to implement the image noise estimation method as described in any of the embodiments herein.
[0040] The beneficial effects of this application are:
[0041] This application provides an image noise estimation method. In one embodiment, the method includes: acquiring an original image; performing convolution or correlation operations on the original image using a second-order difference filter kernel to obtain a second-order difference image; wherein the second-order difference image contains second-order difference information of the original image; and estimating the noise in the original image based on the second-order difference image to obtain a noise estimate of the original image. Since noise in an image often manifests as a local response signal, and the local response signal of an image has strong second-order difference characteristics, the image noise estimation method of this application uses a second-order difference filter kernel to estimate the noise in the image, thereby ultimately achieving a more accurate estimation of the noise distribution in the original image. In another embodiment, the method includes: acquiring an original image and dividing the original image into multiple sub-images to be estimated; judging each sub-image to be estimated based on its gradient image to determine whether it is a valid sub-image to be estimated; performing convolution or correlation operations on the valid sub-images to be estimated using a second-order difference filter kernel to obtain a second-order difference image; estimating the noise in the original image based on the second-order difference image to obtain a noise estimate of the original image; acquiring sub-images to be estimated containing more noise using the method in this embodiment and using them as valid sub-images to be estimated; then using the valid sub-images to be estimated to estimate the noise in the original image, thereby ultimately achieving a more accurate estimation of the noise distribution in the original image. Attached Figure Description
[0042] Figure 1 This is a flowchart illustrating an image noise estimation method according to one embodiment.
[0043] Figure 2 This is a flowchart illustrating another embodiment of an image noise estimation method;
[0044] Figure 3 This is a schematic diagram illustrating the process of obtaining an effective sub-image to be estimated based on the gradient image of the sub-image to be estimated, according to one embodiment.
[0045] Figure 4 This is a schematic diagram illustrating a process for estimating noise in an original image to obtain a noise estimate of the original image, as shown in one embodiment.
[0046] Figure 5 This is a schematic diagram of the modules of an image noise estimation system according to one embodiment;
[0047] Figure 6 This is a schematic diagram of a module of an image noise estimation system according to another embodiment. Detailed Implementation
[0048] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.
[0049] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.
[0050] The serial numbers assigned to components in this document, such as "first" and "second," are used only to distinguish the described objects and have no sequential or technical meaning. The terms "connection" and "linkage" used in this application, unless otherwise specified, include both direct and indirect connections (linkages).
[0051] The technical concept of the image noise estimation method in this application is as follows: Since noise in an image often manifests as a local response signal of the image, and the local response signal of an image (such as the original image below) has strong second-order difference characteristics, the image is sensitive to a second-order difference noise statistic (such as a second-order difference filter kernel below). Therefore, a second-order difference image can be obtained by performing convolution or correlation operations on the image using the second-order difference filter kernel. Then, the noise in the original image can be estimated based on the second-order difference image to obtain the noise estimate value of the original image.
[0052] The image noise estimation method of this application can provide guidance for subsequent focusing, image preprocessing and analysis.
[0053] The technical solution of this application will be described in detail below with reference to the embodiments.
[0054] Please refer to Figure 1 This application provides an image noise estimation method based on image edge characteristics. The image noise estimation method includes:
[0055] Step S100a: Obtain the original image;
[0056] Step S200a: Perform convolution or correlation operations on the original image using a second-order difference filter kernel to obtain a second-order difference image; wherein, the second-order difference image contains the second-order difference information of the original image;
[0057] Step S300a: Based on the second-order difference image, the noise in the original image is estimated to obtain the noise estimate of the original image; wherein, the noise estimate is used to characterize the noise level in the image.
[0058] Because image edges exhibit strong second-order difference properties, and thus are quite sensitive to the second-order Laplacian response, some embodiments may utilize convolutional kernels L1 and L2 to approximate the Laplacian response of the image. Convolutional kernels L1 and L2 are respectively expressed as:
[0059]
[0060] Therefore, in some embodiments, the second-order difference filter kernel w that performs correlation operations on the original image can be expressed as:
[0061]
[0062] It should be noted that those skilled in the art may also choose other second-order differential filter kernels according to the actual application scenario requirements.
[0063] In step S200a above, the second-order difference filter kernel can be used to perform convolution or correlation operations on the original image to obtain a second-order difference image.
[0064] The expression for the second-order difference image obtained by performing correlation operations on the original image as described above can be:
[0065]
[0066] Where f represents the gray value of a pixel in the original image; x and y are the horizontal and vertical coordinates of a pixel in the original image, respectively; s and t represent the relative coordinates of each element in the second-order difference filter kernel with respect to the pixel located at the center of the second-order difference filter kernel. For example, for a 3×3 second-order difference filter kernel, if we assume that the origin of the coordinate system is located at the center of the second-order difference filter kernel, then the relative coordinates of the pixel located at the upper left corner of the second-order difference filter kernel with respect to the pixel located at the center of the second-order difference filter kernel are (-1, -1), and the relative coordinates of the pixel located at the lower right corner of the second-order difference filter kernel with respect to the pixel located at the center of the second-order difference filter kernel are (1, 1). Therefore, it can be understood that the values of s and t are both in the range of [-1, 1]. g(x, y) represents the gray value of the pixel located at (x, y) in the second-order difference image, and w(s, t) represents the value of the element with relative coordinates (s, t) in the second-order difference filter kernel.
[0067] It should be noted that using the above-mentioned second-order difference filter kernel to perform convolution or correlation operations on the original image is common knowledge in this field, so the specific process of using the above-mentioned second-order difference filter kernel to perform convolution or correlation operations on the original image will not be described in detail.
[0068] For noise estimates, they can be noise parameters that characterize the noise level in the image, such as the standard deviation, variance, peak signal-to-noise ratio (PNSR), structural similarity (SSIM), etc. That is, the standard deviation, variance, peak signal-to-noise ratio, structural similarity, etc. of the noise in the original image can be estimated in step S300a.
[0069] It should be noted that the above-described image noise estimation method based on image edge characteristics assumes that the noise in the image is additive noise. Based on this assumption, the above-described image noise estimation method performs noise estimation on the entire image. Since the above-described image noise estimation method based on image edge characteristics performs noise estimation on the entire image, there may be some deviation in the noise estimation of local areas in the image. In one embodiment of this application, another image noise estimation method based on image edge characteristics is also provided, which is beneficial to improving the accuracy of noise estimation in local areas.
[0070] Please refer to Figure 2 Another image noise estimation method based on image edge characteristics provided in this application includes:
[0071] Step S100b: Obtain the original image and divide it into multiple sub-images to be estimated; wherein there is no overlapping region between two adjacent sub-images to be estimated; based on the gradient image of the sub-images to be estimated, judge each sub-image to be estimated to determine whether the sub-image to be estimated is a valid sub-image to be estimated;
[0072] Step S200b: Perform convolution or correlation operations on the effective sub-images to be estimated using a second-order difference filter kernel to obtain a second-order difference image; wherein, the second-order difference image contains the second-order difference information of the original image;
[0073] Step S300b: Based on the second-order difference image, the noise in the original image is estimated to obtain the noise estimate of the original image.
[0074] In step S100b above, the original image can be divided into multiple sub-images of the same size to be estimated. There is no overlap between any two adjacent sub-images to be estimated. The sub-images to be estimated can be square or rectangular. Those skilled in the art can also select other shapes for the sub-images to be estimated according to the actual needs of the scene. Then, based on the gradient image of the sub-images to be estimated, each sub-image to be estimated is judged to determine whether it is a valid sub-image to be estimated.
[0075] The purpose of determining whether the sub-image to be estimated is a valid sub-image is to filter out sub-images containing a lot of noise. Since images contain other edge features besides noise, these edge features are valid image information, but they can interfere with noise estimation. Therefore, in step S100b, the sub-images to be estimated are filtered, and those with more noise are selected as valid sub-images. In subsequent noise estimation, only valid sub-images are considered to improve the accuracy of noise estimation. Since edge features can be analyzed using gradient information, step S100b determines the amount of noise in the sub-image based on its gradient image to determine whether it is a valid sub-image.
[0076] In one embodiment, in step S100b above, based on the gradient image of the sub-image to be estimated, each sub-image to be estimated is judged to determine whether it is a valid sub-image to be estimated, including:
[0077] Step S110b: Obtain the gradient image of the sub-image to be estimated, and determine the effective noise estimation region in the sub-image based on the gradient image; determine whether the sub-image to be estimated is invalid based on the area of the effective noise estimation region, and remove the invalid sub-image from the sub-image to be estimated to obtain the effective sub-image. Here, the effective noise estimation region refers to the region with relatively high noise levels.
[0078] In one embodiment, please refer to Figure 3In step S110b above, the gradient image of the sub-image to be estimated is obtained, the effective noise estimation region in the sub-image to be estimated is determined based on the gradient image, the area of the effective noise estimation region is used to determine whether the sub-image to be estimated is an invalid sub-image, and invalid sub-images are removed from the sub-image to be estimated to obtain the effective sub-image to be estimated, including:
[0079] Step S111b: Calculate the gradient magnitude of each pixel in the sub-image to be estimated to obtain the gradient image of the sub-image to be estimated; calculate the relative gray-level histogram of the gradient image.
[0080] Step S112b: Starting from the minimum gray value of the relative gray histogram, accumulate the probability values of each gray value until the accumulated probability value reaches or exceeds the preset cumulative probability threshold, and obtain the last accumulated gray value as the binarization threshold.
[0081] Step S113b: Obtain the effective noise estimation region in the sub-image to be estimated based on the binarization threshold, wherein the effective noise estimation region is the region in the sub-image to be estimated whose gradient magnitude is less than the binarization threshold;
[0082] Step S114b: Calculate the area of the effective noise estimation region; where the area of the effective noise estimation region represents the number of pixels in the sub-image to be estimated whose gradient magnitude is less than the binarization threshold;
[0083] Step S115b: Determine whether the sub-image to be estimated is invalid based on the area of the effective noise estimation region, and remove the invalid sub-image from the sub-image to be estimated to obtain the effective sub-image.
[0084] In step S111b above, the gradient image of the sub-image to be estimated can be obtained by calculating the gradient magnitude of each pixel in the sub-image. Then, the relative gray-level histogram of the gradient image can be calculated. The gradient magnitude can be calculated using the following expression: Among them, the above This represents the first derivative of a pixel in the sub-image to be estimated along the x-direction; This represents the first derivative of a pixel in the sub-image to be estimated along the y-direction.
[0085] It should be noted that obtaining the gradient image of the sub-image to be estimated by calculating the gradient magnitude of each pixel in the sub-image to be estimated, and obtaining the relative gray-level histogram of the gradient image based on the gradient image, are conventional methods in this field. Therefore, the specific process of obtaining the gradient image and the relative gray-level histogram will not be described in detail.
[0086] Images (such as the original image or a valid sub-image to be estimated) typically contain a certain proportion of noise, and the proportion of pixels representing noise is usually small. This application distinguishes noise from other edge features in the image by using gradient values. Pixels with a small proportion of gradient values are considered noise, while pixels with a large proportion of gradient values are considered valid image information such as other edge features. The relative gray-level histogram counts the gray-level values of the gradient image (i.e., gradient values). Therefore, the image noise estimation method of this application considers pixels with a relatively small proportion of gray-level values in the relative gray-level histogram (that is, the gray-level value interval (i.e., gradient value interval) from the minimum gray-level value (i.e., minimum gradient value) of the relative gray-level histogram to the target gray-level value (i.e., target gradient value) corresponding to the preset cumulative probability threshold as noise in the image. By accumulating the probability values of each gray level, starting from the minimum gray level value of the aforementioned relative gray level histogram, and searching one by one, when the accumulated probability value reaches or exceeds a preset cumulative probability threshold, the image noise estimation method of this application considers the pixels corresponding to the gray level range between the minimum gray level value of the relative gray level histogram and the target gray level value corresponding to the preset cumulative probability threshold as noise in the original image, and considers the pixels corresponding to the gray level range between the target gray level value corresponding to the preset cumulative probability threshold and the maximum gray level value of the relative gray level histogram as valid image information in the original image.
[0087] Therefore, the gradient image can be binarized using the target gray value corresponding to the preset cumulative probability threshold to obtain regions with gradient values less than the binarization threshold. The regions in the sub-image to be estimated that correspond to the obtained regions are then used as effective noise estimation regions. In other words, the effective noise estimation region is the region in the sub-image to be estimated where the gradient magnitude is less than the binarization threshold. Therefore, in step S112b above, starting from the minimum gray value of the relative gray-level histogram of the gradient image, the probability values of each gray value are accumulated until the accumulated probability value reaches or exceeds the preset cumulative probability threshold. The last accumulated gray value (i.e., the target gray value corresponding to the preset cumulative probability threshold) is then used as the binarization threshold for the above binarization process. The preset cumulative probability threshold ranges from 0 to 1.
[0088] In some embodiments, the preset cumulative probability threshold ranges from 0.1 to 0.3.
[0089] It should be noted that those skilled in the art can select a specific preset cumulative probability threshold based on the actual application scenario.
[0090] In some embodiments, the expression for the binarization threshold a can be: a = v min Among them, v minThe target grayscale value corresponding to the preset cumulative probability threshold.
[0091] In step S113b above, the effective noise estimation region in the sub-image to be estimated is obtained according to the binarization threshold. The effective noise estimation region is the region in the sub-image to be estimated where the gradient magnitude is less than the binarization threshold. The vertical axis of the relative grayscale histogram represents the probability or proportion of each grayscale value appearing in the image. For example, the probabilities or proportions corresponding to the minimum grayscale value in a relative grayscale histogram are [0.005, 0.003, 0.01, 0.02, 0, 0.1, 0.2, ...]. If the preset cumulative probability threshold is 0.28, then starting from index 0 of the relative grayscale histogram, the probability values of each grayscale value in the relative grayscale histogram are accumulated until the accumulated probability value reaches or exceeds the preset cumulative probability threshold. The target grayscale value corresponding to the first accumulated probability greater than 0.28 is then used as the binarization threshold a. Then, the gradient image is binarized using the binarization threshold 'a'. The gray values of pixels with gradient magnitudes between [0, a] are set to 255, and the gray values of pixels with gradient magnitudes outside [0, a] are set to 0. Then, the region in the sub-image to be estimated that corresponds to the region formed by pixels with a gray value of 255 in the gradient image is the effective noise estimation region.
[0092] In step S114b above, the area of the effective noise estimation region in the sub-image to be estimated can be calculated by counting the number of pixels with a gray value of 255 in the gradient image. The area of the effective noise estimation region represents the number of pixels in the sub-image to be estimated whose gradient magnitude is less than the binarization threshold.
[0093] In one embodiment, step S115b above, determining whether the sub-image to be estimated is an invalid sub-image based on the area of the effective noise estimation region, and removing invalid sub-images from the sub-images to be estimated to obtain valid sub-images, includes:
[0094] The area ratio of the effective noise estimation region is calculated based on the area of the effective noise estimation region; where the area ratio of the effective noise estimation region represents the percentage of the area of the effective noise estimation region to the area of the sub-image to be estimated corresponding to the effective noise estimation region.
[0095] If the area ratio of the effective noise estimation region is less than the preset area ratio threshold, the sub-image to be estimated corresponding to the effective noise estimation region is regarded as an invalid sub-image to be estimated and is removed; if the area ratio of the effective noise estimation region is greater than or equal to the preset binarization gradient area ratio threshold, the sub-image to be estimated corresponding to the effective noise estimation region is regarded as an effective sub-image to be estimated.
[0096] It should be noted that those skilled in the art can flexibly set the preset area ratio threshold according to the specific application scenario.
[0097] In one embodiment, please refer to Figure 4 In step S300b above, the noise estimate of the original image is obtained by estimating the noise in the original image based on the second-order difference image, including:
[0098] Step S310b: Obtain each second-order difference image and calculate the global average value of each second-order difference image;
[0099] Step S320b: Multiply the global average value of each second-order difference image by... This yields the noise estimate for each second-order difference image;
[0100] Step S330b: Obtain the noise estimate of the original image based on the noise estimate of each second-order difference image.
[0101] In step S310b above, the expression for the global average value E of the second-order difference image can be:
[0102]
[0103] Among them, the above |g i (x,y)| represents the gray value of the pixel with coordinates (x,y) in the i-th second-order difference image, w i and h i Let represent the width and height of the i-th second-order difference image, respectively, and N represent the total number of pixels in the i-th second-order difference image.
[0104] Since the noise in images acquired by photosensitive elements (such as CMOS or CCD photosensitive elements) is generally additive noise, that is, noise superimposed on the image (such as the effective image information in the image below) and unrelated to the image signal; and this noise is usually considered to be Gaussian white noise with a Gaussian distribution and uniform power spectral density, in one embodiment, the magnitude of the noise is measured by its standard deviation. That is to say, the prior knowledge obtained by the above-mentioned photosensitive element is that the noise in the image is a Gaussian model. Therefore, in step S320b above, the expression for the noise estimate of the second-order difference image can be:
[0105]
[0106] The derivation of the expression for the noise estimate is as follows: Since in the continuous case, Therefore, we can conclude that: Digital images are in a discretized form, so the integration operation is transformed into a summation operation, that is:
[0107]
[0108] In step S330b above, the noise estimate of the original image is determined by the following expression: Where, σ i Let β be the noise estimate for the i-th second-order difference image. i Let L be the ratio of the area of the effective noise estimation region of the sub-image to be estimated corresponding to the i-th second-order difference image to the area of the i-th second-order difference image, where L is the total number of second-order difference images.
[0109] Typically, existing noise estimation methods describe or estimate noise in an image by utilizing local variations in its grayscale values. One way to do this is by using the first-order difference of the grayscale values. This is because if the first-order difference of a certain part / pixel in the image shows a large change, it indicates that the part / pixel corresponding to the large change belongs to the edge of the image (such as the original image or a valid sub-image to be estimated); if the first-order difference of a certain part / pixel in the image shows a small change, it indicates that the part / pixel corresponding to the large change belongs to a flat region of the image (i.e., the original image). However, using the first-order difference of the grayscale values cannot accurately describe whether a certain part / pixel in the image belongs to the noise or the valid image information. Therefore, the image noise estimation method of this application adopts another approach by utilizing the local variations in the grayscale values of the image, namely, using the second-order difference of the grayscale values. For example, if the second difference of the gray values in an image is equal to 0, it means that a certain part / pixel corresponding to the gray value with a second difference of 0 (i.e., a critical point) belongs to the edge of the image; if the second difference of the gray values in an image has a maximum or minimum value, it means that a certain part / pixel corresponding to the gray value with a maximum or minimum value does not belong to the edge or high-frequency information of the image. In other words, the image noise estimation method of this application uses the second difference characteristics of image edges to describe the noise in the image.
[0110] Therefore, it can be seen that in some embodiments, the image noise estimation method of this application regards the pixels corresponding to the gradient value range between the minimum gradient value of the relative gray-level histogram and the target gradient value corresponding to the preset cumulative probability threshold as noise in the original image. The image noise estimation method of this application focuses on the proportion of pixels considered as noise in the original image in each sub-image to be estimated. For example, if the proportion of noise in a sub-image to be estimated as belonging to the original image is relatively small (e.g., less than the preset area ratio threshold below), the image noise estimation method of this application considers that the sub-image to be estimated mainly represents the effective image information of the image, and the sub-image to be estimated will also be considered an invalid sub-image to be estimated. If the proportion of noise in a sub-image to be estimated as belonging to the original image is relatively large (e.g., greater than or equal to the preset area ratio threshold below), the image noise estimation method of this application considers that the proportion of noise in the sub-image to be estimated is relatively large, and thus the image noise estimation method of this application considers the sub-image to be estimated as a valid sub-image to be estimated, avoiding the use of invalid sub-images to estimate the noise of the original image, and using valid sub-images to estimate the noise of the original image, thereby ultimately achieving an accurate estimation of the distribution of noise in the original image. In short, the core of the image noise estimation method in this application is to suppress or remove the edge portion of the sub-image to be estimated that contains more effective image information in the image, while retaining only the sub-image to be estimated that contains more noise and using it as the effective sub-image to be estimated; then the effective sub-image to be estimated is used to estimate the noise of the original image, thereby ultimately achieving an accurate estimation of the noise distribution in the original image.
[0111] The above is a description of image noise estimation methods based on image edge characteristics. Some embodiments of this application also disclose an image noise estimation system based on image edge characteristics. Please refer to... Figure 5 The image noise estimation system includes:
[0112] Preprocessing module 100a is configured to acquire the raw image;
[0113] The second-order difference information extraction module 200a is configured to perform convolution or correlation operations on the original image using a second-order difference filter kernel to obtain a second-order difference image; wherein, the second-order difference image contains the second-order difference information of the original image;
[0114] The analysis and processing module 300a is configured to estimate the noise in the original image based on the second-order difference image to obtain a noise estimate of the original image; wherein the noise estimate is used to characterize the noise level in the image.
[0115] This application also discloses another image noise estimation system based on image edge characteristics in some embodiments. Please refer to... Figure 6 The image noise estimation system includes:
[0116] The preprocessing module 100b is configured to acquire the original image and divide the original image into multiple sub-images to be estimated; wherein there is no overlapping region between two adjacent sub-images to be estimated; and based on the gradient image of the original image, each sub-image to be estimated is judged to determine whether the sub-image to be estimated is a valid sub-image to be estimated.
[0117] The second-order difference information extraction module 200b is configured to perform convolution or correlation operations on the effective sub-image to be estimated using a second-order difference filter kernel to obtain a second-order difference image; wherein, the second-order difference image contains the second-order difference information of the original image;
[0118] The analysis and processing module 300b is configured to estimate the noise in the original image based on the second-order difference image to obtain a noise estimate of the original image; wherein the noise estimate is used to characterize the noise level in the image.
[0119] It should be noted that the specific operation process and corresponding technical effects of each module of the image noise estimation system can be found in the aforementioned discussion of image noise estimation methods based on image edge characteristics, and will not be repeated here.
[0120] The above is a description of an image noise estimation system based on image edge characteristics. Some embodiments of this application also disclose a computer-readable storage medium. This medium includes a program that can be executed by a processor to implement the image noise estimation method as described in any embodiment of this application.
[0121] This document describes various exemplary embodiments with reference to them. However, those skilled in the art will recognize that changes and modifications can be made to the exemplary embodiments without departing from the scope of this document. For example, various operational steps and components for performing operational steps can be implemented in different ways depending on the specific application or considering any number of cost functions associated with the operation of the system (e.g., one or more steps can be deleted, modified, or combined with other steps).
[0122] In the above embodiments, implementation can be achieved, in whole or in part, by software, hardware, firmware, or any combination thereof. Furthermore, as those skilled in the art will understand, the principles herein can be reflected in a computer program product on a computer-readable storage medium pre-loaded with computer-readable program code. Any tangible, non-transitory computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu-ray discs, etc.), flash memory, and / or the like. These computer program instructions can be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to form a machine, such that instructions executing on the computer or other programmable data processing apparatus can generate means for performing a specified function. These computer program instructions can also be stored in a computer-readable storage medium that can instruct a computer or other programmable data processing apparatus to operate in a particular manner, such that instructions stored in the computer-readable storage medium can form an article of manufacture including means for implementing the specified function. Computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to perform a series of operational steps on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions executed on the computer or other programmable apparatus provide steps for performing a specified function.
[0123] While the principles herein have been illustrated in various embodiments, numerous modifications to the structures, arrangements, proportions, elements, materials, and components, particularly suited to specific environments and operational requirements, may be used without departing from the principles and scope of this disclosure. These modifications and other alterations or alterations will be included within the scope of this document.
[0124] The foregoing specific descriptions have been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of this disclosure. Therefore, considerations for this disclosure are to be illustrative rather than restrictive, and all such modifications are to be included within its scope. Similarly, advantages, other advantages, and solutions to problems with respect to various embodiments have been described above. However, benefits, advantages, solutions to problems, and any elements that produce these, or make them more explicit, should not be construed as critical, essential, or necessary. The term “comprising” and any other variations thereof as used herein are non-exclusive inclusion, meaning that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed or not part of the process, method, system, article, or apparatus. Furthermore, the term “coupled” and any other variations thereof as used herein refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections, and / or any other connections.
[0125] Those skilled in the art will recognize that many changes can be made to the details of the above embodiments without departing from the basic principles of the invention. Therefore, the scope of the invention should be determined only by the claims.
Claims
1. An image noise estimation method based on image edge characteristics, characterized in that, include: The original image is acquired and divided into multiple sub-images to be estimated; wherein there is no overlapping region between two adjacent sub-images to be estimated; the gradient image of each sub-image to be estimated is acquired, and the effective noise estimation region in the sub-image to be estimated is determined based on the gradient image; whether the sub-image to be estimated is invalid is determined based on the area of the effective noise estimation region; invalid sub-images to be estimated are removed from the sub-images to be estimated to obtain effective sub-images to be estimated; wherein, if the area ratio of the effective noise estimation region is greater than or equal to a preset binarized gradient area ratio threshold, the sub-image to be estimated corresponding to the effective noise estimation region is taken as the effective sub-image to be estimated. A second-order difference image is obtained by performing convolution or correlation operations on the effective sub-image to be estimated using a second-order difference filter kernel; wherein, the second-order difference image contains the second-order difference information of the original image; For each second-order difference image, calculate the global average value of each second-order difference image, and multiply the global average value of each second-order difference image by... The noise estimate of each second-order difference image is obtained, and based on the noise estimate of each second-order difference image, the noise estimate of the original image is obtained; wherein, the noise estimate is used to characterize the noise level in the image, and the expression for the global average value E of the second-order difference image is: The above This indicates that the coordinates in the i-th second-order difference image are... The grayscale value of the pixel. and Let represent the width and height of the i-th second-order difference image, respectively, and let N represent the total number of pixels in the i-th second-order difference image. The noise estimate of the original image is . , Let be the noise estimate for the i-th second-order difference image. Let L be the ratio of the area of the effective noise estimation region of the sub-image to be estimated corresponding to the i-th second-order difference image to the area of the i-th second-order difference image, where L is the total number of second-order difference images.
2. The image noise estimation method as described in claim 1, characterized in that, The process involves obtaining the gradient image of the sub-image to be estimated, determining the effective noise estimation region in the sub-image based on the gradient image, determining whether the sub-image to be estimated is invalid based on the area of the effective noise estimation region, and removing invalid sub-images from the sub-image to obtain effective sub-images. Calculate the gradient magnitude of each pixel in the sub-image to be estimated to obtain the gradient image of the sub-image to be estimated; calculate the relative gray-level histogram of the gradient image. Starting from the minimum gray value of the relative gray histogram, the probability values of each gray value are accumulated until the accumulated probability value reaches or exceeds the preset cumulative probability threshold. The last accumulated gray value is then used as the binarization threshold. The effective noise estimation region in the sub-image to be estimated is obtained according to the binarization threshold, wherein the effective noise estimation region is the region in the sub-image to be estimated whose gradient magnitude is less than the binarization threshold; Calculate the area of the effective noise estimation region; wherein, the area of the effective noise estimation region represents the number of pixels in the sub-image to be estimated whose gradient magnitude is less than the binarization threshold; Based on the area of the effective noise estimation region, determine whether the sub-image to be estimated is an invalid sub-image to be estimated, and remove the invalid sub-images to be estimated from the sub-images to be estimated to obtain the effective sub-images to be estimated.
3. The image noise estimation method as described in claim 2, characterized in that, Determining whether the sub-image to be estimated is invalid based on the area of the effective noise estimation region, and removing invalid sub-images from the sub-images to obtain the effective sub-image, includes: The area ratio of the effective noise estimation region is calculated based on the area of the effective noise estimation region; wherein, the area ratio of the effective noise estimation region represents the percentage of the area of the effective noise estimation region to the area of the sub-image to be estimated corresponding to the effective noise estimation region; If the area ratio of the effective noise estimation region is less than the preset binarization gradient area ratio threshold, then the sub-image to be estimated corresponding to the effective noise estimation region is regarded as an invalid sub-image to be estimated and the invalid sub-image to be estimated is discarded; if the area ratio of the effective noise estimation region is greater than or equal to the preset binarization gradient area ratio threshold, then the sub-image to be estimated corresponding to the effective noise estimation region is regarded as the effective sub-image to be estimated.
4. An image noise estimation system based on image edge characteristics, characterized in that, include: A preprocessing module is configured to acquire an original image, divide the original image into multiple sub-images to be estimated, wherein there is no overlapping region between two adjacent sub-images to be estimated; acquire the gradient image of the sub-images to be estimated, determine the effective noise estimation region in the sub-images to be estimated based on the gradient image, determine whether the sub-image to be estimated is an invalid sub-image based on the area of the effective noise estimation region, and remove invalid sub-images to be estimated to obtain effective sub-images to be estimated; wherein, if the area ratio of the effective noise estimation region is greater than or equal to a preset binarized gradient area ratio threshold, then the sub-image to be estimated corresponding to the effective noise estimation region is taken as the effective sub-image to be estimated. The second-order difference information extraction module is configured to perform convolution or correlation operations on the effective sub-image to be estimated using a second-order difference filter kernel to obtain a second-order difference image; wherein, the second-order difference image contains the second-order difference information of the original image; The analysis and processing module is configured to acquire each second-order difference image, calculate the global average value of each second-order difference image, and multiply the global average value of each second-order difference image by... The noise estimate of each second-order difference image is obtained, and based on the noise estimate of each second-order difference image, the noise estimate of the original image is obtained; wherein, the noise estimate is used to characterize the noise level in the image, and the expression for the global average value E of the second-order difference image is: The above This indicates that the coordinates in the i-th second-order difference image are... The grayscale value of the pixel. and Let represent the width and height of the i-th second-order difference image, respectively, and let N represent the total number of pixels in the i-th second-order difference image. The noise estimate of the original image is . , Let be the noise estimate for the i-th second-order difference image. Let L be the ratio of the area of the effective noise estimation region of the sub-image to be estimated corresponding to the i-th second-order difference image to the area of the i-th second-order difference image, where L is the total number of second-order difference images.
5. A computer-readable storage medium, characterized in that, The method includes a program that can be executed by a processor to implement the image noise estimation method as described in any one of claims 1 to 3.
