Image noise reduction method and device, equipment, storage medium and program product

Abstract

The invention discloses an image noise reduction method and device, equipment, a storage medium and a program product, and the method comprises the steps: carrying out the down-sampling of a target image, and obtaining a first image; performing adaptive non-local mean noise reduction processing on the first image to obtain noise reduction intensity and a corresponding first weight; wherein the noise reduction intensity is determined by the brightness, the variance and the radius; processing the first image by using the first weight, and performing up-sampling on the processed first image to obtain a second image; and calculating a difference absolute value between the second image and the target image, and processing the target image according to the difference absolute value to obtain a final denoised image. In this way, the image signal-to-noise ratio of the image after noise reduction can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image noise reduction method, device, equipment, storage medium, and program product. Background Art

[0002] With the development of the information age, images have become an important source of information acquisition and utilization, and the requirements for image quality are getting higher and higher. In the process of CMOS (Complementary Metal Oxide Semiconductor) sensor collecting images, various noises will be introduced due to the influence of sensor material properties, working environment, electronic components and circuit structure, such as thermal noise caused by resistors, channel thermal noise of field effect transistors, photon noise, dark current noise, and light response non-uniformity noise. For the common non-local mean denoising, the difference between the pixel values ​​in the center point in the window and the neighborhood of the candidate point is calculated, and the Gaussian curve is indexed to obtain the weight of the candidate point. The smaller the difference, the higher the weight, and the larger the difference, the lower the weight. While reducing noise, the edge texture information is better protected.

[0003] The disadvantage is that after processing the image using the relevant image denoising method, the image signal-to-noise ratio of the final image is insufficient. Summary of the invention

[0004] The image denoising method, apparatus, device, storage medium, and program product provided in the present application can improve the image signal-to-noise ratio of the denoised image.

[0005] To solve the above technical problems, a technical solution adopted in the present application is: to provide an image denoising method, the method comprising: downsampling a target image to obtain a first image; performing adaptive non-local mean denoising on the first image to obtain a denoising intensity and a corresponding first weight; wherein the denoising intensity is determined by brightness, variance and radius; processing the first image using the first weight, and upsampling the processed first image to obtain a second image; calculating the absolute value of the difference between the second image and the target image, and processing the target image according to the absolute value of the difference to obtain a final denoised image.

[0006] Among them, the first image is subjected to adaptive non-local mean denoising processing to obtain denoising intensity, including: determining a first neighborhood window centered on a first pixel from the first image; calculating the average brightness, variance and radius size of all pixels in the first neighborhood window; determining a scalar brightness from a brightness scalar curve based on the average brightness; determining a scalar variance from a variance scalar curve based on the variance; determining a scalar radius from a radius scalar curve based on the radius size; and obtaining denoising intensity based on scalar brightness, scalar variance and scalar radius.

[0007] Among them, the denoising intensity is positively correlated with the scalar brightness, the denoising intensity is negatively correlated with the scalar variance, and the denoising intensity is positively correlated with the scalar radius.

[0008] Among them, performing adaptive non-local mean denoising processing on the first image to obtain a denoising intensity and a corresponding first weight includes: performing adaptive non-local mean denoising processing on the first image to obtain a denoising intensity; and determining the first weight according to the denoising intensity.

[0009] Among them, after obtaining the noise reduction intensity, it includes: determining a second neighborhood window centered on a second pixel point from the first image; calculating the brightness difference between the first neighborhood window and the second neighborhood window; determining a first weight according to the noise reduction intensity, including: determining the first weight according to the noise reduction intensity and the brightness difference.

[0010] The first weight is determined according to the noise reduction strength and the brightness difference, including: calculating a target value according to the noise reduction strength and the brightness difference; and using the target value to index a Gaussian curve to obtain the first weight.

[0011] The target image is processed according to the absolute value of the difference to obtain the final denoised image, including: performing soft threshold mapping on the absolute value of the difference to obtain a second weight; and processing the target image and the third image using the second weight to obtain the final denoised image.

[0012] To solve the above technical problems, another technical solution adopted in the present application is: to provide an image denoising device, which includes: an acquisition module, used to downsample the target image to obtain a first image; a first processing module, used to perform adaptive non-local mean denoising on the first image to obtain a denoising intensity and a corresponding first weight; wherein the denoising intensity is determined by the brightness, variance and radius; a second processing module, used to process the first image using the first weight, and upsample the processed first image to obtain a second image; a third processing module, used to process the first image using the first weight, and upsample the processed first image to obtain a second image; a fourth processing module, used to calculate the absolute value of the difference between the second image and the target image, and process the target image according to the absolute value of the difference to obtain a final denoised image.

[0013] To solve the above technical problems, another technical solution adopted in the present application is: to provide an electronic device, which includes a memory and a processor coupled to the memory, the memory stores at least one computer program, and when the at least one computer program is loaded and executed by the processor, it is used to implement the method provided by the above technical solution.

[0014] To solve the above technical problems, another technical solution adopted in the present application is: to provide a computer-readable storage medium / computer program product, wherein the computer-readable storage medium stores computer execution instructions, and the computer execution instructions are used to implement the method provided by the above technical solution when executed by the processor; the computer program product includes a computer program, and the computer program is used to implement the method provided by the above technical solution when executed by the processor.

[0015] The image denoising method, device, equipment, storage medium, and program product provided by the present application, on the basis of non-local mean denoising, adds brightness, variance, and radius size influencing factors to its denoising intensity, accurately controls the denoising intensity, and obtains a higher signal-to-noise ratio image. Secondly, an adaptive back-overlapping process is also added. According to the difference between the target image and the second image after denoising, the details of the target image are adaptively back-overlapped, so that the control of the denoising intensity is more flexible, which is conducive to improving the signal-to-noise ratio of the final image. In addition, the target image is down-sampled to obtain a first image, and then the first image is subjected to adaptive non-local mean processing, and then the denoised image is up-sampled. This method is conducive to saving hardware area and overhead and reducing chip costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0017] Figure 1 It is a flowchart of an embodiment of an image noise reduction method in the present application;

[0018] Figure 2 is a flowchart of another embodiment of the image noise reduction method in the present application;

[0019] Figure 3 is a flowchart of another embodiment of the image noise reduction method in the present application;

[0020] Figure 4 is a flowchart of another embodiment of the image noise reduction method in the present application;

[0021] Figure 5 yes Figure 4 A flow chart of an embodiment of step 405;

[0022] Figure 6 yes Figure 4 A flow chart of an embodiment of step 412;

[0023] Figure 7It is a schematic diagram of the relationship between average brightness and scalar brightness provided in this application;

[0024] Figure 8 It is a schematic diagram of the relationship between variance and scalar variance provided in this application;

[0025] Fig. 9 It is a schematic diagram of the relationship between the radius and the scalar radius provided in this application;

[0026] Fig.10 is a flowchart of another embodiment of the image noise reduction method in the present application;

[0027] Fig.11 It is a structural schematic diagram of an embodiment of an image noise reduction device in the present application;

[0028] Fig.12 It is a structural schematic diagram of an embodiment of an electronic device in the present application;

[0029] Fig.13 It is a schematic diagram of the structure of an embodiment of a computer-readable storage medium in the present application;

[0030] Fig.14 It is a structural diagram of an embodiment of a computer program product in the present application. DETAILED DESCRIPTION

[0031] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0032] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0033] With the development of the information age, images have become an important source for obtaining and utilizing information, and the requirements for image quality are becoming increasingly higher. In the process of CMOS sensor acquisition of images, various noises will be introduced due to the influence of sensor material properties, working environment, electronic components and circuit structure, such as thermal noise caused by resistors, channel thermal noise of field effect transistors, photon noise, dark current noise, and light response non-uniformity noise. For the common non-local mean denoising, the difference between the pixel values ​​in the center point in the window and the neighborhood of the candidate point is calculated, and the Gaussian curve is indexed to obtain the weight of the candidate point. The smaller the difference, the higher the weight, and the larger the difference, the lower the weight. While reducing noise, the edge texture information is better protected.

[0034] The disadvantage is that after processing the image using the relevant image denoising method, the image signal-to-noise ratio of the final image is insufficient.

[0035] Based on this, the present application proposes to use the brightness, variance and radius of the target image to determine the denoising strength, which can achieve accurate control of the denoising strength, and thus the image signal-to-noise ratio of the final denoised image is higher. For details, please refer to any of the following embodiments.

[0036] See also Figure 1 , Figure 1 : is a flow chart of an embodiment of an image noise reduction method in the present application. The method comprises:

[0037] Step 11: Get the target image.

[0038] In some embodiments, the target image can be acquired by a related image acquisition component, such as by using a mobile phone camera or a camera.

[0039] Step 12: Perform adaptive non-local mean denoising on the target image to obtain denoising intensity; wherein the denoising intensity is determined by brightness, variance and radius.

[0040] In some embodiments, non-local mean denoising is a denoising algorithm based on image block similarity, and its core idea is that regions with similar textures in an image should also have similar pixel values. Therefore, the non-local mean denoising algorithm calculates the similarity between all pixel blocks in the image and uses the information of similar regions to estimate the true value of the current pixel, thereby achieving denoising.

[0041] Exemplarily, the steps of the non-local means denoising algorithm are as follows:

[0042] 1. Image block partitioning: Divide the image into NxN pixel blocks.

[0043] 2. Similarity measurement: Calculate the similarity between each pixel block and all other pixel blocks. Similarity measurement methods include Euclidean distance, Manhattan distance, cosine similarity, etc.

[0044] 3. Weight calculation: Based on the similarity measurement results, calculate the weight of each pixel block to the current pixel. The larger the weight, the higher the similarity and the greater the impact on the current pixel.

[0045] 4. Denoising: Use the weighted averaging method to perform weighted averaging on the pixel values ​​of all pixel blocks to obtain the denoising result of the current pixel.

[0046] In some embodiments, the higher the noise reduction strength, the stronger the denoising effect on the image. The noise reduction strength is related to the brightness, variance, and radius of the image. Therefore, the brightness, variance, and radius size influencing factors are added to the noise reduction strength to accurately control the noise reduction strength and obtain a higher signal-to-noise ratio image.

[0047] Step 13: Determine a first weight according to the noise reduction strength.

[0048] The first weight represents the similarity between pixel points x and y in the target image.

[0049] In some embodiments, the Gaussian curve may be indexed according to the noise reduction strength to obtain the first weight.

[0050] Step 14: Process the target image using the first weight to obtain a final denoised image.

[0051] In some embodiments, the first weight is used to perform weighted averaging on corresponding pixels in the target image to obtain a final denoised image.

[0052] In this embodiment, the denoising intensity is determined by using the brightness, variance and radius in the target image, so that the denoising intensity can be accurately controlled, and thus the image signal-to-noise ratio of the final denoised image is higher.

[0053] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of another embodiment of the image noise reduction method in the present application. The method comprises:

[0054] Step 21: Get the target image.

[0055] Step 22: Determine a first neighborhood window centered at the first pixel from the target image.

[0056] In some embodiments, the first neighborhood window may be 3*3 in size, 5*5 in size, or 7*7 in size. The size of the neighborhood window may be determined according to the actual resolution of the target image.

[0057] Step 23: Calculate the average brightness, variance and radius of all pixels in the first neighborhood window.

[0058] In some embodiments, the corresponding average brightness can be calculated by obtaining the brightness sum of all pixels in the first neighborhood window.

[0059] In some embodiments, the variance is calculated using the brightness of each pixel in the first neighborhood window and the average brightness.

[0060] In some embodiments, the radius of each pixel is calculated using the pixel coordinates of each pixel in the first neighborhood window and the width and height of the target image.

[0061] Step 24: Get the noise reduction strength based on the average brightness, variance and radius.

[0062] In some embodiments, a scalar brightness is determined from a brightness scalar curve according to average brightness, a scalar variance is determined from a variance scalar curve according to variance, a scalar radius is determined from a radius scalar curve according to radius size, and noise reduction strength is obtained according to scalar brightness, scalar variance, and scalar radius.

[0063] In some embodiments, the noise reduction strength is positively correlated with the scalar brightness, the noise reduction strength is negatively correlated with the scalar variance, and the noise reduction strength is positively correlated with the scalar radius.

[0064] Step 25: Determine a second neighborhood window centered at the second pixel from the target image.

[0065] In some embodiments, the first pixel point and the second pixel point are different pixel points. The second neighborhood window can be used as a search window.

[0066] Step 26: Calculate the brightness difference sum between the first neighborhood window and the second neighborhood window.

[0067] In some embodiments, the sum of the brightness differences between the pixels in the first neighborhood window and the second neighborhood window may be calculated by traversing the pixels in the first neighborhood window and the second neighborhood window.

[0068] Step 27: Determine a first weight according to the noise reduction strength and the brightness difference.

[0069] In some embodiments, a corresponding value can be calculated based on the noise reduction strength and the brightness difference. The value is used to index the Gaussian curve to obtain a first weight. For example, the value can be used as the X-axis value in the Gaussian curve, and the corresponding Y-axis value can be used as the first weight.

[0070] Step 28: Process the target image using the first weight to obtain a final denoised image.

[0071] In some embodiments, the weight is used to perform weighted averaging on the pixels in the target image to obtain a final denoised image.

[0072] In this embodiment, the denoising intensity is determined by using the brightness, variance and radius in the target image, so that the denoising intensity can be accurately controlled, and thus the image signal-to-noise ratio of the final denoised image is higher.

[0073] See also Figure 3 , Figure 3 FIG. 1 is a flow chart of another embodiment of the image noise reduction method in the present application. The method comprises:

[0074] Step 31: Get the target image.

[0075] Step 32: Downsample the target image to obtain a first image.

[0076] Downsampling the target image can reduce the complexity of the image and the hardware requirements, which is beneficial for the corresponding processing equipment to save hardware area and overhead and reduce hardware costs.

[0077] Furthermore, since the complexity of the image is reduced, the denoising efficiency can be improved faster.

[0078] Step 33: Perform adaptive non-local mean denoising on the first image to obtain denoising intensity; wherein the denoising intensity is determined by brightness, variance and radius.

[0079] In some embodiments, non-local mean denoising is a denoising algorithm based on image block similarity, and its core idea is that regions with similar textures in an image should also have similar pixel values. Therefore, the non-local mean denoising algorithm calculates the similarity between all pixel blocks in the image and uses the information of similar regions to estimate the true value of the current pixel, thereby achieving denoising.

[0080] Exemplarily, the steps of the non-local means denoising algorithm are as follows:

[0081] 1. Image block division: Divide the first image into pixel blocks of size NxN.

[0082] 2. Similarity measurement: Calculate the similarity between each pixel block and all other pixel blocks. Similarity measurement methods include Euclidean distance, Manhattan distance, cosine similarity, etc.

[0083] 3. Weight calculation: Based on the similarity measurement results, calculate the weight of each pixel block to the current pixel. The larger the weight, the higher the similarity and the greater the impact on the current pixel.

[0084] 4. Denoising: Use the weighted averaging method to perform weighted averaging on the pixel values ​​of all pixel blocks to obtain the denoising result of the current pixel.

[0085] In some embodiments, the higher the noise reduction strength, the stronger the noise reduction effect on the first image. The noise reduction strength is related to the brightness, variance, and radius of the first image. Therefore, the brightness, variance, and radius size influencing factors are added to the noise reduction strength to accurately control the noise reduction strength and obtain an image with a higher signal-to-noise ratio.

[0086] Step 34: Determine a first weight according to the noise reduction strength.

[0087] The first weight represents the similarity between pixel points x and y in the first image.

[0088] In some embodiments, the Gaussian curve may be indexed according to the noise reduction strength to obtain the first weight.

[0089] Step 35: Process the first image using the first weight to obtain a second image.

[0090] In some embodiments, the first weight is used to perform weighted averaging on corresponding pixels in the first image to obtain the second image.

[0091] Step 36: Upsample the second image to obtain a third image.

[0092] In some embodiments, the second image is upsampled to obtain a third image, wherein the size of the third image is the same as the size of the target image.

[0093] Step 37: Calculate the absolute value of the difference between the third image and the target image.

[0094] In some embodiments, the absolute value of the difference may be obtained by calculating the difference between the pixels of the third image and the target image.

[0095] Step 38: Process the target image according to the absolute value of the difference to obtain the final denoised image.

[0096] In some embodiments, a corresponding weight may be obtained according to the absolute value of the difference, and then the target image may be processed according to the weight to obtain a final denoised image.

[0097] In this embodiment, on the basis of non-local mean denoising, brightness, variance and radius size influencing factors are added to the denoising intensity, the denoising intensity is accurately controlled, and a higher signal-to-noise ratio image is obtained. Secondly, adaptive back-stack processing is also added. According to the difference between the target image and the third image after denoising, the details of the target image are adaptively back-stacked, so that the control of denoising intensity is more flexible, which is conducive to improving the signal-to-noise ratio of the final image. In addition, the target image is down-sampled to obtain the first image, and then the first image is subjected to adaptive non-local mean processing, and then the second image after denoising is up-sampled. This method is conducive to saving hardware area and overhead and reducing chip costs.

[0098] See also Figure 4 , Figure 4 FIG. 1 is a flow chart of another embodiment of the image noise reduction method in the present application. The method comprises:

[0099] Step 401: Acquire a target image.

[0100] Step 402: down-sample the target image to obtain a first image.

[0101] Step 403: Determine a first neighborhood window centered at the first pixel from the first image.

[0102] Step 404: Calculate the average brightness, variance and radius of all pixels in the first neighborhood window.

[0103] In some embodiments, the corresponding average brightness can be calculated by obtaining the brightness sum of all pixels in the first neighborhood window.

[0104] In some embodiments, the variance is calculated using the brightness of each pixel in the first neighborhood window and the average brightness.

[0105] In some embodiments, the radius of each pixel is calculated using the pixel coordinates of each pixel in the first neighborhood window and the width and height of the target image.

[0106] Step 405: Obtain the noise reduction strength according to the average brightness, variance and radius.

[0107] In some embodiments, see Figure 5 , step 405 may be the following process:

[0108] Step 51: Determine the scalar brightness from the brightness scalar curve according to the average brightness.

[0109] In some embodiments, the brightness scalar curve can be determined in advance. Therefore, the average brightness can be substituted into the brightness scalar curve to obtain the scalar brightness. The noise reduction intensity is positively correlated with the scalar brightness.

[0110] Step 52: Determine the scalar variance from the variance scalar curve according to the variance.

[0111] In some embodiments, the variance scalar curve can be determined in advance. Therefore, the variance can be substituted into the variance scalar curve to obtain the scalar variance. The noise reduction strength is negatively correlated with the scalar variance.

[0112] Step 53: Determine the scalar radius from the radius scalar curve according to the radius size.

[0113] In some embodiments, the radius scalar curve can be determined in advance. Therefore, the radius size can be substituted into the radius scalar curve to obtain the scalar radius. The noise reduction strength is positively correlated with the scalar radius.

[0114] Step 54: Obtain the noise reduction strength according to the scalar brightness, the scalar variance, and the scalar radius.

[0115] In some embodiments, the noise reduction strength may be obtained by multiplying the base noise reduction strength, the scalar brightness, the scalar variance, and the scalar radius.

[0116] Step 406: Determine a second neighborhood window centered at the second pixel point from the first image.

[0117] Step 407: Calculate the sum of brightness differences between the first neighborhood window and the second neighborhood window.

[0118] In some embodiments, the sum of the brightness differences between the pixels in the first neighborhood window and the second neighborhood window may be calculated by traversing the pixels in the first neighborhood window and the second neighborhood window.

[0119] Step 408: Determine a first weight according to the noise reduction strength and the brightness difference.

[0120] In some embodiments, a corresponding target value can be calculated based on the noise reduction strength and the brightness difference. The target value is used to index the Gaussian curve to obtain a first weight. For example, the value can be used as the X-axis value in the Gaussian curve, and the corresponding Y-axis value can be used as the first weight.

[0121] Step 409: Process the first image using the first weight to obtain a second image.

[0122] Step 410: Upsample the second image to obtain a third image.

[0123] Step 411: Calculate the absolute value of the difference between the third image and the target image.

[0124] Step 412: Process the target image according to the absolute value of the difference to obtain a final denoised image.

[0125] Steps 409 to 412 have the same or similar technical solutions as any embodiment of the present application and will not be described in detail here.

[0126] In some embodiments, see Figure 6 , step 412 may be the following process:

[0127] Step 61: Perform soft threshold mapping processing on the absolute value of the difference to obtain a second weight.

[0128] Step 62: Process the target image and the third image using the second weight to obtain a final denoised image.

[0129] In some embodiments, the product between the second weight and the target image may be calculated, and the product between the third image and (1-the second weight) may be calculated, and the two products may be summed to obtain a final denoised image.

[0130] In this embodiment, on the basis of non-local mean denoising, brightness, variance and radius size influencing factors are added to the denoising intensity, the denoising intensity is accurately controlled, and a higher signal-to-noise ratio image is obtained. Secondly, adaptive back-stack processing is also added. According to the difference between the target image and the third image after denoising, the details of the target image are adaptively back-stacked, so that the control of denoising intensity is more flexible, which is conducive to improving the signal-to-noise ratio of the final image. In addition, the target image is down-sampled to obtain the first image, and then the first image is subjected to adaptive non-local mean processing, and then the second image after denoising is up-sampled. This method is conducive to saving hardware area and overhead and reducing chip costs.

[0131] Furthermore, the principle of producing the above effect is as follows:

[0132] The CMOS (Complementary Metal Oxide Semiconductor) noise model follows the Gaussian Poisson distribution, and the noise is proportional to the signal strength, that is, the greater the brightness, the greater the noise. For different brightness areas, different noise reduction intensities are adopted to obtain a higher signal-to-noise ratio noise reduction image.

[0133] The larger the local variance of an image, the richer the details in that area, and the lower the intensity of noise reduction should be.

[0134] Due to the influence of LSC (lens shading correcting) and other factors, the larger the radius of the image midpoint, the greater the noise. At this time, a higher intensity of noise reduction should be adopted to ensure that the image noise is balanced after noise reduction in the radial direction.

[0135] The greater the difference between the original image and the image after denoising, the richer the details in the area, and the more original images should be overlapped. This method of adaptively controlling the overlap strength makes the denoising strength control more flexible, which is conducive to improving the signal-to-noise ratio of the final image.

[0136] By performing adaptive non-local mean processing on the small image (the first image), less data is processed and the hardware throughput is lower, which is beneficial to saving hardware area and overhead and reducing chip costs.

[0137] In an application scenario, the original image (target image) is downsampled to obtain a first image image_ds.

[0138] Then, the first image image_ds is subjected to an adaptive non-local means denoising process to obtain a second image image_ds_nlm. Exemplarily, the implementation process of the adaptive non-local means denoising is as follows.

[0139] (1) Obtain a neighborhood window W1 centered on point x (corresponding to the first neighborhood window mentioned above)

[0140] (2) Calculate the average brightness luma, variance std2 and radius size radius2 of the points in the neighborhood window W1.

[0141]

[0142] radius2=(i1-(height / 2)) 2 +(j1-(width / 2)) 2 .

[0143] Wherein, ds represents the radius of the neighborhood window W1, height represents the image height of the first image, width represents the image width of the first image, i and j represent the coordinates of the pixel point in the first image, and u and v represent the traversal coordinates of the pixel point.

[0144] (3) Calculate the noise reduction strength of the point. For example, the following formula is used for calculation: strength = strength_base*scalar_luma*scalar_std2*scalar_radius2. Wherein, strength represents the noise reduction strength, strength_base represents the basic noise reduction strength, scalar_luma represents the scalar brightness, scalar_std2 represents the scalar variance, and scalar_radius2 represents the scalar radius.

[0145] For the average brightness luma, scalar_luma is calculated by looking up the table. Figure 7Calculate scalar_luma by looking up the table. Calculate scalar_std2 by looking up the table for variance std2. For example, Figure 8 Calculate scalar_std2 by looking up the table. For radius2, calculate scalar_radius2 by looking up the table. For example, Fig. 9 Calculate scalar_radius2 by looking up the table. The CMOS image noise model follows the Gaussian Poisson distribution, and the noise is proportional to the signal strength, that is, the greater the brightness, the greater the noise. Therefore, the noise reduction intensity should be proportional to the brightness. The greater the local variance of the image, the richer the details in the area, and the lower the intensity of noise reduction should be. Therefore, the noise reduction intensity should be inversely proportional to the variance. Due to the influence of LSC, the larger the radius of the image midpoint, the greater the noise. At this time, a higher intensity of noise reduction should be adopted to ensure that the image noise after noise reduction is balanced in the radial direction. Therefore, the noise reduction intensity should be proportional to the radius.

[0146] (4) Loop through the search window -Ds to Ds to obtain the neighborhood window W2 centered on point y (corresponding to the second neighborhood window mentioned above)

[0147] (5) Calculate the sum sad of the brightness differences between windows W1 and W2.

[0148] Exemplarily, sad is calculated using the following formula.

[0149]

[0150] (6) Combine the strength from step (3) and the sad from step (5), index the Gaussian curve, and calculate the weight w.

[0151] (7) Calculate the weighted average of the points in the final window

[0152]

[0153] Among them, I denoised (x) represents the second image, w(x,y) represents the weight, and I(y) represents the first image.

[0154] Further, the second image image_ds_nlm is upsampled to obtain a third image, the absolute value diff of the difference between the third image and the original image is calculated, a soft threshold mapping is performed on the absolute value to obtain a weight, and the original image is adaptively overlapped based on the weight.

[0155] (1) Calculate diff: diff = abs(image_initial - image_ds_nlm_up), where image_initial represents the original image and image_ds_nlm_up represents the third image.

[0156] (2) Soft threshold mapping: Perform soft threshold mapping on diff to obtain the weight weight.

[0157] (3) Adaptive overlap processing: image_denoised = image_initial*weight+image_ds_nlm_up*(1–weight).

[0158] See also Fig.10 , Fig.10 FIG. 1 is a flow chart of another embodiment of the image noise reduction method in the present application. The method comprises:

[0159] Step 101: down-sample the target image to obtain a first image.

[0160] Step 102: Perform adaptive non-local mean denoising on the first image to obtain a denoising intensity and a corresponding first weight; wherein the denoising intensity is determined by brightness, variance and radius.

[0161] Downsampling the target image can reduce the complexity of the image and the hardware requirements, which is beneficial for the corresponding processing equipment to save hardware area and overhead and reduce hardware costs.

[0162] Furthermore, since the complexity of the image is reduced, the denoising efficiency can be improved faster.

[0163] In some embodiments, non-local mean denoising is a denoising algorithm based on image block similarity, and its core idea is that regions with similar textures in an image should also have similar pixel values. Therefore, the non-local mean denoising algorithm calculates the similarity between all pixel blocks in the image and uses the information of similar regions to estimate the true value of the current pixel, thereby achieving denoising.

[0164] Exemplarily, the steps of the non-local means denoising algorithm are as follows:

[0165] 1. Image block division: Divide the first image into pixel blocks of size NxN.

[0166] 2. Similarity measurement: Calculate the similarity between each pixel block and all other pixel blocks. Similarity measurement methods include Euclidean distance, Manhattan distance, cosine similarity, etc.

[0167] 3. Weight calculation: Based on the similarity measurement results, calculate the weight of each pixel block to the current pixel. The larger the weight, the higher the similarity and the greater the impact on the current pixel.

[0168] 4. Denoising: Use the weighted averaging method to perform weighted averaging on the pixel values ​​of all pixel blocks to obtain the denoising result of the current pixel.

[0169] In some embodiments, the higher the noise reduction strength, the stronger the noise reduction effect on the first image. The noise reduction strength is related to the brightness, variance, and radius of the first image. Therefore, the brightness, variance, and radius size influencing factors are added to the noise reduction strength to accurately control the noise reduction strength and obtain an image with a higher signal-to-noise ratio.

[0170] The first weight represents the similarity between pixel points x and y in the first image.

[0171] In some embodiments, the Gaussian curve may be indexed according to the noise reduction strength to obtain the first weight.

[0172] Step 103: Process the first image using the first weight, and upsample the processed first image to obtain a second image.

[0173] In some embodiments, the first weight is used to weight and average the corresponding pixels in the first image to obtain an intermediate image. The intermediate image is upsampled to obtain a second image. At this time, the size of the second image is the same as the size of the target image.

[0174] Step 104: Calculate the absolute value of the difference between the second image and the target image, and process the target image according to the absolute value of the difference to obtain a final denoised image.

[0175] In some embodiments, the absolute value of the difference may be obtained by calculating the difference between the pixels of the second image and the target image.

[0176] In some embodiments, a corresponding weight may be obtained according to the absolute value of the difference, and then the target image may be processed according to the weight to obtain a final denoised image.

[0177] In this embodiment, on the basis of non-local mean denoising, brightness, variance and radius size influencing factors are added to the denoising intensity, the denoising intensity is accurately controlled, and a higher signal-to-noise ratio image is obtained. Secondly, adaptive back-stack processing is also added. According to the difference between the target image and the denoised image, the details of the target image are adaptively back-stacked, making the denoising intensity control more flexible, which is conducive to improving the signal-to-noise ratio of the final image. In addition, the target image is down-sampled to obtain a first image, and then the first image is subjected to adaptive non-local mean processing, and then the denoised second image is up-sampled. This method is conducive to saving hardware area and overhead and reducing chip costs.

[0178] See also Fig.11 , Fig.111 is a schematic diagram of the structure of an embodiment of an image noise reduction device in the present application. The image noise reduction device 110 comprises: an acquisition module 111 , a first processing module 112 , a second processing module 113 , a third processing module 114 and a fourth processing module 115 .

[0179] The acquisition module 111 is used to downsample the target image to obtain a first image.

[0180] The first processing module 112 is used to perform adaptive non-local mean denoising on the first image to obtain a denoising intensity and a corresponding first weight; wherein the denoising intensity is determined by brightness, variance and radius.

[0181] The second processing module 113 is used to process the first image using the first weight, and upsample the processed first image to obtain a second image.

[0182] The third processing module 114 is used to process the first image using the first weight, and upsample the processed first image to obtain a second image.

[0183] The fourth processing module 115 is used to calculate the absolute value of the difference between the second image and the target image, and process the target image according to the absolute value of the difference to obtain a final denoised image.

[0184] In some embodiments, the first processing module 112 is further used to determine a first neighborhood window centered on the first pixel from the first image; calculate the average brightness, variance, and radius of all pixels in the first neighborhood window; determine the scalar brightness from the brightness scalar curve according to the average brightness; determine the scalar variance from the variance scalar curve according to the variance; determine the scalar radius from the radius scalar curve according to the radius; and obtain the noise reduction strength according to the scalar brightness, scalar variance, and scalar radius. In some embodiments, the noise reduction strength is positively correlated with the scalar brightness, negatively correlated with the scalar variance, and positively correlated with the scalar radius.

[0185] In some embodiments, the first processing module 112 is further configured to perform adaptive non-local mean denoising on the first image to obtain a denoising intensity; and determine a first weight according to the denoising intensity.

[0186] In some embodiments, after obtaining the noise reduction intensity, the first processing module 112 is also used to determine a second neighborhood window centered on a second pixel from the first image; calculate the brightness difference between the first neighborhood window and the second neighborhood window, and the first processing module 112 is also used to determine a first weight based on the noise reduction intensity and the brightness difference.

[0187] In some embodiments, the first processing module 112 is further configured to calculate a target value according to the noise reduction intensity and the brightness difference; and use the target value to index the Gaussian curve to obtain a first weight.

[0188] In some embodiments, the fourth processing module 115 is further used to perform soft threshold mapping processing on the absolute value of the difference to obtain a second weight; and use the second weight to process the target image and the third image to obtain a final denoised image.

[0189] It can be understood that the acquisition module 111 , the first processing module 112 , the second processing module 113 , the third processing module 114 and the fourth processing module 115 cooperate with each other to implement the method provided by any of the above embodiments.

[0190] See also Fig.12 , Fig.12 1 is a schematic diagram of the structure of an embodiment of an electronic device in the present application. The electronic device 120 includes a memory 121 and a processor 122 coupled to the memory 121. The memory 121 stores at least one computer program. When the at least one computer program is loaded and executed by the processor 122, it is used to implement the following method:

[0191] Down-sample the target image to obtain a first image; perform adaptive non-local mean denoising on the first image to obtain a denoising intensity and a corresponding first weight; wherein the denoising intensity is determined by brightness, variance and radius; process the first image using the first weight, and up-sample the processed first image to obtain a second image; calculate the absolute value of the difference between the second image and the target image, and process the target image according to the absolute value of the difference to obtain a final denoised image.

[0192] It can be understood that when at least one computer program is loaded and executed by the processor 122, it is also used to implement the method of any of the above embodiments.

[0193] See also Fig.13 , Fig.13 1 is a schematic diagram of a computer-readable storage medium in an embodiment of the present application. The computer-readable storage medium 130 has a computer-executable instruction 131, which is used to implement the following method when loaded and executed by a processor:

[0194] Down-sample the target image to obtain a first image; perform adaptive non-local mean denoising on the first image to obtain a denoising intensity and a corresponding first weight; wherein the denoising intensity is determined by brightness, variance and radius; process the first image using the first weight, and up-sample the processed first image to obtain a second image; calculate the absolute value of the difference between the second image and the target image, and process the target image according to the absolute value of the difference to obtain a final denoised image.

[0195] It can be understood that when the computer execution instruction 131 is loaded and executed by the processor, it is used to implement the method of any of the above embodiments.

[0196] See also Fig.14 , Fig.14 1 is a schematic diagram of a structure of an embodiment of a computer program product in the present application. The computer program product 140 includes a computer program 141, and when the computer program 141 is executed by a processor, it is used to implement the following method:

[0197] Down-sample the target image to obtain a first image; perform adaptive non-local mean denoising on the first image to obtain a denoising intensity and a corresponding first weight; wherein the denoising intensity is determined by brightness, variance and radius; process the first image using the first weight, and up-sample the processed first image to obtain a second image; calculate the absolute value of the difference between the second image and the target image, and process the target image according to the absolute value of the difference to obtain a final denoised image.

[0198] It can be understood that when the computer program 141 is loaded and executed by the processor, it is used to implement the method of any of the above embodiments.

[0199] In summary, the image denoising method, apparatus, device, storage medium, and program product provided by the present application utilize the brightness, variance, and radius size in the target image to determine the denoising intensity, and can achieve precise control of the denoising intensity, thereby ultimately achieving a higher signal-to-noise ratio for the denoised image.

[0200] Furthermore, on the basis of non-local mean denoising, the brightness, variance and radius size influencing factors are added to its denoising intensity to accurately control the denoising intensity and obtain a higher signal-to-noise ratio image. Secondly, adaptive backlapping processing is also added. According to the difference between the target image and the third image after denoising, the details of the target image are adaptively backlapped, making the control of denoising intensity more flexible and conducive to improving the signal-to-noise ratio of the final image. In addition, the target image is downsampled to obtain the first image, and then the first image is subjected to adaptive non-local mean processing, and then the second image after denoising is upsampled. This method is conducive to saving hardware area and overhead and reducing chip costs.

[0201] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0202] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0203] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0204] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0205] The above description is only an implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image denoising method, characterized in that: The method comprises: Downsampling the target image to obtain a first image; Performing adaptive non-local mean denoising on the first image to obtain a denoising intensity and a corresponding first weight; wherein the denoising intensity is determined by brightness, variance and radius; Processing the first image using the first weight, and upsampling the processed first image to obtain a second image; The absolute value of the difference between the second image and the target image is calculated, and the target image is processed according to the absolute value of the difference to obtain a final denoised image.

2. The method according to claim 1, characterized in that The performing adaptive non-local mean denoising processing on the first image to obtain denoising intensity includes: Determine a first neighborhood window centered at a first pixel from the first image; Calculate the average brightness, variance and radius of all pixels in the first neighborhood window; determining a scalar brightness from a brightness scalar curve according to the average brightness; determining a scalar variance from a variance scalar curve based on the variance; Determining a scalar radius from a radius scalar curve according to the radius size; The noise reduction intensity is obtained according to the scalar brightness, the scalar variance, and the scalar radius.

3. The method according to claim 2, characterized in that The noise reduction strength is positively correlated with the scalar brightness, the noise reduction strength is negatively correlated with the scalar variance, and the noise reduction strength is positively correlated with the scalar radius.

4. The method according to claim 1, characterized in that: The step of performing adaptive non-local mean denoising on the first image to obtain a denoising intensity and a corresponding first weight includes: Performing adaptive non-local mean denoising on the first image to obtain denoising intensity; A first weight is determined according to the noise reduction strength.

5. The method according to claim 4, characterized in that After obtaining the noise reduction strength, the method further comprises: Determine a second neighborhood window centered at a second pixel from the first image; Calculating a brightness difference between the first neighborhood window and the second neighborhood window; The determining of the first weight according to the noise reduction strength includes: The first weight is determined according to the noise reduction strength and the brightness difference.

6. The method according to claim 5, characterized in that The determining the first weight according to the noise reduction intensity and the brightness difference includes: Calculating a target value according to the noise reduction intensity and the brightness difference; The first weight is obtained by using the target value to index a Gaussian curve.

7. The method according to claim 1, characterized in that The step of processing the target image according to the absolute value of the difference to obtain a final denoised image includes: Performing soft threshold mapping processing on the absolute value of the difference to obtain a second weight; The target image and the third image are processed using the second weight to obtain a final denoised image.

8. An image noise reduction device, characterized in that: The image noise reduction device comprises: An acquisition module, used for downsampling the target image to obtain a first image; A first processing module, configured to perform adaptive non-local mean denoising on the first image to obtain a denoising intensity and a corresponding first weight; wherein the denoising intensity is determined by brightness, variance and radius; a second processing module, configured to process the first image using the first weight, and upsample the processed first image to obtain a second image; a third processing module, configured to process the first image using the first weight, and upsample the processed first image to obtain a second image; The fourth processing module is used to calculate the absolute value of the difference between the second image and the target image, and process the target image according to the absolute value of the difference to obtain a final denoised image.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor coupled to the memory, the memory stores at least one computer program, and when the at least one computer program is loaded and executed by the processor, it is used to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium / computer program product, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor; The computer program product comprises a computer program, which implements the method according to any one of claims 1 to 7 when executed by a processor.

Images