A denoising method based on image spatial domain

Through the noise reduction method based on image space, adaptive threshold processing is performed using HVS degree and local texture evaluation, which solves the problems of complex image noise reduction calculation and difficult hardware implementation in the prior art, and achieves rapid denoising and efficient image reduction Noise effect, suitable for a wide range of sensor data and meets stringent latency and bandwidth requirements in the medical industry.

CN114881877BActive Publication Date: 2024-11-05SUZHOU FRANKENMAN MEDICAL EQUIP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210486229.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-06
Publication Date
2024-11-05
Estimated Expiration
2042-05-06

AI Technical Summary

Technical Problem

The existing image noise reduction method is complex to calculate, takes time and is difficult to implement hardware. Especially in areas with strict bandwidth and latency requirements, it requires a lot of resources and costs, and when the light source is not illuminated enough, the image signal-to-noise ratio is reduced, and the details are lost, which affects the impact. Image quality.

Method used

Using the denoising method based on image space, the local noise evaluation function is designed through filter window setting, HVS degree evaluation, local texture evaluation and adaptive threshold processing, and the image estimation value after denoising is calculated using weighted summation, and the observation conditions are based on Adjust the noise reduction threshold parameter.

Benefits of technology

It reduces the computational processing volume of image noise reduction, quickly denoising, reduces hardware equipment requirements, adapts to the raw image data of multiple sensors, has good compatibility and flexibility, and can meet latency and bandwidth requirements in industries such as medical care.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114881877B_ABST
    Figure CN114881877B_ABST
Patent Text Reader

Abstract

The present invention discloses a denoising method based on image spatial domain, comprising the following steps: setting a filter window for the image data to be processed, making a difference between the center point value of the filter window and the valid neighborhood point value around the center point value and taking the absolute value to form a filter set accordingly; using the HVS degree evaluation function to obtain the HVS degree; inputting the maximum point value of the filter window into the local texture evaluation function for processing to obtain the local texture evaluation degree; performing adaptive threshold function processing on the maximum value and the minimum value in the set to obtain an adaptive threshold, and performing adaptive weight function processing on each element in the set according to the adaptive threshold to obtain the weight of the corresponding point of each element; and calculating the estimated value of the denoised image by weighted summation. The denoising method based on image spatial domain provided by the present invention reduces the amount of computational processing of image denoising, can quickly denoise, and reduces the requirements for related hardware equipment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image noise suppression, and in particular to a noise reduction method based on image spatial domain. Background Art

[0002] During image acquisition, varying degrees of noise are introduced by the sensor and its surrounding environment. This, especially under insufficient illumination, can reduce the signal-to-noise ratio (SNR) and cause a loss of detail and texture, compromising image quality. This can affect the final image quality and, in medical images, can even drown out the target signal, making lesion identification difficult. International standards organizations have also defined specific noise quantification processes. One comprehensive noise metric that represents the human eye's perception of an image is visual noise (VN). This VN factor accounts for the human eye's sensitivity to grayscale and color signals at different frequencies. A common approach to designing VN-based noise reduction algorithms is to filter noise in the transform domain based on the human eye's contrast sensitivity function (CSF). This method then recovers the signal after noise removal through an inverse transform. However, this method is computationally complex, time-consuming, and challenging to implement in hardware. This approach requires significant resources and cost in applications with stringent bandwidth and latency requirements. Summary of the Invention

[0003] In order to solve the problems of the prior art, the present invention provides a noise reduction method based on image spatial domain, and the technical solution is as follows:

[0004] The present invention provides a noise reduction method based on image spatial domain, comprising the following steps:

[0005] S1, set the filter window for the image data to be processed, and record the center point value of the filter window as P c , the valid neighborhood point values ​​around the center point value are recorded as P0, P1, P2, ..., P n , and accordingly form the set {|P c -P i |} i=0,1,2...n , denoted as D, D i For each element in the set, the maximum and minimum values ​​in the set are respectively denoted as D max and D min ;

[0006] S2, according to the center point value P of the filtering window c , use the HVS degree evaluation function to obtain the HVS degree, denoted as d1;

[0007] S3, designing a local texture evaluation function according to the HVS degree d1 and a preset noise reduction threshold parameter, inputting the maximum point value of the filter window into the local texture evaluation function for processing to obtain a local texture evaluation degree, recorded as d2;

[0008] S4, performing adaptive threshold function processing on the maximum value and the minimum value in the set according to the local texture evaluation degree d2 to obtain an adaptive threshold;

[0009] S5. Performing adaptive weight function processing on each element in the set according to the adaptive threshold to obtain the weight of the corresponding point of each element;

[0010] S6. Calculate the estimated value of the image after noise reduction by using a weighted summation method.

[0011] Furthermore, according to different observation conditions, the noise reduction threshold parameters are adjusted accordingly; the noise reduction threshold parameters include a first noise reduction threshold parameter, a second noise reduction threshold parameter and a third noise reduction threshold parameter, which are respectively denoted as E1, E2, and E3. The first noise reduction threshold parameter is positively correlated with the maximum brightness of the environment in which the display device is located, the second noise reduction threshold parameter is positively correlated with the maximum brightness output by the display device, and the third noise reduction threshold parameter is positively correlated with the visual angle of the image observer to the unit pixel of the display device.

[0012] Furthermore, in step S2, the formula of the HVS degree evaluation function is as follows:

[0013] When L≤2 Depth-1 When d1=2 1-Depth ×L

[0014] When L > 2 Depth-1 When d1=2-2 1-Depth ×L

[0015] Among them, Depth is the image data bit depth, and L is the brightness value of the center point.

[0016] Furthermore, in step S3, the formula of the local texture evaluation function is as follows:

[0017] When D max When ≤E3, d2=1;

[0018] When E3 <D max ≤Th end hour,

[0019] When D max >Th end When d2=0;

[0020] Among them, Thend = E1 - E2×d1 + m1, where m1 is the image noise level parameter.

[0021] Further, in step S4, the formula of the adaptive threshold function is as follows:

[0022] When d2 = 0,

[0023] When 0 < d2 < 1,

[0024] When d2 = 1, Th low = Th high = D max .

[0025] Further, in step S5, the formula of the adaptive weight function is as follows:

[0026] When 0 ≤ D i < Th low then, w i = 1

[0027] When Th low ≤ D i < Th high then,

[0028] When D i > Th high then, w i = 0

[0029] where D i is an element in the set D, and w i is the weight of the corresponding point of P i i = 0, 1, 2...n.

[0030] Further, in step S6, the calculation formula of the image estimated value after denoising the center point value of the filtering window is as follows:

[0031]

[0032] where P i is the valid neighborhood point value around the center point P c , and Output is the image estimated value.

[0033] Further, one filtering window corresponds to one filtering period. According to the local texture evaluation degree in the current filtering period and the local texture evaluation degree in the previous filtering period, a local noise evaluation function is designed. The maximum point value of the filtering window is input into the local noise evaluation function for processing to obtain the local noise evaluation degree, denoted as d3;

[0034] The local noise evaluation degree in the previous filtering cycle is used as the image noise level parameter in the current filtering cycle to calculate the local texture evaluation degree in the current filtering cycle.

[0035] Furthermore, the formula of the local noise evaluation function is as follows:

[0036] d3=d2×D max +(1-d2)×m2

[0037] Among them, m2 is the local texture evaluation degree calculated in the previous filtering cycle.

[0038] Furthermore, when the center point value P c When it is 0 or the maximum value, the HVS degree d1 is 0, the function form of the HVS degree evaluation function is a Gaussian curve; the function form of the local texture evaluation function is a trapezoidal curve.

[0039] The beneficial effects brought about by the technical solution provided by the present invention are as follows:

[0040] a. Reduces the amount of image noise reduction processing, enables rapid denoising, and reduces related hardware requirements;

[0041] b. Able to adapt to the raw image data of various sensors, with good compatibility and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 1 is a flow chart of a method for image spatial domain-based noise reduction provided by an embodiment of the present invention;

[0044] Figure 2 2 is a schematic diagram of an HVS degree evaluation function in an image spatial domain-based denoising method provided by an embodiment of the present invention;

[0045] Figure 3 Schematic diagram of a local texture evaluation function in a denoising method based on image spatial domain provided by an embodiment of the present invention;

[0046] Figure 4 3 is a schematic diagram of a filtering window of a denoising method based on image spatial domain provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0048] It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or devices.

[0049] In one embodiment of the present invention, a noise reduction method based on image spatial domain is provided. Figure 1 , including the following steps:

[0050] S1. Set a filter window for the image data to be processed. One filter window corresponds to one filter cycle. The center point value of the filter window is recorded as P. c , the valid neighborhood point values ​​around the center point value are recorded as P0, P1, P2, ..., P n , and accordingly form the set {|P c -P i |} i=0,1,2...n , denoted as D, the maximum and minimum values ​​in the set are denoted as D max and D min ;

[0051] S2, according to the center point value P of the filtering window c , use the HVS degree evaluation function to obtain the HVS degree, denoted as d1;

[0052] Among them, the HVS degree evaluation function is obtained by P c Calculate the HVS degree d1, the value range of d1 is 0~1. Since the human eye perceives the texture and noise of the middle grayscale more obviously than the too dark and too bright areas, when P c When the value of is close to the middle, the HVS degree evaluation function reaches the maximum value.c When the value is 0 or the maximum value, the HVS degree is 0. A typical function form of the HVS degree evaluation function is as follows Figure 2 The Gaussian curve shown.

[0053] S3, designing a local texture evaluation function according to the HVS degree d1 and a preset noise reduction threshold parameter, inputting the maximum point value of the filter window into the local texture evaluation function for processing to obtain a local texture evaluation degree, recorded as d2;

[0054] The noise reduction threshold parameters are adjusted accordingly based on different observation conditions, such as the distance between the observer and the display, the image display size, the observation angle, and other external observation environments. The noise reduction threshold parameters include a first noise reduction threshold parameter, a second noise reduction threshold parameter, and a third noise reduction threshold parameter, denoted as E1, E2, and E3, respectively. The first noise reduction threshold parameter is positively correlated with the maximum brightness of the display environment, and the second noise reduction threshold parameter is positively correlated with the maximum brightness output by the display device. E1 and E2 represent the degree of participation of the HVS degree in the algorithm, and both together restrict the participation of the image highlight portion in the calculation of d2. When the ambient light brightness significantly exceeds the maximum brightness of the display, more highlighted pixels in the image participate in the calculation of d2. The third noise reduction threshold parameter is positively correlated with the visual angle of the image observer to the unit pixel of the display device, which corresponds to the visual angle of the unit pixel of the display to the human eye and can be calculated by dividing the display pixel size by the observation distance. The noise reduction threshold parameters can be preset manually, or the AI ​​camera can be used to perform real-time identification and detection of the observer and the display device, obtain the current observation conditions through the algorithm, and then adjust the noise threshold parameters accordingly to achieve automatic adjustment of the parameters.

[0055] The value range of local texture evaluation degree d2 is also 0~1, D max The smaller it is, the larger the value of d2 is. A typical function shape of the local texture evaluation function is as follows Figure 3 The trapezoidal function shown, specifically, the formula of the local texture evaluation function is as follows:

[0056] When D max When ≤E3, d2=1;

[0057] When E3 <D max ≤Th end hour,

[0058] When D max >Th end When d2=0;

[0059] Where Th end =E1-E2×d1+m1, where m1 is the image noise level parameter.

[0060] It should be noted that there are two ways to obtain the value of m1, which respectively correspond to two modes of the noise reduction method, namely the FIR (Finite Impulse Response) mode and the IIR (Infinite Impulse Response) mode. In the FIR mode, the filtering of the previous window does not affect the filtering of the next window. Therefore, m1 is input as a fixed parameter to further reduce the amount of computation.

[0061] In the IIR mode, the filtering of the previous window affects the filtering of the next window. Therefore, a forward and backward feedback mechanism is established through m1. In this mode, the noise and local texture evaluation of the algorithm are more accurate, and the noise reduction effect is better. Specifically, according to the degree of local texture evaluation in the current filtering period and the degree of local texture evaluation in the previous filtering period, a local noise evaluation function is designed. The maximum point value of the filtering window is input into the local noise evaluation function for processing to obtain the degree of local noise evaluation, denoted as d3. The formula of the local noise evaluation function is as follows:

[0062] d3 = d2 × D max +(1 - d2) × m2

[0063] where m2 is the degree of local texture evaluation calculated in the previous filtering period.

[0064] The degree of local noise evaluation in the previous filtering period is used as the image noise level parameter m1 in the current filtering period to calculate the degree of local texture evaluation in the current filtering period. In the first filtering period, the image noise level parameter m1 uses a preset initial value.

[0065] S4. According to the degree of local texture evaluation d2, perform an adaptive threshold function process on the maximum and minimum values in the set to obtain an adaptive threshold;

[0066] where a dual adaptive threshold setting method is adopted to obtain the adaptive thresholds Th low , Th high , and the formula of the adaptive threshold function is as follows:

[0067] When d2 = 0,

[0068] When 0 < d2 < 1,

[0069] When d2 = 1, Th low = Th high = D max .

[0070] S5. Performing adaptive weight function processing on each element in the set according to the adaptive threshold to obtain the weight of the corresponding point of each element;

[0071] The formula of the adaptive weight function is as follows:

[0072] When 0≤D i <Th low When i =1

[0073] When Th low ≤D i <Th high hour,

[0074] When D i >Th high When i =0

[0075] Among them, D i is an element in the set D, w i P i The weight of the corresponding point, i=0,1,2...n.

[0076] S6. Calculate the estimated value of the image after noise reduction by using a weighted summation method.

[0077] The calculation formula of the image estimation value after noise reduction of the center point value of the filtering window is as follows:

[0078]

[0079] Among them, P i The center point P c The valid neighborhood point values ​​around it, Output is the image estimation value.

[0080] After step S6, data post-processing is also included, including demosaicing, gamma, sharpening and other processes.

[0081] See also Figure 1, this embodiment utilizes HVS degree evaluator, local texture degree evaluator, local noise degree evaluator, and dual adaptive threshold weight calculator to process the corresponding function algorithm. Among them, the local noise degree evaluator is activated and enabled only in IIR mode. In each filtering window, the above-mentioned evaluator is used to calculate d1 through the HVS degree evaluation function and d2 through the local texture evaluation function. The human eye observation angle can be calculated according to the external environment such as the distance between the observer and the display, the image display size, and the observation angle, and the appropriate noise reduction threshold parameter is adjusted. In spatial filtering, the current filtering noise coefficient is calculated by d1, d2, E1, E2, and E3 to obtain the local noise evaluation degree d3. Since the image has continuity within a certain range, d2 and d3 can be used as feedback data to the local texture degree evaluator and the local noise degree evaluator in the next filtering process. In this case, the filtering algorithm can be transformed from an FIR filter to an IIR filter. Finally, the dual adaptive threshold is calculated to obtain the weight calculation function. The weight of each element in the filtering window is calculated by the difference between each element in the filtering window and the center. The denoising estimate of the target position is obtained by weighted summation.

[0082] Compared to other denoising methods, such as cosine transform, wavelet transform, bilateral filtering, and NLM, this embodiment comprehensively considers the characteristics of the HVS, local texture features, and local noise levels. Furthermore, the algorithm can be switched between IIR and FIR modes depending on the actual application, making it more flexible. Compared to transform-domain denoising methods, the denoising method of this embodiment operates directly in the spatial domain, reducing computational complexity and facilitating hardware implementation. This can also meet stringent latency and bandwidth requirements for specific industries, such as surgical imaging display systems in the medical industry. When the noise perception conditions of the HVS are known, parameters such as the distance between the display device and the image observer and the viewing angle can be obtained. Inputting these necessary parameters into the denoising unit can achieve image quality with improved visual noise perception compared to other denoising methods. Furthermore, the dual adaptive threshold weight calculator can preserve better edge and texture details in the image, similar to the edge-preserving properties of bilateral filtering.

[0083] The following example uses a 5×5 filter window as an example:

[0084] (1) The input image data is the original raw data, the filter window size is 5×5, and the center point value is P c , see Figure 4 , the set D is {|P c -P i |} i=0,1,2...n , see Figure 4, when the center pixel is R or B channel, n=7, when the center pixel is G channel, n=11; D max is the maximum value of the set D, and the working mode is selected as IIR mode.

[0085] (2) Calculate HVS degree d1

[0086] The formula of the HVS degree evaluation function is as follows:

[0087] When L≤2 Depth-1 When d1=2 1-Depth ×L

[0088] When L > 2 Depth-1 When d1=2-2 1-Depth ×L

[0089] Among them, Depth is the image data bit depth, and L is the brightness value of the center point. Since the human eye is relatively sensitive to the G channel, when P c In G channel, L=P c Otherwise, L is obtained by using the demosaic algorithm, for example, L is obtained by interpolation based on the Gr and Gb channels around the center point.

[0090] (3) Calculate the local texture evaluation degree d2

[0091] When D max When ≤E3, d2=1;

[0092] When E3 <D max ≤Th end hour,

[0093] When D max >Th end When d2=0;

[0094] Where Th end =E1-E2×d1+m1, where m1 is the image noise level parameter. In IIR mode, m1 is determined by the parameter passed by the local noise level estimator in the previous filtering cycle. E1 and E2 are the degree of participation of the HVS level d1 in the algorithm, and E3 is positively correlated with the image observer's visual angle per unit pixel.

[0095] (4) Calculate and update the local noise evaluation level d3:

[0096] The formula of the local noise evaluation function is as follows:

[0097] d3=d2×D max +(1-d2)×m2

[0098] Among them, m2 is the local texture evaluation degree calculated in the previous filtering cycle.

[0099] (5) Calculate the adaptive threshold through d2

[0100] (6) Obtain each pixel P in the filter window through the dual adaptive threshold weight calculator i The corresponding weight wi. The estimated value of the denoised image is obtained according to the following formula:

[0101]

[0102] Among them, Output is the image estimation value.

[0103] The image spatial domain-based denoising method provided by the present invention performs spatial filtering denoising based on HVS and texture feature analysis. Because processing is performed in the spatial domain, the computational effort is reduced. The method is also applicable to both raw sensor data and grayscale data, offering good compatibility and flexibility. The image spatial domain-based denoising method provided by the present invention is applicable to imaging devices, imaging modules, ISPs (Image Signal Processors), DSPs (Digital Signal Processors), and display devices.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A denoising method based on image spatial domain, characterized in that: The following steps are involved: S1. Set a filter window for the image data to be processed, and record the center point value of the filter window as P c , the valid neighborhood point values ​​around the central point value are recorded as P0, P1, P2, ..., P n , and accordingly form the set {|P c -P i |} i=0,1,2...n , denoted as D, D i For each element in the set, the maximum and minimum values ​​in the set are denoted as D max and D min ; S2, according to the center point value P of the filtering window c , use the HVS degree evaluation function to obtain the HVS degree, denoted as d1; S3, designing a local texture evaluation function according to the HVS degree d1 and a preset noise reduction threshold parameter, inputting the maximum point value of the filter window into the local texture evaluation function for processing to obtain a local texture evaluation degree, recorded as d2; S4, according to the local texture evaluation degree d2, performing adaptive threshold function processing on the maximum value and the minimum value in the set to obtain an adaptive threshold; S5. According to the adaptive threshold, perform adaptive weight function processing on each element in the set to obtain the weight of the corresponding point of each element; S6. Calculate the estimated value of the image after noise reduction by using a weighted summation method.

2. The image spatial domain based denoising method according to claim 1, characterized in that: According to different observation conditions, the noise reduction threshold parameters are adjusted accordingly; the noise reduction threshold parameters include a first noise reduction threshold parameter, a second noise reduction threshold parameter and a third noise reduction threshold parameter, which are respectively denoted as E1, E2, and E3. The first noise reduction threshold parameter is positively correlated with the maximum brightness of the environment in which the display device is located, the second noise reduction threshold parameter is positively correlated with the maximum brightness output by the display device, and the third noise reduction threshold parameter is positively correlated with the visual angle of the image observer to the unit pixel of the display device.

3. The image spatial domain based denoising method according to claim 1, characterized in that: In step S2, the formula of the HVS degree evaluation function is as follows: When L≤2 Depth-1 When d1=2 1-Depth ×L When L > 2 Depth-1 When d1=2-2 1-Depth ×L Among them, Depth is the image data bit depth, and L is the brightness value of the center point.

4. The method for denoising based on image spatial domain according to claim 2, characterized in that: In step S3, the formula of the local texture evaluation function is as follows: When D max When ≤E3, d2=1; When E3 <D max ≤Th end hour, When D max >Th end When d2=0; Among them, Th end =E1-E2×d1+m1, where m1 is the image noise level parameter.

5. The image spatial domain based denoising method according to claim 1, characterized in that: In step S4, the formula of the adaptive threshold function is as follows: When d2=0, When 0 < d2 < 1, When d2=1, Th low =Th high =D max .

6. The method for denoising based on image spatial domain according to claim 5, characterized in that: In step S5, the formula of the adaptive weight function is as follows: When 0≤D i <Th low When i =1 When low ≤D i <Th high hour, When D i >Th high When i =0 Among them, D i is an element in the set D, w i P i The weight of the corresponding point, i=0,1,2...n.

7. The image spatial domain based denoising method according to claim 6, characterized in that: In step S6, the calculation formula of the image estimation value after denoising the center point value of the filtering window is as follows: Among them, Output is the image estimation value.

8. The image spatial domain based denoising method according to claim 1, characterized in that: One filter window corresponds to one filter cycle. A local noise evaluation function is designed according to the local texture evaluation degree in the current filter cycle and the local texture evaluation degree in the previous filter cycle. The maximum point value of the filter window is input into the local noise evaluation function for processing to obtain the local noise evaluation degree, which is recorded as d3. The local noise evaluation degree in the previous filtering cycle is used as the image noise level parameter in the current filtering cycle to calculate the local texture evaluation degree in the current filtering cycle.

9. The image spatial domain based denoising method according to claim 8, characterized in that: The formula of the local noise evaluation function is as follows: <h2 style=";text-align:left;direction:ltr">d3 = d2 × D<h2 style=";text-align:left;direction:ltr"> max <h2 style=";text-align:left;direction:ltr"> +(1-d2)×m2 Among them, m2 is the local texture evaluation degree calculated in the previous filtering cycle.

10. The image spatial domain based denoising method according to claim 1, characterized in that: When the center point value P c When it is 0 or the maximum value, the HVS degree d1 is 0, the function form of the HVS degree evaluation function is a Gaussian curve; the function form of the local texture evaluation function is a trapezoidal curve.