A method for infrared image denoising based on SGM-delta

Abstract

The present invention discloses an infrared image denoising method based on sgm-delta, which comprises the following steps: S1: determining a point set that can participate in filtering in a neighborhood; S2: performing sgm-delta quantization encoding on the point set; S3: calculating a denoising result through the encoding value. The present invention performs image denoising based on the sgm-delta noise shaping principle, which broadens the bandwidth and divides the noise energy equally in the entire bandwidth range, thereby reducing the background noise and improving the signal-to-noise ratio.

Description

Technical Field

[0001] The invention relates to the technical field of infrared image processing, and in particular to an infrared image denoising method based on SGM-Delta. Background Art

[0002] Infrared images often retain a lot of noise due to the non-uniformity of the detector, the readout circuit, and the dark current noise. Common denoising methods include spatial domain methods based on kernel operations (bilateral filtering, guided filtering, NLM, etc.), methods based on multi-block matching (DNR, BM3D, block SVD, etc.), and transform domain methods (wavelet basis, frequency domain denoising, etc.); among them, the spatial domain method based on kernel operations acts on the center point of the current block through kernel operations, and the denoising effect will leave traces of spatial domain calculations (excessive noise points will be smoothed into "protrusions"). This deficiency is compensated in the method based on multi-block matching, which increases the number of blocks to provide more reference information for denoising, and acts on each point of the current block in the form of block superposition, but the multi-block method increases the algorithm calculation amount and hardware implementation complexity; the transform domain algorithm also has high computational complexity in wavelet basis and frequency domain, and memory resource overhead. Summary of the invention

[0003] Based on the advantages of the multi-block matching denoising idea, the present invention provides more reference information for denoising and considers reducing the complexity of the algorithm. A new infrared image denoising method based on the SGM-delta noise shaping principle is proposed.

[0004] The object of the present invention is to adopt the following technical solution to achieve: an infrared image denoising method based on SGM-Delta, comprising the following steps:

[0005] S1: Determine the point set that can participate in filtering in the neighborhood;

[0006] S2: Perform sgm-delta quantization encoding on the point set;

[0007] S3: Calculate the denoising result through the encoded value.

[0008] Furthermore, step S1 is specifically as follows: determine whether the coordinates of another point (i', j') in the neighborhood of the coordinate point (i, j) meet the preset conditions. If so, the value of the point (i', j') is included in the point set {Y ij};

[0009] The preset conditions are:

[0010] ;

[0011] In the formula, Represents the image value at coordinate (i, j) on the infrared image Ysrc; Represents the image value at coordinate (i', j') on the infrared image Ysrc; Represents the threshold, which is related to the noise level.

[0012] Furthermore, step S2 includes the following sub-steps:

[0013] S21: Calculate filtering reference values ​​Vref+ and Vref-;

[0014] S22: Traverse the point set {Y ij}, calculate the encoding value Bcode.

[0015] Furthermore, step S21 includes the following sub-steps:

[0016] S211: Calculate the point set {Y ij The mean ymean of} is:

[0017] ; where n is the point set {Y ij The number of elements in};

[0018] S212: Calculate the weighted mean value Vref:

[0019] ;Wherein, w represents the weight, which is inversely proportional to the neighborhood variance var;

[0020] S213: Calculate the filtering reference value:

[0021] ;

[0022] ; In the formula, vd represents level of fluctuation.

[0023] Further, step S22 includes the following sub-steps:

[0024] S221: Initialize variable values;

[0025] S222: Calculate the output value of the comparator;

[0026] S223: Calculate the output value of the integrator;

[0027] S224: Determine the sign of the integrator output value and accumulate the code stream;

[0028] S225: Update the reference value of the comparator;

[0029] S226: Return to step S222 until all elements in the point set are traversed.

[0030] Furthermore, the initialization variable value includes one or more of: setting the counter k to zero, setting the negative reference value vdac_k of the comparator to zero, setting the integrator accumulated value sum_k to zero, and setting the code stream accumulated value Bcode to zero.

[0031] Furthermore, step S3 is specifically as follows: calculating the denoising result from the code value Bcode:

[0032] ;

[0033] Where yft is the denoising result.

[0034] Furthermore, the neighborhood includes a spatial domain range, or a temporal domain range, or a spatiotemporal domain range.

[0035] The beneficial effect of the present invention is that: the present invention performs image denoising based on the SGM-delta noise shaping principle, and since it broadens the bandwidth, the noise energy is equally divided in the entire bandwidth range, so that the background noise is reduced, thereby improving the signal-to-noise ratio. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0037] Figure 1 It is a flow chart of the present invention;

[0038] Figure 2 Flowchart for quantizing and encoding point sets;

[0039] Figure 3 Flow chart for calculating filtering reference value;

[0040] Figure 4 Flowchart for calculating coding value;

[0041] Figure 5 This is the block diagram of the Σ-Δ modulator. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0043] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0044] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0045] See also Figure 1 , an infrared image denoising method based on sgm-delta, comprising the following steps:

[0046] S1: Determine the point set that can participate in filtering in the neighborhood;

[0047] S2: Perform sgm-delta quantization encoding on the point set;

[0048] S3: Calculate the denoising result through the encoded value.

[0049] The principle of sgm-delta noise shaping is to use high-precision sampling to move the quantization noise to the high-frequency area, and then filter out the noise through low-pass filtering. Because it widens the bandwidth, the noise energy is evenly divided into the entire bandwidth range, so that the background noise is reduced, thereby improving the signal-to-noise ratio. The premise of sgm-delta noise shaping is based on high-precision sampling, which is equivalent to providing more sampling information in the time domain. However, it is not the best solution to directly increase the sampling rate in the infrared image imaging system. The present invention draws on the denoising idea of ​​multi-block matching and uses multi-block information in the spatial domain to replace high-precision sampling in the time domain, thereby realizing an infrared image denoising method based on sgm-delta.

[0050] Specifically, sgm-delta noise shaping technology refers to the oversampling, noise shaping, low-pass filtering and other technologies of Σ-Δ analog-to-digital converters in the digital electronics and analog electronics fields. The present invention draws on the principle of Σ-Δ analog-to-digital converters and applies it to the denoising of infrared images. The present invention also includes oversampling, noise shaping, and low-pass filtering. The difference is:

[0051] In the oversampling link, the Σ-Δ analog-to-digital converter achieves oversampling in the frequency domain by increasing the sampling rate, while the invention achieves "oversampling" by using multiple points in the neighborhood in the spatial domain. First, consider the frequency domain transfer function of a traditional multi-bit ADC when the input signal is a sine wave. The input is sampled at a frequency Fs. According to the Nyquist theorem, Fs must be at least twice the bandwidth of the input signal. From the FFT analysis results of the digital output, when the signal contains a single tone and a large amount of random noise, it extends from DC to Fs / 2. This noise is called quantization noise, and the result can be considered as follows: the ADC input is a continuous signal with infinite possible states, but the digital output is a discrete function, and the number of its different states depends on the resolution of the converter. Therefore, the conversion from analog to digital loses some information and introduces a certain degree of distortion in the signal. The amplitude of this error is random and the maximum is ±LSB. In order to improve the signal-to-noise ratio and reduce the noise floor, the traditional method is to increase the bit width of the ADC, which increases the overall bit width and power consumption of the system; oversampling technology is different. Increasing the sampling frequency (using the oversampling factor k to reach kFs) can reduce the noise floor while increasing the bit width by only 1 bit, allowing the Σ-Δ converter to achieve a wider dynamic range with a lower resolution ADC.

[0052] In the noise shaping process, the comparator threshold of the Σ-Δ analog-to-digital converter is a fixed value, which is related to the signal amplitude range; the invention adaptively calculates the two thresholds of the comparator through the local information of the image. Take the block diagram of the first-order Σ-Δ modulator as an example (such as Figure 5 ), which includes a differential amplifier, an integrator, and a comparator, as well as a feedback loop that includes a 1-bit DAC. (The DAC is a simple switch that connects the negative input of the differential amplifier to a positive or negative reference voltage.) The purpose of the feedback DAC is to maintain the average output of the integrator close to the reference level of the comparator. The density of "1"s at the output of the modulator is proportional to the input signal. When the input increases, the comparator produces a large number of "1"s; the opposite is true when the input decreases. By summing the error voltage, the integrator acts as a low-pass filter for the input signal and a high-pass filter for the quantization noise. Therefore, most of the quantization noise is moved to higher frequencies. Oversampling not only changes the total noise power, but also changes its distribution.

[0053] In the low-pass filtering stage, the sigma-delta analog-to-digital converter uses a decimation filter to achieve low-pass filtering, while the invention uses (but is not limited to) an averaging filter to achieve low-pass filtering. The output of the sigma-delta modulator is a 1-bit data stream with a sampling rate that can reach the megahertz range. The purpose of the digital and sampling filters is to extract information from this data stream and reduce the data rate to a more useful value. In the sigma-delta ADC, the digital filter averages the 1-bit data stream, improves the ADC resolution, and filters out the out-of-band quantization noise.

[0054] Points are taken within the neighborhood of the current point, and the size of the neighborhood space is a block of large radius (the neighborhood radius of the general denoising algorithm is 3*3 or 5*5, which is a relatively small radius. The neighborhood radius of the present invention is 16*16, 32*32, etc.). The purpose of this is to use spatial information to achieve oversampling, which corresponds to step S1 of the present invention; the noise shaping part corresponds to step S2 of the invention; the low-pass filtering part corresponds to step S3 of the invention.

[0055] It should be noted that, in the present invention, on an infrared image Ysrc with a height of H and a width of W, the infrared image Ysrc is 14-bit original image data after non-uniform correction, and the following denoising process is performed on each point thereon. The image value at the coordinate (i, j) on the infrared image Ysrc is recorded as Ysrc(i, j), and the neighborhood of the point is a square area of ​​D×D, the coordinates of the upper left vertex of the neighborhood are (ir, jr), and the coordinates of the lower right vertex of the neighborhood are (i+r, j+r), D = r+r+1, where r is the neighborhood radius, which can be 32; D is the neighborhood diameter, which can be 65.

[0056] Furthermore, step S1 is specifically as follows: the coordinates of a point in the neighborhood of the coordinate point (i, j) are set to (i', j'). If the following conditions are met, the value of the point (i', j') is included in the point set {Y ij}

[0057] The conditions are:

[0058] ;

[0059] In the formula, Represents the image value at coordinate (i, j) on the infrared image Ysrc; Represents the image value at coordinate (i', j') on the infrared image Ysrc; It represents the threshold value, which is related to the noise level and can be set by the user. The value of 30 is acceptable.

[0060] See also Figure 2 , step S2 includes the following sub-steps:

[0061] S21: Calculate filtering reference values ​​Vref+ and Vref-;

[0062] S22: Traverse the point set {Y ij}, calculate the encoding value Bcode.

[0063] See also Figure 3 , step S21 includes the following sub-steps:

[0064] S211: Calculate the point set {Y ij The mean ymean of} is:

[0065] ; where n is the point set {Y ij The number of elements in};

[0066] S212: Calculate the weighted mean value Vref:

[0067] ; In the formula, w represents the weight, which is inversely proportional to the neighborhood variance var, so that Vref tends to the image value at point (i, j) where the local variance is large , where the local variance is small, Vref tends to the neighborhood mean ;

[0068] The relationship between weight and neighborhood variance can be, but is not limited to:

[0069] ; In the formula, eps is a constant set by the user and can be 100;

[0070] The calculation formula of local variance is:

[0071] ;

[0072] S213: Calculate the filtering reference value:

[0073] ;

[0074] ; In the formula, vd represents The fluctuation level can be set by the user and can be set to 20.

[0075] See also Figure 4 , step S22 includes the following sub-steps:

[0076] S221: Initialize variable values;

[0077] S222: Calculate the output value of the comparator;

[0078] S223: Calculate the output value of the integrator;

[0079] S224: Determine the sign of the integrator output value and accumulate the code stream;

[0080] S225: Update the reference value of the comparator;

[0081] S226: Return to step S222 until all elements in the point set are traversed.

[0082] The initialized variable values ​​include one or more of: setting the counter k to zero, setting the negative reference value vdac_k of the comparator to zero, setting the integrator accumulated value sum_k to zero, and setting the code stream accumulated value Bcode to zero.

[0083] Calculate the output value ydiff_k of the comparator: ;

[0084] Calculate the output value sum_k of the integrator: ;

[0085] Judge the sign of the output value of the integrator and accumulate the bitstream:

[0086] ;

[0087] Update the reference value of the comparator:

[0088] ;

[0089] Increment the counter k by 1 and return to step S222; when k = n, exit.

[0090] Step S3 specifically is: Calculate the denoising result from the coding value Bcode:

[0091] ;

[0092] Where yft is the denoising result.

[0093] Based on the above three steps (steps S1 - S3), complete the denoising filtering at the coordinate (i, j), traverse each coordinate position on the infrared image Ysrc, and complete the filtering at each point.

[0094] It should be noted that:

[0095] For the pixels at the boundary of the infrared image Ysrc, for example, when i < r, the ordinate of the upper - left vertex of the neighborhood is 1, and the ordinate of the lower - right vertex of the neighborhood is D; similarly, when i > H - r, the ordinate of the upper - left vertex of the neighborhood is H - D + 1, and the ordinate of the lower - right vertex of the neighborhood is H.

[0096] For the pixels at the boundary of the infrared image Ysrc, for example, when j < r, the abscissa of the upper - left vertex of the neighborhood is 1, and the abscissa of the lower - right vertex of the neighborhood is D; similarly, when j > W - r, the abscissa of the upper - left vertex of the neighborhood is W - D + 1, and the abscissa of the lower - right vertex of the neighborhood is W.

[0097] The neighborhood of the points mentioned in the present invention is the spatial domain range, and it can also be extended to the time domain range, the spatio - temporal domain range.

[0098] The sgm - delta quantization coding mentioned in the present invention is of the first order, and it can also be extended to higher orders.

[0099] It should be noted that, for the above embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification belong to preferred embodiments, and the actions involved are not necessarily required by the present application.

[0100] The above embodiments describe the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the changes and modifications made by those skilled in the art shall be within the scope of protection of the appended claims of the present invention without departing from the spirit and scope of the present invention.

Claims

1. An infrared image denoising method based on sgm-delta, characterized in that: The steps include: S1: Determine the point set that can participate in filtering in the neighborhood; S2: Perform sgm-delta quantization encoding on the point set; step S2 includes the following sub-steps: S21: Calculate filtering reference values ​​Vref+ and Vref-; Step S21 includes the following sub-steps: S211: Calculate the point set {Y ij The mean ymean of} is: ; where n is the point set {Y ij The number of elements in}; S212: Calculate the weighted mean value Vref: ; In the formula, w represents the weight, which is inversely proportional to the neighborhood variance var, so that Vref tends to the image value at point (i, j) where the local variance is large , where the local variance is small, Vref tends to the neighborhood mean ; The relationship between weight and neighborhood variance is: ; In the formula, eps is a constant set by the user; The calculation formula of local variance is: ； S213: Calculate the filtering reference value: ； ; In the formula, vd represents The level of volatility; S22: Traverse the point set {Y ij }, calculate the code value Bcode; step S22 includes the following sub-steps: S221: Initialize variable values; S222: Calculate the output value of the comparator; S223: Calculate the output value of the integrator; S224: Determine the sign of the integrator output value and accumulate the code stream; S225: Update the reference value of the comparator; S226: Return to step S222 until all elements in the point set are traversed; wherein the initialized variable values ​​include: setting the counter k to zero, setting the negative reference value vdac_k of the comparator to zero, setting the integrator accumulated value sum_k to zero, and setting the code stream accumulated value Bcode to zero. Calculate the comparator output value ydiff_k: ; Calculate the output value sum_k of the integrator: ; Determine the sign of the integrator's output value and accumulate the code stream: ； Update the reference value of the comparator: ； The counter k is incremented by 1, and the process returns to step S222; when k=n, the process exits; S3: Calculate the denoising result by the code value; Step S3 is specifically: calculate the denoising result by the code value Bcode: ； Where yft is the denoising result.

2. The infrared image denoising method based on SGM-Delta according to claim 1, characterized in that: Step S1 is specifically: determine whether the coordinates of another point (i', j') in the neighborhood of the coordinate point (i, j) meet the preset conditions. If so, the value of the point (i', j') is included in the point set {Y ij }; The preset conditions are: ； In the formula, Represents the image value at coordinate (i, j) on the infrared image Ysrc; Represents the image value at coordinate (i', j') on the infrared image Ysrc; Represents the threshold, which is related to the noise level.

3. The infrared image denoising method based on SGM-Delta according to any one of claims 1 to 2, characterized in that: The neighborhood includes a spatial domain range, a temporal domain range, or a spatiotemporal domain range.

Images