A method for estimating motion blur parameters in uncooled infrared images

Abstract

The application provides a non-refrigeration infrared image motion blur parameter estimation method, which is applied to non-refrigeration infrared motion blur image restoration. First, an obvious middle stripe part of a motion blur image spectrum diagram is cut out, then the spectrum diagram is denoised through a Gaussian low-pass filter, then an edge is extracted through a structure forest edge detection algorithm, a straight line is detected through Radon transformation on the edge diagram to estimate a blur angle, finally, the blur scale of the cut spectrum diagram is estimated through a projection method, and motion blur parameter estimation is realized. The application uses the structure forest edge extraction algorithm, overcomes the cross bright line and a large number of noise point interference existing in the non-refrigeration infrared image spectrum diagram, effectively extracts the motion blur image spectrum stripe edge, realizes high-accuracy and high-robustness estimation of the motion blur parameter, and can be applied to non-refrigeration infrared motion blur image restoration.

Description

TECHNICAL FIELD

[0001] The present application relates to non-cooled infrared motion blur image restoration technology, in particular to a non-cooled infrared image motion blur parameter estimation method. BACKGROUND

[0002] Non-cooled infrared detection system has the characteristics of low cost, small size and low power consumption, and has been widely used in civilian vehicle night vision, security and military fields. However, due to the long thermal response time (exposure) of non-cooled detectors, motion blur is easily generated during imaging, which affects the imaging quality and subsequent target detection and tracking related processing. Therefore, how to restore the clear image from the motion blurred image is of great significance for the application of non-cooled detection system.

[0003] For motion blur, image restoration can be performed by estimating prior information, i.e. point spread function (PSF). Since all forms of motion can be transformed into cumulative uniform linear motion under certain conditions, the study of uniform linear motion blur is more general and universal. The PSF parameters of uniform linear motion blur include blur angle (direction) and blur scale (length). The estimation methods of blur angle mainly include direction differential based method, image spectrum based method and image cepstrum based method, etc. The estimation methods of blur scale mainly include projection method and autocorrelation method, etc.

[0004] Most of the existing researches focus on visible light images, and few on infrared images, especially non-cooled infrared images. Non-cooled infrared images have less details and more noise, and their spectrum graph has obvious cross bright lines and a large number of noise points. The existing methods have low accuracy and poor robustness in estimating the PSF of non-cooled infrared motion blurred images, and the algorithm steps are complicated and have strong parameter dependence, which makes it difficult to realize image restoration. SUMMARY

[0005] The technical problem to be solved by the present application is to overcome the shortcomings of the prior art and provide a non-cooled infrared image motion blur parameter estimation method, which effectively extracts the spectrum stripe edge of the motion blurred image and realizes high-accuracy and high-robustness estimation of the motion blur parameters, and can be used for non-cooled infrared motion blurred image restoration.

[0006] The technical solution of the present application is:

[0007] The present application discloses a non-cooled infrared image motion blur parameter estimation method, comprising:

[0008] obtaining the spectrum graph of the motion blurred image;

[0009] cropping the middle part of the spectrum graph to obtain the cropped spectrum graph;

[0010] Gaussian low-pass filtering is performed on the cropped spectrum diagram to suppress cross bright lines and noise interference, and a spectrum stripe diagram is obtained;

[0011] A structure forest algorithm is used to extract a spectrum stripe edge from the spectrum stripe diagram;

[0012] The spectrum stripe edge diagram is binarized to obtain a main lobe stripe edge;

[0013] Radon transformation is performed on the main lobe stripe edge to estimate a blur angle;

[0014] According to the blur angle estimation result, a blur scale is estimated by a spectrum projection method, and motion blur parameter estimation is realized.

[0015] Further, in the above estimation method, the spectrum diagram of the motion blurred image is obtained in the following manner:

[0016] S1, two-dimensional discrete Fourier transform is performed on the blurred image f(x, y), and the formula is:

[0017]

[0018] wherein F(u, v) represents a spectrum diagram of the blurred image, N represents the side length of the image, x and y represent the rows and columns of the blurred image, and u and v represent the rows and columns of the Fourier transform result;

[0019] S2, the dynamic range is compressed by taking the logarithm, and the formula is:

[0020] F(u, v) = log(|F(u, v)| + 1)

[0021] wherein |·| represents the amplitude, log(·) represents the logarithm with base 10, and u and v represent the rows and columns of the Fourier transform result;

[0022] S3, the first and third quadrants and the second and fourth quadrants of F(u, v) are exchanged respectively to perform spectrum centering, and a motion blurred image spectrum diagram g(x, y) is obtained.

[0023] Further, in the above estimation method, the middle part of the spectrum diagram is cropped to obtain a cropped spectrum diagram, and the specific method is as follows:

[0024]

[0025] wherein g(x, y) is a spectrum diagram function, N is the side length of the spectrum diagram, and x and y represent the rows and columns of the spectrum diagram, respectively.

[0026] Further, in the above estimation method, the cropped spectrum is subjected to Gaussian low-pass filtering to suppress cross bright line and noise interference, and a spectrum fringe pattern is obtained, specifically as follows:

[0027] G(u, v) = G'(u, v) * H lowpass (u, v)

[0028]

[0029] wherein G(u, v) is the spectrum fringe pattern, D 2 (u, v) is the distance of point (u, v) from the center point, D0 is the low-pass filter cutoff frequency, G'(u, v) is the two-dimensional Fourier transform of the spectrum, and u and v are the row and column of the Fourier transform result.

[0030] Further, in the above estimation method, the spectrum fringe pattern is subjected to a structure forest algorithm to extract the spectrum fringe edge, which uses structured learning for edge detection, uses image blocks instead of pixels, trains a random forest classifier on an image segmentation standard dataset, and performs edge detection on the image to obtain an edge map g edge (x, y).

[0031] Further, in the above estimation method, the spectrum fringe edge map is binarized to obtain the main lobe fringe edge, specifically as follows:

[0032]

[0033] wherein g binary (x, y) is the spectrum Figure Two binarized result, max(g edge (x, y)) is the maximum gray value of the edge map, and x and y are the row and column of the image.

[0034] Further, in the above estimation method, the main lobe fringe edge is subjected to Radon transform to estimate the blur angle, specifically as follows:

[0035]

[0036] θ estimate = {θ | max(R)}

[0037] wherein R(ρ, θ) is the Radon transform result, g binary (x, y) is the spectrum Figure Two binarized result, θ estimate is the blur angle, ρ is the distance of a straight line to the origin, θ is the angle between the vertical line of the straight line passing through the origin and the X-axis, x and y represent the row and column of the image, and δ(·) is the Dirac function.

[0038] Further, in the above estimation method, the blur scale is estimated by a spectrum projection method according to the blur angle estimation result, and the motion blur parameter estimation is realized, specifically as follows.

[0039] According to the blur angle estimation result, the cropped spectrum graph is rotated clockwise by the same angle to obtain a rotated spectrum graph;

[0040] The gray value of the rotated spectrum graph is calculated, and the sum is calculated by column to obtain a detection curve;

[0041] According to the minimum point of the detection curve, the average interval d between adjacent minimum points is calculated, and the blur scale L is obtained estimate ;

[0042] According to the blur scale L estimate , the Wiener filtering is performed on the rotated spectrum graph to realize image restoration.

[0043] Further, in the above estimation method, the blur scale L estimate is obtained, specifically as follows.

[0044]

[0045] Wherein, L estimate is the blur scale, d is the average interval, and N is the side length of the image.

[0046] Further, in the above estimation method, the Wiener filtering is performed on the rotated spectrum graph to realize image restoration, specifically as follows.

[0047]

[0048]

[0049] Wherein, K is a constant, L estimate is the blur scale, h(x, y) is the point spread function, H(u, v) is the Fourier transform of the point spread function, is the Fourier transform of the restored image, G(u, v) is the spectrum stripe graph, θ estimate is the blur angle, x and y are the row and column of the point spread function, and u and v are the row and column of the Fourier transform result.

[0050] The beneficial effects of the present application over the prior art are as follows:

[0051] (1) The present application crops the middle part of the spectrum graph for blur parameter estimation, retains the stripe information with obvious main lobe, avoids the influence of the annular stripe around, and improves the algorithm operation speed;

[0052] (2) The present application uses structure forest algorithm to extract fringe edge, can inhibit cross bright line and noise point interference, edge is more complete and accurate, robustness is stronger, algorithm parameter dependence is smaller;

[0053] (3) The present application uses Radon transform to estimate fuzzy angle, and spectrum projection method is used to estimate fuzzy scale, and the result is more accurate.

[0054] (4) The present application has wide application prospect in the field of non-cooled infrared motion blur image restoration. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 It is the flow chart of the motion blur parameter estimation method of the present application;

[0056] Figure 2 It is the schematic diagram of the non-cooled infrared motion blur image of the present application;(a) is a clear image, (b) is a blurred image after motion blur is applied;

[0057] Figure 3 It is the schematic diagram of the spectrum graph after cutting of the present application;(a) is a blurred image spectrum graph, (b) is a part of spectrum graph after cutting and low-pass filtering;

[0058] Figure 4 It is the schematic diagram of the edge extraction result of structure forest of the present application;(a) is the result graph after edge extraction of structure forest algorithm, (b) is the main lobe edge result graph after binarization;

[0059] Figure 5 It is the schematic diagram of Radon transform result of the present application;

[0060] Figure 6 It is the schematic diagram of the projection method for estimating fuzzy scale of the present application;(a) is a rotated spectrum graph, (b) is a vertical projection result graph;

[0061] Figure 7 It is the schematic diagram of the motion blur image restoration result of the present application;(a) is a motion blur image, (b) is the result graph after fuzzy parameter estimation and image restoration by using the present application. DETAILED DESCRIPTION

[0062] The present application patent will be further described in detail in combination with the drawings and specific embodiments.

[0063] As shown in Figure 1 The present application provides a non-cooled infrared image motion blur parameter estimation method, comprising the following steps:

[0064] A non-cooled infrared image motion blur parameter estimation method is to use a structure forest algorithm to extract edges of a blurred image spectrum diagram, and then realize estimation of motion blur parameters. First, a blurred image spectrum diagram is calculated, and a middle stripe obvious part is cropped. Then, the spectrum diagram is denoised through a Gaussian low-pass filter. Then, a structure forest edge detection algorithm is used to extract edges. The edge diagram is detected through a Radon transform to estimate a blur angle. Finally, the cropped spectrum diagram is estimated through a projection method to estimate a blur scale, and motion blur parameter estimation is realized. The specific implementation is as follows:

[0065] Step one: two-dimensional fast Fourier transform is performed on a blurred image f(x, y):

[0066]

[0067] Wherein, F(u, v) represents a blurred image spectrum diagram, N represents an image size of N*N, x and y represent rows and columns of the blurred image, and u and v represent rows and columns of a Fourier transform result.

[0068] The dynamic range is compressed through a logarithmic method:

[0069] F(u, v) = log(|F(u, v)| + 1) 2)

[0071] Wherein, |·| represents an amplitude value, log(-) represents a logarithm with 10 as a base, and u and v represent rows and columns of a Fourier transform result.

[0072] The first quadrant and the third quadrant and the second quadrant and the fourth quadrant are exchanged to obtain a spectrum diagram g(x, y);

[0073] Step two: it is assumed that a spectrum diagram height and width are N, a middle N / 4 to 3N / 4 and N / 4 to 3N / 4 part of the spectrum diagram is cropped for parameter estimation, interference of a four-around annular stripe edge is reduced, and operation speed is improved:

[0074]

[0075] Wherein, g(x, y) is a spectrum diagram function, N is a spectrum diagram height and width, and x and y represent rows and columns of the spectrum diagram.

[0076] Step three: a Gaussian low-pass filter is performed on the cropped spectrum diagram, a higher cutoff frequency is selected, cross bright lines, burrs and noise points are inhibited to a certain extent, and stripe edge information is reserved;

[0077]

[0078] G(u, v) = G'(u, v).*H lowpass (u, v) 5)

[0080] where G(u, v) is the spectrum fringe pattern, D 2 (u, v) is the distance of point (u, v) from the center point, D0 is the low-pass filter cut-off frequency, G'(u, v) is the two-dimensional Fourier transform of the spectrum pattern, u, v are the row and column of the Fourier transform result.

[0081] Step four: the fringe edges of the spectrum pattern are extracted by using the structure forest algorithm, and then binarized with the threshold of 0.9 times the maximum gray value to obtain the obvious main lobe fringe edges;

[0082]

[0083] where g binary (x, y) is the spectrum Figure Two value after binarization, max(g edge (x, y)) represents the maximum gray value of the edge map, x, y represent the row and column of the image.

[0084] Step five: Radon transform is performed from 1° to 180° with an interval of 1°, and the angle corresponding to the maximum value in the transform result is the blur angle θ estimate .

[0085]

[0086] θ estimate = {θ | max(R)} 8)

[0088] where R(ρ, θ) is the Radon transform result, g binary (x, y) is the spectrum Figure Two value after binarization, θ estimate is the blur angle, ρ is the distance of the straight line to the origin, θ is the angle between the vertical line of the straight line passing through the origin and the X-axis, x, y represent the row and column of the image.

[0089] Step six: according to the blur angle estimation result, the cropped spectrum pattern is rotated clockwise by the same angle, the gray values are summed by column, the average distance d of the minimum value points of the curve is calculated, and the blur scale L estimate is obtained to realize the motion blur parameter estimation.

[0090]

[0091] where L estimate is the blur scale, d is the average distance, and N is the width and height of the image.

[0092] Finally, the image is restored by Wiener filtering.

[0093]

[0094]

[0095] where K is usually taken as 0.01, L estimate is the blur scale, h(x, y) is the point spread function, H(u, v) is the Fourier transform of the point spread function, is the Fourier transform of the restored image, G(u, v) is the Fourier transform of the blurred image, θ estimate is the blur angle, x, y represent the row and column of the point spread function, u, v represent the row and column of the Fourier transform result.

[0096] Embodiment

[0097] As shown in Figure 2 , an unknown motion blur is applied to the clear image shown in Figure 2 (a) to obtain the motion blurred image shown in Figure 2 (b);

[0098] As shown in Figure 3 , a two-dimensional Fourier transform is performed on the blurred image shown in Figure 2 (b), and after compression of the dynamic range and quadrant conversion, the frequency spectrum shown in Figure 3 (a) is obtained, and after frequency spectrum clipping and Gaussian low-pass filtering, the partial frequency spectrum shown in Figure 3 (b) is obtained;

[0099] As shown in Figure 4 , the partial frequency spectrum shown in Figure 3 (b) is subjected to edge extraction using a structure forest algorithm to obtain the edge map shown in Figure 4 (a), and after binarization, the main lobe edge map shown in Figure 4 (b) is obtained;

[0100] As shown in Figure 5 , a Radon transform is performed on the main lobe edge map of Figure 4 (b) to obtain the result shown in Figure 5 , and the angle corresponding to the bright spot in the figure is the blur angle;

[0101] As shown in Figure 6 , the partial frequency spectrum of Figure 3 (b) is rotated according to the blur angle, so that the stripes are in the vertical direction, as shown in Figure 6 (a), and projection is performed in the vertical direction to obtain the curve shown in Figure 6 (b), and the average distance between adjacent minimum value points is detected to obtain the blur scale;

[0102] As shown in Figure 7 , the partial frequency spectrum of Figure 7(a) the motion-blurred image shown, with the blur parameter estimation and image restoration according to the present application, resulting in Figure 7 (b) the restored image shown.

[0103] Although the present application has been described in detail by the foregoing preferred embodiments, it is to be understood that such foregoing description is to be considered as illustrative of the principles of the application. Various modifications and alternative forms of the application will be apparent to those skilled in the art upon reading this description. Therefore, the scope of the application should be determined not with reference to the above description but should instead be determined with reference to the appended claims, along with their full scope of equivalents.

[0104] The details of the application described herein are not intended to limit the scope of the application. Rather, the scope of the application is limited only by the claims.

Claims

1. A method for non-cooled infrared image motion blur parameter estimation, characterized in that, The method comprises the following steps: obtaining a spectrum of a motion-blurred image; cutting a middle part of the spectrum to obtain a cut spectrum; performing Gaussian low-pass filtering on the cut spectrum to suppress cross bright lines and noise point interference and obtain a spectrum fringe image; extracting a spectrum fringe edge from the spectrum fringe image by using a structure forest algorithm; performing binarization on the spectrum fringe edge image to obtain a main lobe fringe edge; performing Radon transformation on the main lobe fringe edge to estimate a blur angle; estimating a blur scale by a spectrum projection method according to the estimation result of the blur angle, and realizing motion-blurred parameter estimation; The spectral fringe pattern is subjected to a structure forest algorithm to extract spectral fringe edges, which uses structured learning for edge detection, uses image blocks instead of pixels, trains a random forest classifier on an image segmentation benchmark dataset, and performs edge detection on the image to obtain an edge map g edge (x,y).

2. The method of claim 1, wherein: The method for obtaining the spectrum of the motion-blurred image comprises the following steps: S1, performing two-dimensional discrete Fourier transform on the blurred image f(x, y), and the formula is as follows: Wherein, F(u, v) represents the spectrum of the blurred image, N represents the side length of the image, x and y represent the row and column of the blurred image, and u and v represent the row and column of the Fourier transform result; S2, compressing the dynamic range by taking logarithm, and the formula is as follows: F(u, v) = log(|F(u, v)| + 1) Wherein, |·| represents taking the amplitude, log(·) represents taking the logarithm with 10 as the base, and u and v represent the row and column of the Fourier transform result; S3, respectively exchanging the first and third quadrants and the second and fourth quadrants of F(u, v) to perform spectrum centering, and obtaining a motion-blurred image spectrum g(x, y). 3.The non-cooled infrared image motion blur parameter estimation method of claim 1, wherein: The method for cutting the middle part of the spectrum to obtain the cut spectrum comprises the following steps: Wherein, g(x, y) is a spectrum function, N is the side length of the spectrum, and x and y represent the row and column of the spectrum.

4. The non-cooled infrared image motion blur parameter estimation method of claim 1, wherein: The method for performing Gaussian low-pass filtering on the cut spectrum to suppress cross bright lines and noise point interference and obtain a spectrum fringe image comprises the following steps: G(u,v) = G'(u,v) * H lowpass (u,v) where G(u,v) is the frequency spectrum fringe pattern, D 2 (u,v) is the distance of point (u,v) from the center point, D0is the low-pass filter cut-off frequency, G'(u,v) is the two-dimensional Fourier transform of the spectrum pattern, and u and v are the row and column of the Fourier transform result.

5. The non-cooled infrared image motion blur parameter estimation method of claim 1, wherein: The method for performing binarization on the spectrum fringe edge image to obtain a main lobe fringe edge comprises the following steps: where g binary (x, y) is the result of binarization of the spectrum map, max(g edge (x, y)) is the maximum gray value of the edge map, x and y are the rows and columns of the image.

6. The non-cooled infrared image motion blur parameter estimation method of claim 1, wherein: The method for performing Radon transformation on the main lobe fringe edge to estimate a blur angle comprises the following steps: θ estimate = {θ|max(R)} where R(p, q) is the Radon transform result, g binary (x, y) is the binarization result of the spectrum map, q estimate is the blur angle, p is the distance of the straight line to the origin, q is the angle between the vertical line of the straight line passing through the origin and the X-axis, x and y represent the rows and columns of the image; and d(·) is the Dirac function.

7. The method of claim 1, wherein: The method for estimating a blur scale by a spectrum projection method according to the estimation result of the blur angle, and realizing motion-blurred parameter estimation comprises the following steps: According to the estimation result of the blur angle, the cut spectrum is rotated clockwise by the same angle to obtain a rotated spectrum; Calculating the gray value of the rotated spectrum and summing by column to obtain a detection curve; According to the minimum point of the detection curve, the average interval d between adjacent minimum points is calculated to obtain the fuzzy scale L estimate ; According to the fuzzy scale L estimate The rotated frequency spectrum is subjected to Wiener filtering to realize image restoration.

8. The method of claim 7, wherein: The blur scale L is calculated estimate Specifically, where L estimate is the blur scale, d is the average spacing, and N is the side length of the image.

9. The method of claim 7, wherein: The method for performing Wiener filtering on the rotated spectrum to realize image restoration comprises the following steps: where K is a constant, L estimate is the blur scale, h(x, y) is the point spread function, H(u, v) is the Fourier transform of the point spread function, is the Fourier transform of the restored image, G(u, v) is the spectrum fringe pattern, θ estimate is the blur angle, x, y are the rows and columns of the point spread function, u, v are the rows and columns of the Fourier transform result.

Images