A method for restoring a sinusoidal motion-blurred image based on a prior model

Abstract

The present invention discloses a method for restoring sinusoidal motion blurred images based on a priori models. The point spread function of the sinusoidal motion sequence images is calculated by using the gradient optical flow method and the Gaussian fitting method. Based on the prior information of the sinusoidal excitation signal, the motion image at the maximum displacement position is used as a reference frame with a zero point spread function blur kernel. The Lucy-Richardson restoration method and the solved point spread function are used to enhance the characteristic edges of the corresponding sinusoidal motion blurred images to obtain restored images with clear characteristic edges. The noise of the restored images is eliminated by the NL-Means method to suppress the boundary ringing effect of the restored images and improve the signal-to-noise ratio. Finally, objective image quality evaluation indicators and sub-pixel edge detection methods based on Zernike moments are used to evaluate the effect of the restored images and extract the characteristic edges respectively. This method can meet the growing development needs of high-precision linear and angular vibration measurement, pose estimation, target detection, etc. in machine vision.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and more specifically, particularly relates to a method for restoring sinusoidal motion blurred images. Background Art

[0002] In recent years, acceleration sensors have been widely used in fields such as aircraft motion trajectory estimation, building structure health detection, robot navigation and positioning, machining, and precision instrument manufacturing. Since sinusoidal vibration can form arbitrary planar and spatial motion trajectories as a basic dynamic unit, it is a commonly used excitation signal in sensor vibration calibration. However, the edge extraction accuracy of the sinusoidal motion blurred sequence images captured by monocular vision directly affects the calibration parameters of the acceleration sensor. Therefore, studying the method for restoring sinusoidal motion blurred images is crucial for the effectiveness and accuracy of measurement data.

[0003] Motion blur causes the gray distribution of the image step edge to turn into a gradually transitional gray change, resulting in the inability to accurately extract the edge information of the acquired image. In recent years, image restoration methods have generally been divided into blind restoration and non-blind restoration, and the accuracy of the calculation of the point spread function directly determines the restoration quality of the image. For the blind restoration of images, the point spread function is used as an unknown quantity, and its solution accuracy greatly affects the image restoration effect. The non-blind restoration method deconvolves the sinusoidal motion blurred image using an accurate point spread function, and its restoration quality is more accurate and efficient, and is of great significance for the development of blind restoration algorithms.

[0004] Therefore, for the current methods for restoring motion blurred images, there are deficiencies such as loss of edge detail information, large noise, and poor preservation of texture features, and it is difficult to be applied to the field of high-precision linear and angular vibration calibration in machine vision. The present invention proposes a method for restoring sinusoidal motion blurred images with advantages such as clear edges, small noise, and high preservation degree of texture features, which can meet the growing development needs of high-precision linear and angular vibration measurement in machine vision. Summary of the Invention

[0005] Aiming at the deficiencies of the current methods for restoring motion blurred images, such as loss of edge detail information, large noise, and poor preservation of texture features, the present invention proposes a method for restoring sinusoidal motion blurred images with clear edges, small noise, and high preservation degree of texture features, including:

[0006] Obtaining the point spread function of the sinusoidal motion blurred image based on the prior model: used to implement the solution of the point spread function for each frame, including: calculating the point spread function of the sinusoidal motion sequence image by the gradient optical flow method and the Gaussian fitting method, using the prior model of the sinusoidal excitation signal to take the motion image at the maximum displacement as the reference frame with a zero blur kernel, and then obtaining the point spread function of each frame of image;

[0007] The LR restoration method enhances the characteristic edges of a sinusoidal motion blurred image: The restoration coefficient H is obtained by deconvolving the solved point spread function with the identity matrix (0) (x, y), determine the restored image of the first iteration, and solve the blurring coefficient K by the ratio of the sinusoidal motion blurred image to the restored image of the first iteration (1) (x, y) to complete the next iteration process and obtain a restored image with clear characteristic edges;

[0008] The NL-Means method is used to solve and eliminate the noise in the image: For the restored image with clear characteristic edges obtained, noise reduction is achieved by calculating the texture similarity weight between pixel neighborhoods and the mean value of pixel neighborhoods. The Gaussian weighted Euclidean distance is used to determine the similarity degree between pixels and their neighborhoods, and the mean value of pixel neighborhoods is used to smooth the noise, improving the boundary ringing effect and signal-to-noise ratio of the restored image;

[0009] The technical solution adopted by the present invention is a non-blind restoration method for sinusoidal motion blurred images based on a prior model, and the method includes the following steps:

[0010] S1: Use the gradient optical flow method and the Gaussian fitting method to calculate the point spread function of the sinusoidal motion blurred image. Based on the prior information of sinusoidal excitation, the sinusoidal motion blurred image at the maximum displacement position is used as the reference frame where the point spread function blur kernel is zero, and then the point spread function of the remaining sinusoidal motion blurred images is solved;

[0011] S2: Adopt the LR restoration method and the point spread function solved in S1 to enhance the characteristic edges of the corresponding sinusoidal motion blurred image to obtain a sinusoidal motion blurred restored image with clear characteristic edges;

[0012] S3: Eliminate the noise of the restored image of the sinusoidal motion blur with clear characteristic edges in S2 based on the NL-Means method to improve the boundary ringing effect and peak signal-to-noise ratio of the restored image;

[0013] S4: Evaluate and extract the characteristic edges of the restored image in S3 through objective image quality evaluation indicators and the sub-pixel edge detection method based on Zernike moments respectively to verify the improvement effect of the restoration method on the sinusoidal motion blurred image;

[0014] The solution of the point spread function of the sinusoidal motion sequence image based on the sinusoidal excitation prior model specifically includes:

[0015] First, use the gradient optical flow method to judge the change of the motion direction of the sinusoidal motion blurred image, and fit the gray gradient in the neighborhood of the motion characteristic edge of the blurred image through the Gaussian fitting method of the following formula:

[0016]

[0017] In the formula: μx and μ y are the fitted grayscale gradient coordinates of the feature edge; A max is the maximum grayscale gradient; σ x and σ y are the standard deviations in the x - direction and y - direction, e is the natural base, and (x, y) are the coordinates in the x - direction and y - direction of the sinusoidal motion - blurred image.

[0018] Secondly, based on the prior information of the sinusoidal excitation model, select the motion image at the maximum displacement position as the reference frame where the point - spread function blur kernel is zero; finally, solve the point - spread function of the sinusoidal motion - blurred sequence image by the difference between the blur kernels corresponding to the motion images at the remaining positions and the reference - frame blur kernel.

[0019] The LR restoration method realizes the restoration of the motion - blurred image through iterative loops. First, use the solved point - spread function G(x, y) to obtain the restoration coefficient H (0) (x, y) for the first - iteration restoration of the corresponding sinusoidal motion - blurred image F(x, y):

[0020]

[0021] where is convolution; E is the identity matrix, and the superscript T represents matrix transpose; then, multiply the obtained restoration coefficient H (0) (x, y) by the sinusoidal motion - blurred image to get the restored image I (1) (x, y) for the first - iteration:

[0022] I (1) (x, y) = F(x, y)×H (0) (x, y) (3)

[0023] Then, use the ratio of the blurred image to the iterative image to obtain the blur coefficient K (1) (x, y) for the first - iteration:

[0024]

[0025] Finally, from equations (2) - (4), obtain the LR - restored image I (n) (x, y) after n - iterations:

[0026]

[0027] Use the LR restoration method to denoise the restored image I (n) (x, y), which can be obtained by the NL - Means method as follows:

[0028]

[0029] Where: NLMeans(x,y) is the denoised image of the sinusoidal motion blurred restored image, is the average value of the pixels within the search window in I(x,y); w(x0,y0,u0,v0) is the texture similarity between the denoised pixel (x0,y0) and the comparison pixel (u0,v0) within the search window, and its expression is:

[0030]

[0031] Where: N8(x0,y0) and N8(u0,v0) are the 8-neighborhoods of the denoised pixel (x0,y0) and the comparison pixel (u0,v0) within the search window respectively; a is the smoothing noise coefficient; is the Gaussian weighted Euclidean distance corresponding to the two 8-neighborhoods of the denoised pixel and the comparison pixel; the normalization coefficient z(x0,y0) is expressed as:

[0032]

[0033] In addition, the NL-Means method can effectively reduce the boundary ringing effect introduced by noise to the restored image through iterative denoising.

[0034] The non-blind restoration method of sinusoidal motion blurred images based on the prior model of the present invention has the following advantages:

[0035] (1) The method of the present invention obtains a restored image with clear edges and high texture feature retention through the solved point spread function and image iteration, and can be applied to the research of the key technology of high-precision linear angle vibration measurement in machine vision.

[0036] (2) The method of the present invention can obtain a restored image with a high signal-to-noise ratio by using the texture similarity weight between pixel neighborhoods and the pixel neighborhood mean, and can meet the requirements of high signal-to-noise ratio. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flowchart of a non-blind restoration method for sinusoidal motion blurred images based on a prior model;

[0038] Figures 2-3 are respectively the restored image of the sinusoidal motion blurred image and the sub-pixel coordinate error scatter plot of the feature edge in the specific implementation example of the method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0039] In view of the deficiencies of current motion-blurred image restoration methods, such as loss of edge detail information, high noise, and low accuracy in feature edge detection, and it is difficult to be applied to the field of high-precision linear and angular vibration calibration in machine vision. The present invention proposes a sine motion-blurred image restoration method with advantages such as clear edges, low noise, and high texture feature retention, which can meet the growing development needs of high-precision linear and angular vibration measurement in machine vision. The following makes a detailed description of the present invention in combination with the accompanying drawings and specific implementation examples.

[0040] Reference Figure 1 It is a flowchart of a non-blind restoration method for sine motion-blurred images based on a prior model. The method of the present invention mainly includes the following steps:

[0041] Step S1: Calculate the point spread function of the sine motion sequence image by using the gradient optical flow method and the Gaussian fitting method. Based on the prior information of the sine excitation signal, the motion image at the maximum displacement is used as the reference frame with a zero blur kernel, and then the point spread functions of the remaining images are obtained;

[0042] Step S2: Use the LR restoration method and the solved point spread function to restore the motion image, which includes: obtaining the restoration coefficient H (0) (x, y) by deconvolving the solved point spread function and the identity matrix, determining the restored image of the first iteration, and completing the next iteration process through the blur coefficient K (1) (x, y) obtained by the ratio of the sine motion-blurred image and the restored image of the first iteration to obtain a restored image with clear feature edges;

[0043] Step S3: Eliminate the noise of the restored image by the NL-Means method, which includes: setting the search window of the restored image, calculating the texture similarity between the original pixel (x, y) and the comparison pixel (u, v) in the search window by using the Gaussian weighted Euclidean distance, and smoothing the noise by the pixel mean in the search window to obtain a restored image with noise eliminated;

[0044] To verify the image restoration quality of the method of the present invention and the sub-pixel coordinate error of the extracted feature edges respectively, the sine motion-blurred images generated by the 0.5Hz planar motion displacement are restored and compared with other restoration methods. Table 1 shows the comparison table of the image quality evaluation results of the method of the present invention and other restoration methods. It can be seen from the results in Table 1 that the image quality evaluation results of the method of the present invention and the LR restoration method are similar, but its MSE, PSNR, EPI, and MAE indicators are more objective. In addition, although the EPI of the method of the present invention is 0.0175 higher than that of the direct inverse filtering method, the former has better MAE, MSE, and PSRN effects.

[0045] Table 1 Comparison table of image quality evaluation results of the method of the present invention and other restoration methods

[0046]

[0047] The above description is a detailed introduction to the embodiments of the present invention, which is not used to limit the present invention in any form. Those skilled in the relevant art can make a series of optimizations, improvements, and modifications based on the present invention. Therefore, the protection scope of the present invention should be defined by the appended claims.

Claims

1. A method for restoring a sinusoidal motion-blurred image based on a prior model, characterized in that: The method includes the following steps: S1: Calculate the point spread function of the sinusoidal motion blurred image using the gradient optical flow method and the Gaussian fitting method. Based on the prior information of the sinusoidal excitation, use the sinusoidal motion blurred image at the maximum displacement position as the reference frame where the point spread function blur kernel is zero, and then solve to obtain the point spread functions of the remaining sinusoidal motion blurred images; S2: Adopt the LR restoration method and the point spread function solved in S1 to enhance the feature edges of the corresponding sinusoidal motion blurred image to obtain a sinusoidal motion blurred restored image with clear feature edges; S3: Eliminate the noise of the sinusoidal motion blurred restored image with clear feature edges in S2 based on the NL-Means method to improve the boundary ringing effect and peak signal-to-noise ratio of the restored image; S4: Evaluate and extract the feature edges of the restored image in S3 through the objective image quality evaluation index and the sub-pixel edge detection method based on Zernike moments respectively to verify the improvement effect of the restoration method on the sinusoidal motion blurred image.

2. A method for restoring a sinusoidal motion blurred image based on a prior model according to claim 1, characterized in that: The solution of the point spread function of the sinusoidal motion sequence image based on the sinusoidal excitation prior model specifically includes: First, use the gradient optical flow method to judge the change in the motion direction of the sinusoidal motion blurred image, and fit the gray gradient in the neighborhood of the motion feature edge of the blurred image through the Gaussian fitting method of the following formula: Where: μ x and μ y are the fitted characteristic edge gray gradient coordinates; A max is the maximum gray gradient; σ x and σ y are the standard deviations in the x and y directions, e is the natural base, and (x, y) are the x and y direction coordinates of the sinusoidal motion blurred image; Secondly, based on the prior information of the sinusoidal excitation model, select the motion image at the maximum displacement position as the reference frame where the point spread function blur kernel is zero; finally, solve the point spread function of the sinusoidal motion blurred sequence image by the difference between the blur kernels of the motion images at other positions and the blur kernel of the reference frame.

3. A method for restoring a sinusoidal motion blurred image based on a prior model according to claim 1, characterized in that: The LR restoration method realizes the restoration of motion-blurred images through iterative cycles; first, the restoration coefficient H for the first iteration of the corresponding sinusoidal motion-blurred image F(x, y) is obtained by using the solved point spread function G(x, y). (0) (x, y): Among them, is convolution; E is the identity matrix, and the superscript T represents matrix transpose; then, from the obtained restoration coefficient H (0) (x, y) is multiplied by the sinusoidal motion-blurred image to obtain the restored image I (1) (x, y): I (1) (x,y) = F(x,y) × H (0) (x,y) (3) Using the ratio of the blurred image to the iterative image again, the blurring coefficient K for the first iteration is obtained (1) (x, y): Finally, from equations (2)-(4), the LR restored image I after n iterations is obtained (n) (x,y):

4. A method for restoring a sinusoidal motion blurred image based on a prior model according to claim 3, characterized in that: Using the LR restoration method to restore the image I (n) (x, y) to achieve noise reduction, which can be obtained by the NL-Means method as follows: Where: NLMeans(x, y) is the denoised image of the sinusoidal motion blurred restored image, which is the average value of the pixels within the search window in I(x, y); w(x0, y0, u0, v0) is the texture similarity between the denoised pixel (x0, y0) and the comparison pixel (u0, v0) within the search window, and its expression is: Where: N8(x0, y0) and N8(u0, v0) are the 8-neighborhoods of the noise-reduced pixel (x0, y0) and the comparison pixel (u0, v0) within the search window respectively; a is the smoothing noise coefficient; is the Gaussian weighted Euclidean distance corresponding to the two 8-neighborhoods of the noise-reduced pixel and the comparison pixel; the normalization coefficient z(x0, y0) is expressed as: In addition, the NL-Means method can effectively reduce the boundary ringing effect introduced by noise to the restored image through iterative noise reduction.

Images