A method and system for defocus image restoration based on spot image estimation and point spread function

Abstract

This invention discloses a method and system for restoring defocused images based on estimating the point spread function from a spot image. The method includes: 1) reading the original spot image and the defocused blurred image; 2) cropping the area where the spot is located; 3) projecting the cropped spot image into grayscale along the vertical axis to obtain the original line spread function curve; 4) performing Gaussian fitting on the original line spread function curve to obtain a smooth line spread function curve; and then calculating the standard deviation of the Gaussian function; 5) calculating the blur coefficient of the Gaussian defocus model using the standard deviation; 6) substituting the blur coefficient into the Gaussian defocus model to obtain the point spread function; 7) performing Wiener filtering on the defocused blurred image in the frequency domain to obtain the initial restored image in the frequency domain; 8) performing a second Wiener filtering on the initial restored image to obtain the final restored image in the frequency domain; and 9) performing an inverse Fourier transform to obtain the restored image. This invention effectively reduces the blurriness of defocused images and solves the problem of computational complexity.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method and system for restoring defocused images based on the point spread function estimated from a spot image. Background Technology

[0002] Currently, autofocus technology is a crucial function in imaging systems and has received widespread attention. However, due to the unique properties of the imaging target itself, inaccurate focusing frequently occurs. For example, the surface of a wafer is almost mirror-like, making precise focusing and obtaining a clear image difficult; in other words, the wafer images acquired are usually out-of-focus images. Analyzing and processing these out-of-focus wafer images and extracting useful information is quite challenging. To extract features from out-of-focus wafer images for defect detection, image restoration, or image sharpening, is essential.

[0003] In image restoration, based on whether the point spread function (PSF) is known, defocused image restoration methods are divided into blind convolution restoration methods and non-blind convolution restoration methods. The former is suitable when the PSF of a blurred image is unknown, requiring prior knowledge of the blurred image itself to recover a sharp image, but it faces highly ill-conditioned problems. The latter is suitable when the PSF of a blurred image is known, allowing direct deconvolution operations on the defocused image, effectively improving image sharpness while suppressing various noises, and is currently the main method for defocused image restoration. For example, training an RBF neural network in the wavelet domain to estimate the PSF and using an iterative Wienner filter to complete the defocused image restoration. This method improves image quality and has good robustness, but is computationally complex. Another method obtains the PSF through the edge-edge method and then completes the image restoration through inverse filtering. Although this method has good restoration results, singular or near-singular values ​​in the matrix can lead to unsatisfactory results during calculation. Yet another method uses the differential autocorrelation method to estimate the PSF and then uses the IE-Wienner filtering algorithm to restore the defocused image. This method reduces ringing, but it is not effective when processing images with rich detail. Summary of the Invention

[0004] To address the shortcomings of existing technologies and solve problems such as the quality of blurred images and the computational complexity in the restoration process, this invention provides a method and system for restoring defocused images based on the point spread function of light spot images, thereby improving the quality of defocused images, highlighting image details, and reducing computational complexity.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for restoring defocused images based on estimating the point spread function from a spot image includes:

[0007] Step 1: Read the original image of the light spot and the corresponding out-of-focus blurred image;

[0008] Step 2: Crop the area containing the light spot from the original image of the light spot;

[0009] Step 3: Project the cropped spot image onto the vertical axis to obtain the original line spread function curve;

[0010] Step 4: Perform Gaussian fitting on the original linear diffusion function curve to obtain a smoothed linear diffusion function curve; calculate the standard deviation of the Gaussian function based on the smoothed linear diffusion function curve.

[0011] Step 5: Calculate the blur coefficient of the Gaussian defocus model using the standard deviation of the Gaussian function;

[0012] Step 6: Substitute the blur coefficients of the Gaussian defocus model into the Gaussian defocus model to obtain the point spread function;

[0013] Step 7: Convert the out-of-focus blurred image to the frequency domain, and perform Wiener filtering on the out-of-focus blurred image in the frequency domain based on the point spread function to obtain the initial restored image in the frequency domain.

[0014] Step 8: Perform a second Wiener filter on the initial restored image in the frequency domain to obtain the final restored image in the frequency domain;

[0015] Step 9: Perform an inverse Fourier transform on the final restored image in the frequency domain to obtain the final restored image.

[0016] To optimize the above technical solution, the specific measures also include:

[0017] Furthermore, step three specifically involves:

[0018] The cropped spot image is converted to grayscale to obtain a grayscale matrix image of the cropped spot image. The grayscale distribution along the vertical axis of the grayscale matrix image is then projected to obtain the line spread function curve.

[0019] Furthermore, step four specifically involves:

[0020] Gaussian fitting is performed on the original linear diffusion function curve using the following formula to obtain a smooth linear diffusion function curve;

[0021]

[0022] In the formula, a is the amplitude, b is the expectation, e represents the natural constant, c is the standard deviation of the Gaussian function, q is the input variable, representing the pixel sequence value of the original line spread function curve, and p is the grayscale projection value of the smoothed line spread function curve after Gaussian fitting.

[0023] The standard deviation of the Gaussian function is determined from the spread function curve of the smooth line.

[0024] Furthermore, step five specifically involves:

[0025] The blur coefficient of the Gaussian defocus model is calculated using the following formula:

[0026] σ=kc

[0027] In the formula, σ is the blur coefficient of the Gaussian defocus model, k represents a constant, and c is the standard deviation of the Gaussian function.

[0028] Furthermore, in step six, the Gaussian defocus model specifically refers to:

[0029]

[0030] Where h(x,y) represents the point spread function, x and y represent the x and y coordinates of the pixels in the defocused blurred image, and σ is the blur coefficient of the Gaussian defocus model.

[0031] Furthermore, step seven specifically involves:

[0032] The defocused blurred image is converted to the frequency domain to obtain the defocused blurred image G(u,v) in the frequency domain;

[0033] Wiener filtering is applied to the defocused blurred image in the frequency domain based on the point spread function to obtain the initial restored image F(u,v) in the frequency domain, which can be expressed by the following formula:

[0034]

[0035] Where [] represents the Wiener filter, H(u,v) represents the point spread function in the frequency domain, H * (u,v) represents the conjugate of H(u,v), where m is a constant.

[0036] Furthermore, step eight specifically includes:

[0037] A second Wiener filter is applied to the initial restored image in the frequency domain to obtain the final restored image in the frequency domain, as expressed by the following formula:

[0038]

[0039] In the formula, M(u,v) is the final restored image in the frequency domain, H(u,v) represents the point spread function in the frequency domain, and H... * (u,v) represents the conjugate of H(u,v), n is a constant, and F(u,v) is the initial restored image in the frequency domain.

[0040] This invention also proposes a defocus image restoration system based on the point spread function estimation of spot image, comprising: an image acquisition module, an image cropping module, a grayscale projection module, a Gaussian fitting module, a point spread function calculation module, a Wiener filtering module, and an inverse Fourier transform module;

[0041] The image acquisition module reads the original image of the light spot and the corresponding defocused blurred image;

[0042] The image cropping module crops out the area where the light spot is located from the original image of the light spot;

[0043] The grayscale projection module performs grayscale projection on the cropped spot image in the vertical axis to obtain the original line spread function curve.

[0044] The Gaussian fitting module performs Gaussian fitting on the original line spread function curve to obtain a smooth line spread function curve.

[0045] The point spread function calculation module calculates the blur coefficient of the Gaussian defocus model using the standard deviation of the Gaussian function; the blur coefficient of the Gaussian defocus model is then substituted into the Gaussian defocus model to obtain the point spread function.

[0046] The Wiener filtering module converts the out-of-focus blurred image to the frequency domain, performs Wiener filtering on the out-of-focus blurred image in the frequency domain based on the point spread function, and obtains the initial restored image in the frequency domain; a second Wiener filtering is performed on the initial restored image in the frequency domain to obtain the final restored image in the frequency domain.

[0047] The inverse Fourier transform module performs an inverse Fourier transform on the final restored image in the frequency domain to obtain the final restored image.

[0048] The beneficial effects of this invention are:

[0049] This invention primarily uses spot images to obtain line spread function curves for defocused images of varying degrees of defocus, providing a more convenient foundation for subsequent point spread function calculation. Gaussian fitting is then applied to the line spread function curves to accurately determine their standard deviation. The point spread function is then calculated using a Gaussian defocus model. Finally, a quadratic Wiener filtering restoration algorithm is employed to restore the defocused image, effectively reducing its blurriness and addressing issues of image detail and computational complexity. This method can be used to process defocused images acquired by digital imaging systems and has broad application value. Attached Figure Description

[0050] Figure 1 This is a flowchart of a defocused image restoration method based on the point spread function estimation of a spot image, as proposed in this invention.

[0051] Figure 2This is a comparison of the defocused blur image restoration effect of the defocused image restoration method based on the point spread function estimation of spot image proposed in this invention;

[0052] Figure 3 This is the restored image under the condition of a defocus amount of 0.07mm in the defocus image restoration method based on the point spread function estimation of spot image proposed in this invention;

[0053] Figure 4 This is the restored image under the condition of a defocus amount of 0.14mm in the defocus image restoration method based on the point spread function estimation of spot image proposed in this invention;

[0054] Figure 5 This is the restored image under the condition of a defocus amount of 0.21mm in the defocus image restoration method based on the point spread function of the spot image proposed in this invention. Detailed Implementation

[0055] The invention will now be described in further detail with reference to the accompanying drawings.

[0056] In one embodiment, the present invention proposes a method for restoring defocused images based on estimating the point spread function of a spot image, see [link to relevant documentation]. Figures 1-5 The method includes:

[0057] Step 1: Read the original image of the light spot and the corresponding out-of-focus blurred image;

[0058] Step 2: Crop the area where the light spot is located from the original image of the light spot; Since the light spot image occupies a small area in the whole image, the area where the light spot is located can be cut out from the whole image by image cropping.

[0059] Step 3: Under ideal conditions, the impulse response of an imaging or optical system to an ideal line light source is called the line spread function. The cropped spot image is projected in grayscale along the vertical axis to obtain the original line spread function curve; the cropped spot image exhibits a diffusion effect along the horizontal axis. Step 3 specifically involves:

[0060] The cropped spot image is converted to grayscale to obtain a grayscale matrix image of the cropped spot image. The grayscale distribution along the vertical axis of the grayscale matrix image is then projected to obtain the line spread function curve.

[0061] Step 4: Due to the fluctuations in the line spread function curve, significant errors are introduced into the point spread function estimation. To overcome this problem, Gaussian fitting is performed on the original line spread function curve to obtain a smoothed line spread function curve; the standard deviation of the Gaussian function is then calculated based on the smoothed line spread function curve; Step 4 specifically involves:

[0062] Gaussian fitting is performed on the original linear diffusion function curve using the following formula to obtain a smooth linear diffusion function curve;

[0063]

[0064] In the formula, a is the amplitude, b is the expectation, e represents the natural constant, c is the standard deviation of the Gaussian function, q is the input variable, representing the pixel sequence value of the original line spread function curve, and p is the grayscale projection value of the smoothed line spread function curve after Gaussian fitting.

[0065] The standard deviation of the Gaussian function is determined from the spread function curve of the smooth line.

[0066] Step 5: Calculate the blur coefficient of the Gaussian defocus model using the standard deviation of the Gaussian function. Based on multiple experimental results, under certain conditions, there is a quantitative relationship between the standard deviation of the Gaussian function and the blur coefficient of the Gaussian defocus model. This relationship can be expressed by the following expression:

[0067] σ=kc

[0068] The blur coefficient of the Gaussian defocus model is calculated using the above formula;

[0069] In the formula, σ is the blur coefficient of the Gaussian defocus model, k represents a constant, and c is the standard deviation of the Gaussian function.

[0070] Step Six: Substitute the blur coefficients of the Gaussian defocus model into the Gaussian defocus model to obtain the point spread function (PSF); the Gaussian defocus model is specifically as follows:

[0071]

[0072] Where h(x,y) represents the point spread function, x and y represent the x and y coordinates of the pixels in the defocused blurred image, and σ is the blur coefficient of the Gaussian defocus model.

[0073] Step 7: Convert the out-of-focus blurred image to the frequency domain to obtain the out-of-focus blurred image G(u,v) in the frequency domain; perform Wiener filtering on the out-of-focus blurred image in the frequency domain based on the point spread function to obtain the initial restored image in the frequency domain.

[0074] The principle of Wiener filtering restoration for restoring the original image is as follows:

[0075]

[0076] Where: [] represents the Wiener filter, H(u,v) represents the point spread function in the frequency domain, H * (u,v) represents the conjugate of H(u,v), P f (u,v) represents the power spectrum of the undistorted image, P n(u,v) represents the noise power spectrum. F(u,v) represents the initial restored image in the frequency domain, and G(u,v) represents the defocused blurred image in the frequency domain. Typically, the power spectrum is difficult to estimate accurately, so the following formula is often used to approximate the Wiener filter:

[0077]

[0078] Where [] represents the Wiener filter, H(u,v) represents the point spread function in the frequency domain, H * (u,v) represents the conjugate of H(u,v), and m is a constant, usually ranging from 0.0001 to 0.01. In different experiments, the choice of m depends on the experimental conditions.

[0079] Step 8: Perform a second Wiener filter on the initial restored image in the frequency domain to obtain the final restored image in the frequency domain; Step 8 specifically involves:

[0080] A second Wiener filter is applied to the initial restored image in the frequency domain to obtain the final restored image in the frequency domain, as expressed by the following formula:

[0081]

[0082] In the formula, M(u,v) is the final restored image in the frequency domain, H(u,v) represents the point spread function in the frequency domain, and H... * (u,v) represents the conjugate of H(u,v), n is a constant, usually ranging from 0.0001 to 0.01, and F(u,v) is the initial restored image in the frequency domain.

[0083] Step 9: Perform an inverse Fourier transform on the final restored image in the frequency domain to obtain the final restored image.

[0084] In another embodiment, the present invention proposes a defocus image restoration system based on the point spread function estimation of spot image, characterized in that it includes: an image acquisition module, an image cropping module, a grayscale projection module, a Gaussian fitting module, a point spread function calculation module, a Wiener filtering module, and an inverse Fourier transform module.

[0085] The image acquisition module reads the original image of the light spot and the corresponding defocused blurred image;

[0086] The image cropping module crops out the area where the light spot is located from the original image of the light spot;

[0087] The grayscale projection module performs grayscale projection on the cropped spot image in the vertical axis to obtain the original line spread function curve.

[0088] The Gaussian fitting module performs Gaussian fitting on the original line spread function curve to obtain a smooth line spread function curve.

[0089] The point spread function calculation module calculates the blur coefficient of the Gaussian defocus model using the standard deviation of the Gaussian function; the blur coefficient of the Gaussian defocus model is then substituted into the Gaussian defocus model to obtain the point spread function.

[0090] The Wiener filtering module converts the out-of-focus blurred image to the frequency domain, performs Wiener filtering on the out-of-focus blurred image in the frequency domain based on the point spread function, and obtains the initial restored image in the frequency domain; a second Wiener filtering is performed on the initial restored image in the frequency domain to obtain the final restored image in the frequency domain.

[0091] The inverse Fourier transform module performs an inverse Fourier transform on the final restored image in the frequency domain to obtain the final restored image.

[0092] The implementation methods of each module in the system are completely consistent with the steps of the method in Example 1.

[0093] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for restoring defocused images based on estimating the point spread function of a spot image, characterized in that, include: Step 1: Read the original image of the light spot and the corresponding out-of-focus blurred image; Step 2: Crop the area containing the light spot from the original image of the light spot; Step 3: Project the cropped spot image onto the vertical axis to obtain the original line spread function curve; Step 4: Perform Gaussian fitting on the original linear diffusion function curve to obtain a smoothed linear diffusion function curve; calculate the standard deviation of the Gaussian function based on the smoothed linear diffusion function curve. Step 5: Calculate the blur coefficient of the Gaussian defocus model using the standard deviation of the Gaussian function; Step 6: Substitute the blur coefficients of the Gaussian defocus model into the Gaussian defocus model to obtain the point spread function; Step 7: Convert the out-of-focus blurred image to the frequency domain, and perform Wiener filtering on the out-of-focus blurred image in the frequency domain based on the point spread function to obtain the initial restored image in the frequency domain. Step 8: Perform a second Wiener filter on the initial restored image in the frequency domain to obtain the final restored image in the frequency domain; Step 9: Perform an inverse Fourier transform on the final restored image in the frequency domain to obtain the final restored image.

2. The method for restoring defocused images based on the point spread function estimation of spot images according to claim 1, characterized in that, Step three specifically involves: The cropped spot image is converted to grayscale to obtain a grayscale matrix image of the cropped spot image. The grayscale distribution along the vertical axis of the grayscale matrix image is then projected to obtain the line spread function curve.

3. The method for restoring defocused images based on the point spread function estimation of spot images according to claim 1, characterized in that, Step four is as follows: Gaussian fitting is performed on the original linear diffusion function curve using the following formula to obtain a smooth linear diffusion function curve; In the formula, a is the amplitude, b is the expectation, e represents the natural constant, c is the standard deviation of the Gaussian function, q is the input variable, representing the pixel sequence value of the original line spread function curve, and p is the grayscale projection value of the smoothed line spread function curve after Gaussian fitting. The standard deviation of the Gaussian function is determined from the spread function curve of the smooth line.

4. The method for restoring defocused images based on the point spread function estimation of spot images according to claim 1, characterized in that, Step five is as follows: The blur coefficient of the Gaussian defocus model is calculated using the following formula: σ=kc In the formula, σ is the blur coefficient of the Gaussian defocus model, k represents a constant, and c is the standard deviation of the Gaussian function.

5. The method for restoring defocused images based on the point spread function estimation of spot images according to claim 1, characterized in that, In step six, the Gaussian defocus model specifically refers to: Where h(x,y) represents the point spread function, x and y represent the x and y coordinates of the pixels in the defocused blurred image, and σ is the blur coefficient of the Gaussian defocus model.

6. The method for restoring defocused images based on the point spread function estimation of spot images according to claim 1, characterized in that, Step seven is as follows: The defocused blurred image is converted to the frequency domain to obtain the defocused blurred image G(u,v) in the frequency domain; Wiener filtering is applied to the defocused blurred image in the frequency domain based on the point spread function to obtain the initial restored image F(u,v) in the frequency domain, which can be expressed by the following formula: Where [] represents the Wiener filter, H(u,v) represents the point spread function in the frequency domain, H * (u,v) represents the conjugate of H(u,v), where m is a constant.

7. The method for restoring defocused images based on the point spread function estimation of spot images according to claim 1, characterized in that, Step eight is as follows: A second Wiener filter is applied to the initial restored image in the frequency domain to obtain the final restored image in the frequency domain, as expressed by the following formula: In the formula, M(u,v) is the final restored image in the frequency domain, H(u,v) represents the point spread function in the frequency domain, and H... * (u,v) represents the conjugate of H(u,v), n is a constant, and F(u,v) is the initial restored image in the frequency domain.

8. A defocus image restoration system based on the point spread function estimation of a spot image, characterized in that, include: The system includes an image acquisition module, an image cropping module, a grayscale projection module, a Gaussian fitting module, a point spread function calculation module, a Wiener filtering module, and an inverse Fourier transform module. The image acquisition module reads the original image of the light spot and the corresponding defocused blurred image; The image cropping module crops out the area where the light spot is located from the original image of the light spot; The grayscale projection module performs grayscale projection on the cropped spot image in the vertical axis to obtain the original line spread function curve. The Gaussian fitting module performs Gaussian fitting on the original line spread function curve to obtain a smooth line spread function curve. The point spread function calculation module calculates the blur coefficient of the Gaussian defocus model using the standard deviation of the Gaussian function; the blur coefficient of the Gaussian defocus model is then substituted into the Gaussian defocus model to obtain the point spread function. The Wiener filtering module converts the out-of-focus blurred image to the frequency domain, performs Wiener filtering on the out-of-focus blurred image in the frequency domain based on the point spread function, and obtains the initial restored image in the frequency domain; a second Wiener filtering is performed on the initial restored image in the frequency domain to obtain the final restored image in the frequency domain. The inverse Fourier transform module performs an inverse Fourier transform on the final restored image in the frequency domain to obtain the final restored image.

Images