Method and system for eliminating wavefront aberration based on space-variant point spread function

Abstract

The invention discloses a method and a system for eliminating wavefront aberration based on a space-variant point spread function. The method comprises the following steps: acquiring a real-time random noise image and a real-time blurred image of a space-variant system; inputting the real-time blurred image into the aberration regression network model to obtain a real-time aberration coefficient map; inputting the real-time random noise image and the real-time aberration coefficient atlas into the space-variant degradation model to generate a real-time degraded image; acquiring a real-time blurred image; acquiring a pixel value difference value between the real-time blurred image and the real-time degraded image according to the real-time blurred image and the real-time degraded image; and according to the pixel value difference value, updating iteration through a back propagation algorithm until the pixel value difference value is smaller than a threshold value, and outputting an optimized image. According to the method and the system for eliminating the wavefront aberration based on the space-variant point spread function, provided by the invention, the spatial variability under various complex imaging conditions can be effectively handled, and enough calculation efficiency can be kept.

Description

Technical Field

[0001] The present invention relates to the technical field of image restoration, and in particular to a method and system for eliminating wavefront aberration based on space-varying point spread function. Background Art

[0002] Image degradation refers to the deterioration of image quality during the formation, transmission and recording of images due to the imperfections of the imaging system, transmission media and equipment, which usually manifests as blur, distortion or noise. In order to solve the problem of imaging degradation, the image needs to be restored. Image restoration is based on the prior knowledge of image degradation, to establish a mathematical model of the degradation phenomenon (imaging degradation model), and then perform deduction operations based on the model to restore the original scene image. The key to image restoration is to know the image degradation process, that is, the image degradation model, and use the reverse process to obtain the original image.

[0003] The degradation model of imaging is generally written in the form of point spread function (PSF) convolution plus noise interference term. This model is based on the premise of spatial invariance. However, in practical applications, since the light involved in the imaging process is not limited to paraxial light, but also involves light far from the optical axis, these light rays have a large angle with the optical axis and cannot be simply described by the paraxial approximation. These non-paraxial light rays (also called oblique light rays or wide-angle light rays) may produce various aberrations when passing through the optical system, such as spherical aberration, coma, astigmatism, distortion and field curvature, etc. These aberrations will affect the quality and accuracy of imaging, so it is difficult to establish strict spatial invariance. In addition, ray tracing can be used to calculate the light wave transmission process from the object plane to the image plane, so as to calculate the PSF of the sampling point. In the paper "Spatially varying defocus map estimation from asingle image based on spatial aliasing sampling method", the PSF of the actual system is related to the spatial position. Only one PSF cannot represent the degradation relationship of the entire field of view, so it is difficult to inversely solve the aberration-free image through the PSF.

[0004] Lens aberrations and the three-dimensional structural depth of the object itself will cause the PSF to change at different spatial positions. This change makes the light energy distribution at each point closely related to other points, greatly increasing the difficulty of accurately inferring PSF information by observing the image. In the past few years, in order to solve this problem, researchers have proposed a variety of methods for estimating PSF, which can be roughly divided into two categories: non-parametric methods and parametric methods.

[0005] Parametric methods have received extensive attention and in-depth research because they can provide more intuitive and easy-to-understand physical explanations. Such methods are usually based on some assumptions about the characteristics of the optical system, such as the Gaussian blur model, and then adjust the model parameters through optimization algorithms to match the actual observation data. However, although some existing advanced parameterization techniques have been able to better handle the spatially varying Gaussian PSF caused by factors such as defocus, when faced with more complex aberration types (such as spherical aberration, coma, etc.), existing technologies can often only provide a simplified solution - that is, assuming that the PSF in the entire field of view or a larger area remains unchanged, and it is impossible to accurately describe the unique PSF characteristics of each specific pixel.

[0006] Invention patent 202411017542.X, a digital image deblurring method, device, equipment and storage medium, discloses constructing an image PSF field based on a point spread function, using an optimization algorithm to solve the image deblurring inversion objective function, and obtaining a deblurred digital image. This method does not consider the influence of optical aberrations on the point spread function surface shape, and secondly, it is necessary to obtain the coordinate parameter information of the image acquisition observation device, and it is impossible to directly obtain the PSF through a single image.

[0007] Therefore, how to develop a new method that can effectively deal with the spatial variability under various complex imaging conditions while maintaining sufficient computational efficiency has become a key issue that needs to be urgently addressed in this field. Summary of the invention

[0008] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a method and system for eliminating wavefront aberrations based on a space-varying point spread function, so as to solve the problems of inaccurate results and low computational efficiency caused by spatial variability under complex imaging conditions in the prior art.

[0009] In a first aspect, the present invention provides a method for eliminating wavefront aberration based on a space-varying point spread function, comprising: acquiring a real-time random noise image and a real-time blurred image of a space-varying system, wherein the space-varying system is a system in which a point spread function varies with pixel position.

[0010] The real-time blurred image is input into an aberration regression network model to obtain a real-time aberration coefficient map. The aberration regression network model is obtained by training historical blurred images and historical aberration coefficient maps of a space-variable system.

[0011] The real-time random noise image and the real-time aberration coefficient map are input into the space-variant degradation model, and the pixel value of each pixel position is diffused according to the real-time aberration coefficient map to generate a real-time degraded image.

[0012] Get real-time blurred images.

[0013] According to the real-time blurred image and the real-time degraded image, a pixel value difference between the real-time blurred image and the real-time degraded image is obtained.

[0014] According to the pixel value difference, the back propagation algorithm is used to update and iterate until the pixel value difference is less than the threshold, and the optimized image is output.

[0015] Optionally, the aberration regression network model is obtained by training the historical blurred images and historical aberration coefficient maps of the space-variant system, including:

[0016] Get the historical wave aberration of the space-variant system.

[0017] The historical wave aberration is simulated to generate a historical blurred image and a historical aberration coefficient map.

[0018] Multiple groups of historical blurred images and historical aberration coefficient maps are paired to construct a training dataset.

[0019] The historical blurred image is used as the input of the model, the corresponding historical aberration coefficient map is used as the output of the model, and the backbone network is set to HRNet for model training.

[0020] The mean square error is used to measure the difference between the predicted aberration coefficient map and the true value.

[0021] The Adam optimization algorithm is used for gradient descent, the network parameters are adjusted, the loss function is minimized, and the aberration regression network model is obtained.

[0022] Optionally, simulating the historical wave aberration to generate a historical blurred image and a historical aberration coefficient map includes:

[0023] Wave aberrations are represented using Zernike polynomials.

[0024] According to the wave aberration, the pupil function is calculated by Fresnel diffraction.

[0025] The pupil function is Fourier transformed to obtain the amplitude point spread function.

[0026] By taking the square of the modulus of the amplitude point spread function, the intensity point spread function of the incoherent imaging system is obtained.

[0027] Aberration-free image degradation via intensity point spread function.

[0028] Optionally, the wave aberration of the space-variant system is set in the range of 0 to 1λ.

[0029] Optionally, it also includes: constructing a PSF model for each pixel position in the space-variant system according to the pupil function and the amplitude point spread function, the model is as follows:

[0030]

[0031] Among them, A(x,y) is the aperture shape function, W(x,y) is the polynomial coefficient matrix, and PSF is the point spread function.

[0032] Optionally, the space-variant degradation model is:

[0033]

[0034] Among them, S(x,y) is the item to be optimized, B(x',y') is the blurred image after degradation, η is the image noise, and PSF(x'-x,y'-y) is the point spread function of the current pixel position.

[0035] Optionally, the threshold is set to 0.1 and the maximum number of iterations is 200.

[0036] In a second aspect, the present invention provides a system for eliminating wavefront aberrations based on a space-varying point spread function, which is applied to the method for eliminating wavefront aberrations based on a space-varying point spread function described in the first aspect, comprising:

[0037] The acquisition module is used to acquire a real-time random noise image and a real-time blurred image of a space-variant system, wherein the space-variant system is a system in which a point spread function varies with pixel positions.

[0038] The aberration regression network module is used to input the real-time blurred image into the aberration regression network model to obtain a real-time aberration coefficient map. The aberration regression network model is obtained by training the historical blurred images and historical aberration coefficient maps of the space-variable system.

[0039] The space-variant degradation module is used to input the real-time random noise image and the real-time aberration coefficient map into the space-variant degradation model, and diffuse the pixel value of each pixel position according to the real-time aberration coefficient map to generate a real-time degraded image.

[0040] The real-time blurred image module is used to obtain the real-time blurred image.

[0041] The aberration module is used to obtain the pixel value difference between the real-time blurred image and the real-time degraded image according to the real-time blurred image and the real-time degraded image.

[0042] The output module is used to update and iterate according to the pixel value difference through the back propagation algorithm until the pixel value difference is less than the threshold value, and output the optimized image.

[0043] It can be seen from the above technical scheme that the embodiment of the present invention provides a method and system for eliminating wavefront aberration based on space-varying point spread function, wherein the method includes: obtaining a real-time random noise image and a real-time blurred image of a space-varying system, wherein the space-varying system is a system in which the point spread function changes with the pixel position; inputting the real-time blurred image into an aberration regression network model to obtain a real-time aberration coefficient spectrum, wherein the aberration regression network model is obtained by training the historical blurred image and the historical aberration coefficient spectrum of the space-varying system; inputting the real-time random noise image and the real-time aberration coefficient spectrum into a space-varying degradation model, and diffusing the pixel values ​​of each pixel position according to the real-time aberration coefficient spectrum to generate a real-time degraded image; obtaining a real-time blurred image; obtaining the pixel value difference between the real-time blurred image and the real-time degraded image according to the real-time blurred image and the real-time degraded image; according to the pixel value difference, updating and iterating through the back propagation algorithm until the pixel value difference is less than a threshold value, and outputting the optimized image. A method and system for eliminating wavefront aberration based on space-varying point spread function provided by the present invention can effectively cope with spatial variability under various complex imaging conditions, and can maintain sufficient computational efficiency.

[0044] The following are the beneficial effects of the present invention:

[0045] Aiming at the defect that only a single point spread function can be used to describe the image degradation process in the prior art, the present invention calculates the aberration of each pixel position by Zernike polynomials, thereby solving the blurring phenomenon that the prior art cannot simulate the existence of a space-variant system.

[0046] By using a single image, the parameters of the point spread function at each pixel position can be obtained after passing through a space-variant aberration regression network, without the need for additional coordinate parameter information of an image acquisition observation device, and without the need to use image stack data with multiple focuses.

[0047] The space-variant system degradation model performs energy diffusion on the pixel value at each pixel position according to the aberration coefficient map, which is more in line with the actual imaging process than the traditional convolution operation.

[0048] The iterative optimization model of the space-variable system simulates the process of light reaching the image plane, which includes spherical aberration, coma, field curvature, astigmatism and distortion. It is not limited to a specific system and has stronger generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 It is a flow chart of a method for eliminating wavefront aberration based on space-varying point spread function provided by an embodiment of the present invention;

[0051] Figure 2 It is a schematic diagram of a process for constructing an aberration regression network model provided by an embodiment of the present invention;

[0052] Figure 3 is an aberration type diagram provided by an embodiment of the present invention;

[0053] Figure 4 It is a schematic diagram of the process of constructing a space-variant degradation model provided by an embodiment of the present invention;

[0054] Figure 5 It is a framework structure diagram of a system for eliminating wavefront aberration based on a space-varying point spread function provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

[0056] In the field of image restoration, when faced with complex imaging conditions, it is usually impossible to adapt to spatial variability, resulting in poor image optimization effects. In addition, existing optimization algorithms are too cumbersome.

[0057] Therefore, in order to solve the above technical problems, an embodiment of the present invention proposes a solution, that is, to provide a method and system for eliminating wavefront aberration based on space-varying point spread function.

[0058] See also Figure 1 , a schematic diagram of a flow chart of a method for eliminating wavefront aberration based on a space-varying point spread function provided by an embodiment of the present invention; an embodiment of the present invention provides a method for eliminating wavefront aberration based on a space-varying point spread function, comprising:

[0059] S1, obtaining a real-time random noise image and a real-time blurred image of a space-variant system, wherein the space-variant system is a system in which a point spread function varies with pixel positions.

[0060] A space-variant system is a system in which the point spread function (PSF) varies with the pixel position. The characteristic of this system is that the point spread function of each pixel position is different, so the image degradation process is space-variant.

[0061] S2, inputting the real-time blurred image into an aberration regression network model to obtain a real-time aberration coefficient map, wherein the aberration regression network model is obtained by training historical blurred images and historical aberration coefficient maps of a space-variable system.

[0062] See also Figure 2 , a schematic diagram of a flow chart of constructing an aberration regression network model provided by an embodiment of the present invention; specifically, the aberration regression network model is obtained by training the historical blurred images and historical aberration coefficient maps of the space-variant system, including:

[0063] S101, obtaining the historical wave aberration of the space-variant system.

[0064] See also Figure 3 , is an aberration type diagram provided in an embodiment of the present invention; the types of aberrations include but are not limited to spherical aberration, coma, field curvature, astigmatism and distortion.

[0065] S102, simulating the historical wave aberration to generate a historical blurred image and a historical aberration coefficient map.

[0066] Specifically, simulating the historical wave aberration to generate a historical blurred image and a historical aberration coefficient map includes:

[0067] S201, using Zernike polynomials to represent wave aberrations.

[0068] Zernike polynomials are a set of orthogonal polynomials on the unit circle, which are often used to describe wavefront aberrations in optical systems.

[0069] Specifically, the wave aberration of the space-variant system is set within the range of 0 to 1λ.

[0070] S202, obtaining a pupil function by Fresnel diffraction calculation according to the wave aberration.

[0071] S203, performing Fourier transform on the pupil function to obtain an amplitude point spread function.

[0072] The pupil function describes the complex amplitude distribution of light in an optical system.

[0073] S204, obtaining an intensity point spread function of an incoherent imaging system by taking the square of the modulus of the amplitude point spread function.

[0074] S205, performing aberration-free image degradation processing by using an intensity point spread function.

[0075] Specifically, the problem of detecting aberrations is transformed into the problem of estimating the polynomial coefficients of the PSF at each pixel position, which also includes: constructing a PSF model for each pixel position in the space-variant system according to the pupil function and the amplitude point spread function. The model is as follows:

[0076]

[0077] Among them, A(x,y) is the aperture shape function, W(x,y) is the polynomial coefficient matrix, and PSF is the point spread function.

[0078] S103, pairing multiple groups of historical blurred images and historical aberration coefficient maps to construct a training data set.

[0079] S104, taking the historical blurred image as the input of the model, and the corresponding historical aberration coefficient map as the output of the model, setting the backbone network to HRNet, and performing model training.

[0080] S105, using the mean square error to measure the difference between the predicted aberration coefficient map and the true value.

[0081] S106, using the Adam optimization algorithm to perform gradient descent, adjust network parameters, minimize the loss function, and obtain the aberration regression network model.

[0082] S3, inputting the real-time random noise image and the real-time aberration coefficient map into a space-variant degradation model, and diffusing the pixel value of each pixel position according to the real-time aberration coefficient map to generate a real-time degraded image.

[0083] See also Figure 4 , is a schematic diagram of a flow chart of constructing a space-variant degradation model provided by an embodiment of the present invention; a space-variant degradation model is constructed to optimize online a clear image process with aberration removed. Specifically, the space-variant degradation model is:

[0084]

[0085] Among them, S(x,y) is the item to be optimized, B(x',y') is the blurred image after degradation, η is the image noise, and PSF(x'-x,y'-y) is the point spread function of the current pixel position.

[0086] The space-variant degradation model mainly includes the process of input, energy diffusion and back-propagation optimization. Specifically, the input is: Random noise image: This is an image with random noise collected by the system. Aberration coefficient map: This is the aberration coefficient map predicted by the space-variant aberration regression network (as mentioned above, the backbone network is HRNet), which is used to describe the spatially varying point spread function (PSF) in the image.

[0087] Energy diffusion: Based on the aberration coefficient map: The space-variant system degradation model uses the aberration coefficient map to guide the energy diffusion at each pixel position. The aberration coefficient at each pixel position determines the point spread function (PSF) at that position, which affects how the energy diffuses in the image. Energy diffusion: The model diffuses the pixel values ​​at each pixel position based on the aberration coefficient map to generate a degraded image.

[0088] Back propagation and optimization: Difference calculation: Calculate the pixel value difference (E) between the degraded image and the blurred image collected by the system. Back propagation: According to the difference (E), update the model parameters through the back propagation algorithm to reduce the difference. Iterative optimization: Repeat the above steps until the difference (E) is less than the preset threshold, then exit the iterative optimization.

[0089] S4, obtain the real-time blurred image.

[0090] S5, obtaining a pixel value difference between the real-time blurred image and the real-time degraded image according to the real-time blurred image and the real-time degraded image.

[0091] S6, according to the pixel value difference, update and iterate through the back propagation algorithm until the pixel value difference is less than the threshold, and output the optimized image.

[0092] Specifically, the threshold is set to 0.1, and the maximum number of iterations is 200.

[0093] In the iterative optimization process, the optimization item S(x,y) is first degraded according to the historical aberration coefficient map obtained by the aberration regression network model, and the difference E between the pixel values ​​of the degraded image B(x′,y′) and the blurred image B(x,y) collected by the system is back-propagated to update the optimization item S(x,y) until the difference E is less than the threshold threshold. 1 Exit iterative optimization when .

[0094] See also Figure 5 , is a framework structure diagram of a system for eliminating wavefront aberration based on space-varying point spread function provided by an embodiment of the present invention; a second aspect of an embodiment of the present invention provides a system for eliminating wavefront aberration based on space-varying point spread function, which is applied to the method for eliminating wavefront aberration based on space-varying point spread function described in the first aspect, comprising:

[0095] The acquisition module is used to acquire a real-time random noise image and a real-time blurred image of a space-variant system, wherein the space-variant system is a system in which a point spread function varies with pixel positions.

[0096] The aberration regression network module is used to input the real-time blurred image into the aberration regression network model to obtain a real-time aberration coefficient map. The aberration regression network model is obtained by training the historical blurred images and historical aberration coefficient maps of the space-variable system.

[0097] The space-variant degradation module is used to input the real-time random noise image and the real-time aberration coefficient map into the space-variant degradation model, and diffuse the pixel value of each pixel position according to the real-time aberration coefficient map to generate a real-time degraded image.

[0098] The real-time blurred image module is used to obtain the real-time blurred image.

[0099] The aberration module is used to obtain the pixel value difference between the real-time blurred image and the real-time degraded image according to the real-time blurred image and the real-time degraded image.

[0100] The output module is used to update and iterate according to the pixel value difference through the back propagation algorithm until the pixel value difference is less than the threshold value, and output the optimized image.

[0101] It can be seen that the method and system for eliminating wavefront aberration based on space-varying point spread function provided by the embodiments of the present invention can effectively cope with the spatial variability under various complex imaging conditions while maintaining sufficient computational efficiency.

[0102] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

Claims

1. A method for eliminating wavefront aberration based on space-varying point spread function, characterized in that: include: Acquire a real-time random noise image and a real-time blurred image of a space-variable system, wherein the space-variable system is a system in which a point spread function varies with pixel position; Inputting the real-time blurred image into an aberration regression network model to obtain a real-time aberration coefficient map, wherein the aberration regression network model is obtained by training historical blurred images and historical aberration coefficient maps of a space-variable system; Inputting the real-time random noise image and the real-time aberration coefficient map into a space-varying degradation model, and diffusing the pixel value of each pixel position according to the real-time aberration coefficient map to generate a real-time degraded image; Get real-time blurred images; According to the real-time blurred image and the real-time degraded image, a pixel value difference between the real-time blurred image and the real-time degraded image is obtained; According to the pixel value difference, the back propagation algorithm is used to update and iterate until the pixel value difference is less than the threshold, and the optimized image is output.

2. The method for eliminating wavefront aberration based on space-varying point spread function according to claim 1, characterized in that: The aberration regression network model is obtained by training the historical blurred images and historical aberration coefficient maps of the space-variant system, including: Obtain the historical wave aberration of the space-variant system; Simulating the historical wave aberration to generate a historical blurred image and a historical aberration coefficient map; Pair multiple groups of historical blurred images and historical aberration coefficient maps to construct a training data set; The historical blurred image is used as the input of the model, the corresponding historical aberration coefficient map is used as the output of the model, the backbone network is set to HRNet, and the model training is performed; The mean square error is used to measure the difference between the predicted aberration coefficient map and the true value; The Adam optimization algorithm is used for gradient descent, the network parameters are adjusted, the loss function is minimized, and the aberration regression network model is obtained.

3. The method for eliminating wavefront aberration based on space-varying point spread function according to claim 2, characterized in that: The historical wave aberration is simulated to generate a historical blurred image and a historical aberration coefficient map, including: Use Zernike polynomials to represent wave aberrations; According to the wave aberration, the pupil function is calculated by Fresnel diffraction; Perform Fourier transform on the pupil function to obtain the amplitude point spread function; The intensity point spread function of the incoherent imaging system is obtained by taking the square of the modulus of the amplitude point spread function; Aberration-free image degradation via intensity point spread function.

4. The method for eliminating wavefront aberration based on space-varying point spread function according to claim 2, characterized in that: The wave aberration of the space-variable system is set in the range of 0 to 1λ.

5. The method for eliminating wavefront aberration based on space-varying point spread function according to claim 2, characterized in that: Also includes: According to the pupil function and the amplitude point spread function, the PSF model of each pixel position in the space-variant system is constructed as follows: Among them, A(x,y) is the aperture shape function, W(x,y) is the polynomial coefficient matrix, and PSF is the point spread function.

6. The method for eliminating wavefront aberration based on space-varying point spread function according to claim 1, characterized in that: The space-variant degradation model is: Among them, S(x,y) is the item to be optimized, B(x',y') is the blurred image after degradation, η is the image noise, and PSF(x'-x,y'-y) is the point spread function of the current pixel position.

7. The method for eliminating wavefront aberration based on space-varying point spread function according to claim 1, characterized in that: The threshold is set to 0.1 and the maximum number of iterations is 200.

8. A system for eliminating wavefront aberration based on space-varying point spread function, applied to the method for eliminating wavefront aberration based on space-varying point spread function according to any one of claims 1 to 7, characterized in that: include: An acquisition module, used to acquire a real-time random noise image and a real-time blurred image of a space-variable system, wherein the space-variable system is a system in which a point spread function varies with pixel positions; An aberration regression network module is used to input the real-time blurred image into an aberration regression network model to obtain a real-time aberration coefficient map, wherein the aberration regression network model is obtained by training the historical blurred images and historical aberration coefficient maps of the space-variable system; A space-variant degradation module, used for inputting the real-time random noise image and the real-time aberration coefficient map into the space-variant degradation model, and diffusing the pixel value of each pixel position according to the real-time aberration coefficient map to generate a real-time degraded image; A real-time blurred image module, used for obtaining a real-time blurred image; An aberration module, used for obtaining a pixel value difference between the real-time blurred image and the real-time degraded image according to the real-time blurred image and the real-time degraded image; The output module is used to update and iterate according to the pixel value difference through the back propagation algorithm until the pixel value difference is less than the threshold value, and output the optimized image.

Images