An optical degradation image restoration method of parametric point spread function regression retrieval

Abstract

This invention provides a method for optically degraded image restoration using parametric point spread function regression retrieval. The method includes: extracting low-dimensional parametric descriptors for each reference point spread function in a reference point spread function library of the target imaging system; inputting the local patch and field-of-view coordinates of the degraded image into a point spread function prediction network to obtain a predicted descriptor corresponding to the local patch; obtaining a target point spread function matching the predicted descriptor in the reference point spread function library through nearest neighbor retrieval; inputting the target point spread function into a frequency-spatial joint restoration network to perform coarse frequency-domain restoration and fine spatial-domain restoration of the local patch of the degraded image, outputting the final clear image. This invention can improve retrieval efficiency, ensure the physical realizability of point spread functions, improve the modeling accuracy and adaptability for spatially variable degradation, and enhance the edge restoration quality and detail preservation of the final clear image.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method for restoring optically degraded images using parametric point spread function regression retrieval. Background Technology

[0002] In incoherent optical imaging systems, image degradation typically manifests as the convolution of a sharp image with the system's point spread function (PSF), coupled with added noise. For practical optical systems such as single-lens, wide-angle, microscopic, and remote sensing systems, the PSF often exhibits significant spatial variability under different field-of-view positions and imaging conditions due to the influence of off-axis aberrations, focal plane position, object distance variations, aperture conditions, and wavelength conditions. This spatial variability blurring leads to image edge diffusion, loss of detail, and severe contrast degradation, significantly reducing the system's target recognition rate and measurement accuracy based on its modulation transfer function (MTF).

[0003] Traditional solutions for spatially variable blurring can be broadly categorized into blind kernel estimation methods, direct high-dimensional kernel prediction methods, and pure end-to-end image restoration networks. Blind kernel estimation methods, exemplified by non-uniform camera shake deblurring, primarily rely on the image's internal statistical features or self-similar priors, making it difficult to accurately characterize the systematic degradation caused by off-axis aberrations and changes in field of view. Methods utilizing deep learning to directly predict high-dimensional PSF kernels or aberration parameters (such as Multi-scale Attention Networks (MANet) and Spatial-Frequency Enhancement Networks (SFE-Net)) suffer from industry bottlenecks such as excessively high output dimensionality, complex network training, and extremely high computational costs. Furthermore, in the absence of sufficient physical constraints, their predictions tend to deviate from the achievable physical constraints of real optical systems. Additionally, purely data-driven end-to-end image restoration networks (such as Restormer, Nonlinear Activation Free Networks (NAFNet), and Residual Denoising Diffusion models) also fall into these categories. Models (RDDM, etc.) only learn the statistical mapping from degraded images to clear images, failing to explicitly utilize the existing physical causes and degradation priors of the imaging system. When faced with complex field-dependent non-uniform blur and real noise conditions, the interpretability and generalization stability of the recovery results are significantly insufficient.

[0004] In summary, image restoration schemes in related technologies generally face deeply intertwined technical bottlenecks, including the lack of physical constraints in blind estimation, the lack of flexibility in fixed PSFs, and the lack of interpretability in end-to-end models. When dealing with full-field-of-view restoration, directly matching, storing, or retrieving using pixel-domain high-dimensional PSFs incurs significant storage and retrieval overhead, hindering efficient implementation in practical engineering applications. Therefore, how to construct a unified image restoration framework that balances physical interpretability, retrieval efficiency, and restoration stability while preserving spatial variability and ensuring the prediction kernel meets real physical realizability has become an urgent technical problem to be solved. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a method for optically degraded image restoration using parametric point spread function regression retrieval. The technical problem to be solved by this invention is achieved through the following technical solution: In a first aspect, the present invention provides a method for optically degraded image restoration using parametric point spread function regression retrieval, comprising: A reference point spread function library for the target imaging system is constructed, and a low-dimensional parameterized descriptor is extracted for each reference point spread function in the reference point spread function library; Obtain a local patch of the degraded image and its field of view coordinates. Input the local patch of the degraded image and its field of view coordinates into a point spread function prediction network. Output the prediction descriptor corresponding to the local patch of the degraded image through the point spread function prediction network. In the reference point diffusion function library, the distance between the predicted descriptor and each of the parameterized descriptors is compared, and the target point diffusion function that matches the predicted descriptor is obtained through nearest neighbor retrieval; The target point diffusion function is input into the frequency-spatial joint restoration network. The frequency-spatial joint restoration network performs coarse frequency-domain restoration and fine spatial restoration on local blocks of the degraded image, and outputs the final clear image.

[0006] In one embodiment of the present invention, the reference point spread function library for constructing the target imaging system includes: Discrete sampling is performed on the target imaging system within the effective working domain to obtain the spread function of each reference point, and the reference point spread function library is constructed based on each reference point spread function; the effective working domain includes at least the field of view position and the object distance.

[0007] In one embodiment of the present invention, the point spread function prediction network includes an image branch and a coordinate branch. The input feature map of the image branch is composed of the spatial domain image of the local block of the degraded image, the frequency domain amplitude image after fast Fourier transform, and the spectral amplitude image after logarithmic compression. The coordinate branch encodes the field of view coordinates into a learnable embedding vector; The step of outputting the predicted descriptor corresponding to the local patch of the degraded image through the point spread function prediction network includes: The point spread function prediction network fuses the input feature map of the image branch with the embedding vector of the coordinate branch, and outputs the predicted descriptor through the regression head.

[0008] In one embodiment of the present invention, the step of comparing the distance between the predicted descriptor and each of the parameterized descriptors in the reference point diffusion function library, and obtaining the target point diffusion function matching the predicted descriptor through nearest neighbor retrieval, includes: In the reference point diffusion function library, the distance between the predicted descriptor and each of the parameterized descriptors is calculated; The physically realizable point spread function corresponding to the parameterized descriptor that has the smallest distance from the current predicted descriptor is used as the target point spread function.

[0009] In one embodiment of the present invention, the frequency domain coarse recovery includes: Based on the frequency domain form of the local blocks of the degraded image and the spread function components of the target point, a Wiener-like deconvolution with a learnable regularization term is performed in the frequency domain to obtain a coarse frequency domain restoration result.

[0010] In one embodiment of the present invention, the spatial refinement and restoration includes: The frequency domain coarse recovery result is subjected to inverse Fourier transform to obtain the spatial domain coarse recovery result; The spatial domain coarse restoration result is concatenated with the local block of the degraded image in the channel dimension to obtain the concatenated feature; The splicing features are input into a spatial domain refinement network, and the splicing features are processed by the backbone network of the spatial domain refinement network, which has residual downsampling blocks, skip connections, and upsampling decoding structures.

[0011] In one embodiment of the present invention, the parameterized descriptor is a three-parameter descriptor, the three-parameter descriptor comprising: Steller ratio, which characterizes the degree to which the center peak of the actual point spread function is preserved relative to the diffraction-limited point spread function; The energy radius is surrounded by a preset proportion representing the degree to which the energy of the point diffusion function diffuses outward; Full width at half maximum (FWHM) characterizes the degree of main lobe broadening of the point spread function. After extracting the low-dimensional parameterized descriptor, the method further includes: Unit energy normalization is performed on the diffusion function of each reference point, and min-max normalization is performed on the parameterized descriptor of each parameterized descriptor.

[0012] In a second aspect, the present invention provides an optically degraded image restoration apparatus based on parametric point spread function regression retrieval, comprising: The feature extraction module is used to construct a reference point spread function library for the target imaging system and extract a low-dimensional parameterized descriptor for each reference point spread function in the reference point spread function library. The prediction output module is used to obtain a local patch of the degraded image and the field coordinates of the local patch, input the local patch of the degraded image and the field coordinates of the field to the point spread function prediction network, and output the prediction descriptor corresponding to the local patch of the degraded image through the point spread function prediction network; The prior retrieval module is used to compare the distance between the predicted descriptor and each parameterized descriptor in the reference point diffusion function library, and obtain the target point diffusion function that matches the predicted descriptor through nearest neighbor retrieval; The joint restoration module is used to input the target point diffusion function into the frequency domain-spatial domain joint restoration network, and to perform coarse frequency domain restoration and fine spatial domain restoration on the local blocks of the degraded image through the frequency domain-spatial domain joint restoration network, and output the final clear image.

[0013] Thirdly, the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the steps of the optical degraded image restoration method using parameterized point spread function regression retrieval provided in the first aspect.

[0014] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the optical degradation image restoration method by parameterized point spread function regression retrieval provided in the first aspect.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The optical degradation image restoration method based on parametric point spread function regression retrieval provided by this invention constructs a reference point spread function library for the target imaging system and extracts a low-dimensional parametric descriptor for each reference point spread function in the library. It then obtains a local patch of the degradation image and its field-of-view coordinates, inputs these coordinates into a point spread function prediction network, and outputs a predicted descriptor corresponding to the local patch. In the reference point spread function library, the predicted descriptor is compared with each parametric descriptor, and a target point spread function matching the predicted descriptor is obtained through nearest neighbor retrieval. Finally, the target point spread function is input into a frequency-spatial joint restoration network, which performs coarse frequency-domain restoration and fine spatial-domain restoration of the local patch of the degradation image, outputting a final clear image. On the one hand, extracting low-dimensional parameterized descriptors from the reference point spread function and using the regressed predicted descriptors to perform nearest neighbor retrieval in the reference point spread function library can reduce prediction complexity, thereby improving retrieval efficiency and ensuring the physical realizability of the point spread function. On the other hand, inputting local patches of the degraded image and field coordinates into the point spread function prediction network, which integrates field coordinates and multi-domain features, can improve the modeling accuracy and adaptability for spatial variation degradation. Furthermore, by using a frequency-spatial joint restoration network to perform coarse frequency domain restoration and fine spatial domain restoration on local patches of the degraded image, the combination of coarse frequency domain restoration and fine spatial domain restoration improves the edge restoration quality and detail preservation capability of the final clear image while ensuring physical interpretability.

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] Figure 1 This is a schematic diagram illustrating the implementation process of an optically degraded image restoration method based on parametric point spread function regression retrieval provided in an embodiment of the present invention; Figure 2 This is a schematic diagram comparing point spread function prediction results provided in an embodiment of the present invention; Figure 3 This is a schematic diagram comparing the results of degraded image restoration provided in an embodiment of the present invention; Figure 4 This is a schematic diagram comparing real image restoration results and a modulation transfer function curve provided by an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an optically degraded image restoration device using parametric point spread function regression retrieval provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the hardware entity of an electronic device to which embodiments of the present invention are applied. Detailed Implementation

[0018] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0019] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating the implementation process of an optically degraded image restoration method based on parametric point spread function regression retrieval provided in an embodiment of the present invention. Figure 1 As shown, the optically degraded image restoration method based on parametric point spread function regression retrieval provided in this embodiment of the invention may include the following steps: Step S101: Construct a reference point spread function library for the target imaging system, and extract a low-dimensional parameterized descriptor for each reference point spread function in the reference point spread function library.

[0020] In one possible implementation, the parameterized descriptor is a three-parameter descriptor, which includes: the Strehl ratio (SR), which characterizes the degree to which the actual point spread function retains the central peak relative to the diffraction-limited point spread function; the energy radius, which is a preset proportion, characterizes the degree to which the energy of the point spread function diffuses outward; and the full width at half maximum (FWHM), which characterizes the degree to which the main lobe of the point spread function broadens. Taking the preset proportion as 90% as an example, the mathematical expression of the three-parameter descriptor is shown in the following formula (1): (1); in, It is a three-parameter descriptor. Strelby ratio is used to characterize the degree to which the central peak of the actual point spread function is preserved relative to the diffraction-limited point spread function, and can reflect the peak attenuation caused by aberrations. The energy radius is 90%, used to characterize the degree to which the energy of the point spread function diffuses outwards. The full width at half maximum (FWHM) is used to characterize the broadening of the main lobe of the point spread function.

[0021] The Strelby ratio in the three-parameter descriptor can be determined by the following formula (2): (2); in, For Strelby, Let be the actual point spread function. is the diffraction-limiting point diffusion function.

[0022] The 90% enclosing energy radius in the three-parameter descriptor can be determined by the following formula (3): (3); in, The energy radius is 90% of the surrounding area. The parameter is The point diffusion energy function, The total energy diffused at a point.

[0023] The full width at half maximum (FWHM) of a three-parameter descriptor can be determined by the following formula (4): (4); in, It is half the height and full width. The horizontal spacing is half the height and full width. The vertical spacing is half the height and full width.

[0024] After extracting the low-dimensional parameterized descriptors, unit energy normalization can be performed on the diffusion function at each reference point, and min-max normalization can be performed on each parameterized descriptor. For example, min-max normalization can be performed on each parameterized descriptor using the following formula (5): (5); in, For the extracted parameterized descriptors, For the normalized parameterized descriptor, and These are the minimum and maximum values ​​for each descriptive dimension in the reference point diffusion function library, respectively.

[0025] In one possible implementation, the target imaging system can be discretely sampled within the effective working domain to obtain the spread function of each reference point, and a reference point spread function library can be constructed based on the spread functions of each reference point. The effective working domain includes at least the field of view position and the object distance. The field of view position and object distance can be extended to wavelength, focal plane position, aperture state, or other system state variables.

[0026] For example, during the discrete sampling process of a target imaging system within its effective working domain, a higher sampling density can be used in regions where the state parameters change rapidly, and a lower sampling density can be used in regions where the state parameters change slowly, thereby controlling the size of the reference point spread function library while meeting the recovery accuracy requirements.

[0027] Step S102: Obtain the local patch of the degraded image and its field of view coordinates. Input the local patch of the degraded image and its field of view coordinates into the point spread function prediction network. Output the prediction descriptor corresponding to the local patch of the degraded image through the point spread function prediction network.

[0028] In one possible implementation, the Point Spread Function Prediction Network (PSF-PNet) includes an image branch and a coordinate branch. The input feature map of the image branch is composed of the spatial domain image of the local patch of the degraded image, the frequency domain amplitude image after fast Fourier transform, and the spectral amplitude image after logarithmic compression. The coordinate branch encodes the field coordinates into learnable embedding vectors.

[0029] For example, the input feature map of the image branch can be determined by the following formula (6): (6); in, The input feature map for the image branch. For the spatial domain image of local patches of the degraded image, This is a spectrum amplitude image.

[0030] In one possible implementation, the input feature map of the image branch and the embedding vector of the coordinate branch can be fused through a point spread function prediction network, and the predicted descriptor can be output through a regression head. For example, the predicted descriptor can be determined by the following formula (7): (7); in, For predicting descriptors, For the regression head function, The input feature map for the image branch, For coordinate branches, Embedded vector fusion for coordinate branches.

[0031] Step S103: In the reference point diffusion function library, compare the distance between the predicted descriptor and each parameterized descriptor, and obtain the target point diffusion function that matches the predicted descriptor through nearest neighbor retrieval.

[0032] In one possible implementation, the distance between the predicted descriptor and each parameterized descriptor can be calculated from the reference point spread function library; the physically realizable point spread function corresponding to the parameterized descriptor with the smallest distance to the current predicted descriptor is taken as the target point spread function.

[0033] For example, the target point spread function can be determined by the following formula (8): (8); in, Let be the target point spread function. The normalized prediction descriptor Reference point diffusion function library, The first reference point diffusion function in the library The normalized prediction descriptor corresponding to the point spread function.

[0034] During the descriptor regression process of the training point spread function prediction network, the regression loss can be determined by the following formula (9): (9); in, To regress the loss, For predicting descriptors, These are the extracted parameterized descriptors. It's understandable that the smaller the regression loss, the closer the regression result is to the descriptors needed for subsequent retrieval.

[0035] Step S104: Input the target point diffusion function into the Frequency-Spatial Fusion Network (FSFNet). The FSFNet performs coarse frequency domain restoration and fine spatial domain restoration on local blocks of the degraded image, and outputs the final clear image.

[0036] For example, the spatial domain relationship between a degraded image local patch, the original sharp image without optical degradation and noise contamination, the target point spread function, and noise can be expressed as the following formula (10): (10); After performing a frequency domain transformation on equation (10), the corresponding frequency domain expression can be obtained, as shown in equation (11): (11); in, For local patches of degraded image, For the original clear image, Let be the target point spread function. For noise, To degrade the form of local patches in the frequency domain of an image, The original sharp image in the frequency domain. The target point spread function is in the frequency domain form. This represents the form of noise in the frequency domain.

[0037] In one possible implementation, coarse frequency domain restoration includes: performing a Wiener-like deconvolution with a learnable regularization term in the frequency domain based on the frequency domain form of the local patch of the degraded image and the spread function component of the target point, to obtain the coarse frequency domain restoration result. The coarse frequency domain restoration result can be determined by the following formula (12): (12); in, This is the result of coarse frequency domain recovery. The target point spread function is in the frequency domain form. To degrade the form of local patches in the frequency domain of an image, This is a learnable regularization term used to suppress direct inverse filtering instabilities near spectral zeros and reduce ringing artifacts. This is a user-defined, minimal constant.

[0038] The learnable regularization term can be determined by the following formula (13): (13); in, For learnable regular terms, To make the regularized prediction network lightweight, The target point spread function is in the frequency domain form. This represents the frequency domain form of local blocks in a degraded image.

[0039] In one possible implementation, the spatial domain refinement recovery includes: performing an inverse Fourier transform on the frequency domain coarse recovery result to obtain the spatial domain coarse recovery result, as shown in formula (14): (14); in, This is the result of coarse spatial domain reconstruction. This is the result of coarse frequency domain recovery.

[0040] The coarse spatial domain reconstruction result can be stitched together with local patches of the degraded image along the channel dimension to obtain stitched features. These stitched features are then input into a spatial domain thinning network. The backbone network within this network, which incorporates residual downsampling blocks, skip connections, and upsampling decoding structures, processes the stitched features to output the final, sharp image. The spatial domain thinning network can employ a ResUNet-style backbone network with residual downsampling blocks, skip connections, and upsampling decoding structures. Furthermore, multi-scale convolutional branches and coordinate-guided gating modules can be introduced to adapt to variations in the point spread function scale and degradation level at different viewpoint locations.

[0041] In one possible implementation, the target optical imaging system corresponding to the image to be recovered and its effective working domain can be determined, including lens parameters, sensor pixel size, working band, aperture state, focal plane position range, and object distance range. A state domain, including at least the spatial variable of field of view position, is defined according to application requirements, further incorporating object distance, focal plane offset, wavelength, or other system state variables. Based on this, reference point spread function (PFS) samples are collected or generated, and a reference PFS library is constructed. This can be obtained through experimental calibration, simulation using optical design software, or a hybrid approach of "experimental + simulation" to obtain PFS samples under different field of view positions and different state variables. The sampling density is increased in parameter regions where the PFS changes rapidly, and relatively sparse sampling is used in regions where the PFS changes slowly, so that the reference PFS library more closely covers the effective family of continuous PFS. Subsequently, the reference PFS is parameterized by first normalizing each PFS to unit energy, and then calculating its... , and Thus, the parameterized descriptor is obtained. Furthermore, normalization is performed on all parameterized descriptors in the reference point diffusion function library to obtain a normalized set of parameterized descriptors. .

[0042] Then, degraded training samples are constructed and a dataset is established. Original clear image samples can be selected, and local convolutional degradation is performed on the images according to the point spread function corresponding to different field-of-view positions or different states in the reference point spread function library. Spatial variability degradation is simulated using block convolution or position-related convolution. Subsequently, a noise model matched to the target sensor is superimposed, including Poisson photon noise, Gaussian readout noise, dark current noise, and quantization noise, to form degraded data that more closely resembles real imaging conditions. During the training phase of the point spread function prediction network PSF-PNet, local blocks of degraded images are... With corresponding field of view coordinates As joint inputs, the image branch preferably receives a spatial domain image and its spectral amplitude correlation input, while the coordinate branch preferably receives discrete index or continuous coordinate encoding; through supervised learning, the network outputs a predicted descriptor. And by applying error constraints to the true descriptor, the mapping relationship from degenerate local observations and spatial locations to low-dimensional point spread function descriptors is learned. For the predicted descriptor... The system searches for the parameterized descriptor with the smallest distance from the reference point spread function library and returns the corresponding target point spread function. It can use nearest neighbor retrieval, k-nearest neighbor weighted retrieval, or interpolation retrieval to avoid the network directly outputting a high-dimensional kernel that is not physically constrained.

[0043] During the training phase of the frequency-spatial joint restoration network, the target point spread function obtained above is compared with the local patch of the degraded image. They are sent together into the recovery link; first, in the frequency domain, based on local blocks of the degraded image Performing a Wiener-like deconvolution with a learnable regularization term on the spread function of the retrieved target points yields a coarse spatial domain recovery result. Then With degraded image local patches The input spatial domain is spliced ​​to refine the network, and the final clear image is output. In the early stage of training, a certain degree of mixing of the retrieval results with the real point spread function is introduced to improve the training stability; in the recovery training stage, a loss function combining pixel reconstruction term and structural similarity term can be used, and the form of the loss function is shown in Equation (15): (15); in, and These are the weighting coefficients. For the final clear image, The original, clear image is used. This loss is used to simultaneously constrain pixel consistency and structure preservation capability. Finally, the method's effectiveness is evaluated and its deployment optimized. For simulated data, metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), and Kernel PSNR can be used to evaluate the descriptor retrieval and recovery performance; for real-world data, metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), and Kernel PSNR can be used. The spatial frequency value corresponding to the MTF curve dropping to 50% of the maximum contrast, edge sharpness, and subjective visual quality are evaluated. During engineering deployment, the size of the reference point spread function library, retrieval strategy, network depth, or block size can be pruned and accelerated according to hardware computing power requirements. Through the above implementation process, a complete implementation chain can be formed from "target system calibration or simulation library construction" to "descriptor regression prediction," then to "physically constrained point spread function retrieval" and "frequency-spatial joint restoration," thereby achieving image restoration that balances accuracy, stability, and interpretability in spatially variable optical degradation scenarios.

[0044] In one possible implementation, a single-lens fixed-focal-length imaging system can be used as the verification platform. This system can consist of a Basler a2A2590-60umPRO camera and a GCL-010109N single lens with a focal length of 50.8mm and an aperture of 25.4mm, operating in the visible light band. In one embodiment, calibration and library construction can be performed within an object distance range of 1000mm to 3500mm, and at each object distance, [the following is likely a separate, unrelated sentence:] ... A field-of-view sampling strategy is employed to construct a reference PSF distribution covering the entire field of view. (See attached image.) Figure 4 As shown, the PSF of this type of imaging system varies significantly with the field of view, making it suitable as a verification platform for the embodiment of the spatially variable optical degradation image restoration method of the present invention. Furthermore, block-wise convolutional degradation can be performed on the clear image based on the reference PSF, and Poisson photon noise, readout noise, dark current noise, and quantization noise can be superimposed to generate degradation data that approximates the imaging conditions of a real complementary metal-oxide-semiconductor (CMOS). In one embodiment, the sample size can be set to 20,000 degradation images, divided into a training set, a validation set, and a test set in an 8:1:1 ratio.

[0045] In the verification of PSF parameterization and regression retrieval performance, the method in this invention can be compared with methods such as Kernel Generative Adversarial Network (KernelGAN), Local Motion Guidance (LMG), MANet, and SFE-Net. Figure 2 The diagram shows a blurred image (corresponding to a degraded image local patch in the above implementation), a PSF distribution map obtained through KernelGAN, a PSF distribution map obtained through LMG, a PSF distribution map obtained through MANet, a PSF distribution map obtained through SFE-Net, a PSF distribution map obtained through the method of this invention, and a true image (the PSF distribution map used when simulating the blurred image). For the field-dependent PSF prediction task, the PSF obtained by this invention through descriptor regression and reference point spread function library retrieval is closer to the true PSF. The corresponding quantization results show that this invention can achieve a KernelPSNR of 36.26 dB and an SSIM of 0.8713 on the simulated PSF prediction task, with a computational cost of approximately 2.52 GFLOPs. This indicates that this invention can significantly reduce the computational complexity caused by direct high-dimensional kernel prediction while maintaining high prediction accuracy.

[0046] In verifying the simulated image restoration effect, the method of this invention can be compared with methods such as LMG, Restormer, NAFNet, and RDDM. Figure 3The diagram shows a ground truth image (corresponding to the original sharp image in the above implementation), a degraded image (corresponding to a local patch of the degraded image in the above implementation), an image obtained through LMG, an image obtained through Restormer, an image obtained through NAFNet, an image obtained through RDDM, and an image obtained through the method of this invention. It is evident that the method of this invention exhibits superior overall performance in edge restoration, texture preservation, and artifact suppression. Corresponding quantization results show that this invention achieves a PSNR of 22.43 dB and an SSIM of 0.6713 in simulated restoration tasks, representing improvements of 31.71% and 33.94% respectively compared to the direct imaging results of the target optical system.

[0047] In real-world image reconstruction verification, the trained network parameters can be directly applied to a target imaging system consistent with the simulation without additional fine-tuning. Since it is difficult to obtain ideal, sharp ground truth values ​​in real-world scenes, MTF and [other methods] are preferred. As a primary objective evaluation indicator. For example... Figure 4 As shown, Figure 4 Figure (a) shows a direct image taken by the target imaging system. Images obtained through LMG Images obtained through Restormer Images obtained through NAFNet Images obtained through RDDM and the image obtained by the method in this invention. . Figure 4 Figure (b) shows a graph of spatial frequency values ​​versus MTF. It can be seen that this invention, after obtaining a final clear image, can test the system... The value improved from 0.0230 in the original acquired image to 0.1133, and this improvement was more significant in the off-axis region, indicating that the retrieved field-dependent PSF prior can effectively adapt to the spatial variability degradation in real optical systems.

[0048] The optical degradation image restoration method based on parametric point spread function regression retrieval provided by this invention constructs a reference point spread function library for the target imaging system and extracts a low-dimensional parametric descriptor for each reference point spread function in the library. It then obtains a local patch of the degradation image and its field-of-view coordinates, inputs these coordinates into a point spread function prediction network, and outputs a predicted descriptor corresponding to the local patch. In the reference point spread function library, the predicted descriptor is compared with each parametric descriptor, and a target point spread function matching the predicted descriptor is obtained through nearest neighbor retrieval. Finally, the target point spread function is input into a frequency-spatial joint restoration network, which performs coarse frequency-domain restoration and fine spatial-domain restoration of the local patch of the degradation image, outputting a final clear image. On the one hand, extracting low-dimensional parameterized descriptors from the reference point spread function and using the regressed predicted descriptors to perform nearest neighbor retrieval in the reference point spread function library can reduce prediction complexity, thereby improving retrieval efficiency and ensuring the physical realizability of the point spread function. On the other hand, inputting local patches of the degraded image and field coordinates into the point spread function prediction network, which integrates field coordinates and multi-domain features, can improve the modeling accuracy and adaptability for spatial variation degradation. Furthermore, by using a frequency-spatial joint restoration network to perform coarse frequency domain restoration and fine spatial domain restoration on local patches of the degraded image, the combination of coarse frequency domain restoration and fine spatial domain restoration improves the edge restoration quality and detail preservation capability of the final clear image while ensuring physical interpretability.

[0049] Please see Figure 5 , Figure 5 This is a schematic diagram of an optical image restoration device using parametric point spread function regression retrieval provided in an embodiment of the present invention. The optical image restoration device using parametric point spread function regression retrieval provided by the present invention includes a feature extraction module 501, a prediction output module 502, a priori retrieval module 503, and a joint restoration module 504. Wherein: The feature extraction module 501 is used to construct a reference point spread function library for the target imaging system and extract a low-dimensional parameterized descriptor for each reference point spread function in the reference point spread function library. The prediction output module 502 is used to obtain the local patch of the degraded image and the field coordinates of the local patch of the degraded image, input the local patch of the degraded image and the field coordinates of the field to the point spread function prediction network, and output the prediction descriptor corresponding to the local patch of the degraded image through the point spread function prediction network. The prior retrieval module 503 is used to compare the distance between the predicted descriptor and each parameterized descriptor in the reference point diffusion function library, and obtain the target point diffusion function that matches the predicted descriptor through nearest neighbor retrieval. The joint restoration module 504 is used to input the target point diffusion function into the frequency domain-spatial domain joint restoration network, and to perform coarse frequency domain restoration and fine spatial domain restoration on local blocks of the degraded image through the frequency domain-spatial domain joint restoration network, and output the final clear image.

[0050] Please see Figure 6 , Figure 6 This is a schematic diagram of the hardware entity of an electronic device to which an embodiment of the present invention applies. The electronic device provided in this embodiment includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other via the communication bus 604. Memory 603 is used to store computer programs; When the processor 601 executes the program stored in the memory 603, it implements the steps in the above method embodiments.

[0051] The communication bus 604 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 604 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus.

[0052] The communication interface 602 is used for communication between the above-mentioned electronic device and other devices.

[0053] The memory 603 may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory 603 may also be at least one storage device located remotely from the aforementioned processor.

[0054] The processor 601 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0055] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0056] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps provided in the above-described method embodiments.

[0057] For the embodiments of the device / electronic device / storage medium, since they are basically similar to the method embodiments, the description is relatively simple. For specific details and beneficial effects, please refer to the description of the method embodiments.

[0058] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0059] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0060] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects, all of which are collectively referred to herein as "modules" or "systems." Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The computer program may be stored / distributed in a suitable medium, provided with or as part of other hardware, or may take other distribution forms, such as via the Internet or other wired or wireless telecommunications systems.

[0061] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0062] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0064] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method of optical degraded image restoration by parametric point spread function regression retrieval, characterized in that, include: A reference point spread function library for the target imaging system is constructed, and a low-dimensional parameterized descriptor is extracted for each reference point spread function in the reference point spread function library; Obtain a local patch of the degraded image and its field of view coordinates. Input the local patch of the degraded image and its field of view coordinates into a point spread function prediction network. Output the prediction descriptor corresponding to the local patch of the degraded image through the point spread function prediction network. In the reference point diffusion function library, the distance between the predicted descriptor and each of the parameterized descriptors is compared, and the target point diffusion function that matches the predicted descriptor is obtained through nearest neighbor retrieval; The target point diffusion function is input into the frequency-spatial joint restoration network. The frequency-spatial joint restoration network performs coarse frequency-domain restoration and fine spatial restoration on local blocks of the degraded image, and outputs the final clear image.

2. The parametric point spread function regression retrieved optical degraded image restoration method according to claim 1, characterized in that, The reference point spread function library for constructing the target imaging system includes: Discrete sampling is performed on the target imaging system within the effective working domain to obtain the spread function of each reference point, and the reference point spread function library is constructed based on each reference point spread function; the effective working domain includes at least the field of view position and the object distance.

3. The parametric point spread function regression retrieved optical degraded image restoration method according to claim 1, wherein, The point spread function prediction network includes an image branch and a coordinate branch. The input feature map of the image branch is composed of the spatial domain image of the local block of the degraded image, the frequency domain amplitude image after fast Fourier transform, and the spectral amplitude image after logarithmic compression. The coordinate branch encodes the field of view coordinates into a learnable embedding vector; The step of outputting the predicted descriptor corresponding to the local patch of the degraded image through the point spread function prediction network includes: The point spread function prediction network fuses the input feature map of the image branch with the embedding vector of the coordinate branch, and outputs the predicted descriptor through the regression head.

4. The method for optically degraded image restoration by parametric point spread function regression retrieval according to claim 1, characterized in that, The step of comparing the distance between the predicted descriptor and each parameterized descriptor in the reference point diffusion function library, and obtaining the target point diffusion function that matches the predicted descriptor through nearest neighbor retrieval, includes: In the reference point diffusion function library, the distance between the predicted descriptor and each of the parameterized descriptors is calculated; The physically realizable point spread function corresponding to the parameterized descriptor that has the smallest distance from the current predicted descriptor is used as the target point spread function.

5. The method for optically degraded image restoration by parametric point spread function regression retrieval according to claim 1, characterized in that, The frequency domain coarse recovery includes: Based on the frequency domain form of the local blocks of the degraded image and the spread function components of the target point, a Wiener-like deconvolution with a learnable regularization term is performed in the frequency domain to obtain a coarse frequency domain restoration result.

6. The method for optically degraded image restoration by parametric point spread function regression retrieval according to claim 5, characterized in that, The spatial refinement and restoration includes: The frequency domain coarse recovery result is subjected to inverse Fourier transform to obtain the spatial domain coarse recovery result; The spatial domain coarse restoration result is concatenated with the local block of the degraded image in the channel dimension to obtain the concatenated feature; The splicing features are input into a spatial domain refinement network, and the splicing features are processed by the backbone network of the spatial domain refinement network, which has residual downsampling blocks, skip connections, and upsampling decoding structures.

7. The method for optically degraded image restoration by parametric point spread function regression retrieval according to claim 1, characterized in that, The parameterized descriptor is a three-parameter descriptor, which includes: Steller ratio, which characterizes the degree to which the center peak of the actual point spread function is preserved relative to the diffraction-limited point spread function; The energy radius is surrounded by a preset proportion representing the degree to which the energy of the point diffusion function diffuses outward; Full width at half maximum (FWHM) characterizes the degree of main lobe broadening of the point spread function. After extracting the low-dimensional parameterized descriptor, the method further includes: Unit energy normalization is performed on the diffusion function of each reference point, and min-max normalization is performed on the parameterized descriptor of each parameterized descriptor.

8. A device for optically degraded image restoration using parametric point spread function regression retrieval, characterized in that, include: The feature extraction module is used to construct a reference point spread function library for the target imaging system and extract a low-dimensional parameterized descriptor for each reference point spread function in the reference point spread function library. The prediction output module is used to obtain a local patch of the degraded image and the field coordinates of the local patch, input the local patch of the degraded image and the field coordinates of the field to the point spread function prediction network, and output the prediction descriptor corresponding to the local patch of the degraded image through the point spread function prediction network; The prior retrieval module is used to compare the distance between the predicted descriptor and each parameterized descriptor in the reference point diffusion function library, and obtain the target point diffusion function that matches the predicted descriptor through nearest neighbor retrieval; The joint restoration module is used to input the target point diffusion function into the frequency domain-spatial domain joint restoration network, and to perform coarse frequency domain restoration and fine spatial domain restoration on the local blocks of the degraded image through the frequency domain-spatial domain joint restoration network, and output the final clear image.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes the computer program stored in the memory, it implements the steps of the optical degraded image restoration method by parametric point spread function regression retrieval as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the optical degraded image restoration method by parametric point spread function regression retrieval as described in any one of claims 1-7.

Images