Large-view-field reflection optical system super-resolution imaging method based on computational imaging

Abstract

The invention provides a large-view-field reflection optical system super-resolution imaging method based on calculation imaging, and relates to the field of calculation imaging and super-resolution reconstruction, in order to meet the requirements of a large-view-field reflection optical system for high-resolution imaging and gradual light weight at the same time. The method comprises the following steps of: solving an initial structure of the large-view-field optical system by adopting a reflective Wassermann-Wolf equation; modeling is carried out to obtain an imaging model; performing optical simulation on the down-sampled low-resolution image by adopting an imaging model to obtain a low-resolution blurred image; constructing an image restoration super-resolution reconstruction network; taking the high-resolution and low-resolution blurred image pairs as a training data set, constructing a loss function joint training data set, and optimizing optical system parameters and image restoration super-resolution reconstruction network parameters; and reconstructing a high-definition high-resolution image by adopting the optimized image restoration super-resolution reconstruction network. The method is suitable for computational imaging and super-resolution reconstruction.

Description

Technical Field

[0001] The present invention relates to the fields of computational imaging and super-resolution reconstruction, and particularly to a super-resolution imaging method for a large field-of-view reflective optical system based on computational imaging. Background Art

[0002] Large field-of-view optical systems for high-definition imaging play an important role in promoting human exploration of the unknown world. In the past, in order to improve the imaging quality, optical designers chose to increase the number of optical elements and the system complexity to correct aberrations. However, in fields such as security monitoring and aerospace, due to the limited mass that the platform can carry, the quality of the optical system is strictly restricted. The increasingly complex lenses do not meet the requirements of miniaturization and light weight of optoelectronic systems in the above scenarios, and there is an urgent need to develop high-quality imaging technologies with reduced optical complexity. In recent years, computational imaging provides strong support for the design of simple optical imaging systems with large field-of-view and high resolution. The core idea is to jointly optimize the optical system design and the image processing algorithm at the backend, and reduce the strict requirements for the optical system design through the image restoration algorithm, so as to obtain a large field-of-view optical system with a simple structure and clear imaging.

[0003] Currently, the methods for simplifying optical systems based on the idea of computational imaging mainly focus on the degradation of imaging quality caused by optical aberration blur at the backend processing algorithm, while ignoring the limitations of the inherent physical characteristics of the optoelectronic receiving device itself. During the recording, processing, and transmission processes after image formation, the insufficient resolution of the optical detector will also lead to the degradation of image quality. Super-resolution imaging is expected to solve this problem. It reconstructs one or more low-resolution images using information processing methods to obtain a high-resolution image containing more high-frequency information. The "super" represents breaking through the limit. The super-resolution technology can break through the resolution limit of the original low-resolution system at a relatively low cost. Therefore, combining the optical system design with the image restoration super-resolution reconstruction technology can break through the hardware limitations of the detector and reconstruct the low-resolution image with aberrations and resolution limited by the detector sampling frequency to obtain a high-resolution image with rich details. Summary of the Invention

[0004] In order to meet the requirements of high-resolution imaging and gradual light weight of a large field-of-view reflective optical system, the present invention proposes a super-resolution imaging method for a large field-of-view reflective optical system based on computational imaging.

[0005] To achieve the above object, the present invention provides the following technical solutions: The present invention proposes a super-resolution imaging method for a large field-of-view reflective optical system based on computational imaging, and the method includes the following steps: Step S1: Solve the initial structure of the large field-of-view optical system using the reflective Wassermann-Wolf equation; Step S2: Model the initial structure of the optical system to obtain the imaging model of the large field-of-view optical system; Step S3: Obtain the original high-resolution image and perform downsampling. Use the imaging model to perform optical simulation on the downsampled low-resolution image to obtain the low-resolution blurred image degraded by the optical system; Step S4: Construct an image restoration super-resolution reconstruction network; Step S5: Use the high- and low-resolution blurred image pairs as the training data set, construct a loss function to jointly train the data set, and use the gradient backpropagation mechanism of deep learning to jointly optimize the optical system parameters and the image restoration super-resolution reconstruction network parameters; Step S6: Use the optimized image restoration super-resolution reconstruction network to reconstruct the low-resolution blurred image into a high-definition high-resolution image.

[0006] Further, the above Step S1 is specifically as follows: Step S11: By solving the Wassermann-Wolf equation, obtain the optical parameters of the corrector surface shape with aberration correction for different fields of view respectively; Step S12: Use the least squares method to fuse multiple correctors to obtain a unified corrector, thereby obtaining the initial structure of the large field-of-view optical system.

[0007] Further, use python to model the initial structure of the above optical system.

[0008] Further, the above Step S3 is specifically as follows: Step S31: According to the imaging model of the optical system, track the light rays from the object plane through the optical system to the image plane in the entire field of view, and process the intersection points of a large number of sampled light rays on the image plane to obtain the point spread function PSF; Step S32: Simulate the low-resolution blurred image degraded by the optical system according to the point spread function PSF.

[0009] Further, the above low-resolution blurred image degraded by the optical system B is expressed as:

[0010] where, I is the clear scene image, h is the optical system point spread function, n is the random noise.

[0011] Further, the above image restoration super-resolution reconstruction network is an improved dual-branch generative adversarial network. The input low-resolution blurred image enters two parallel feature extraction modules respectively for feature extraction, and then the optical blur feature and the super-resolution reconstruction feature are extracted specifically. Then, effective features are further extracted through the feature fusion module to reconstruct a high-definition high-resolution image.

[0012] Further, a deblurring loss, a reconstruction loss, and a mean square error (MSE) loss are combined to obtain the above loss function.

[0013] The super-resolution imaging method for a large field-of-view reflective optical system based on computational imaging according to the present invention can be entirely implemented by computer software. Therefore, correspondingly, the present invention also provides a super-resolution imaging system for a large field-of-view reflective optical system based on computational imaging. The system specifically includes: A storage device for solving the initial structure of the large field-of-view optical system by using the reflective Wassermann-Wolf equation; A storage device for modeling the initial structure of the optical system to obtain an imaging model of the large field-of-view optical system; A storage device for acquiring an original high-resolution image and performing downsampling, and performing optical simulation on the downsampled low-resolution image by using the imaging model to obtain a low-resolution blurred image degraded by the optical system; A storage device for constructing an image restoration super-resolution reconstruction network; A storage device for using the high- and low-resolution blurred image pairs as a training data set, constructing a loss function to jointly train the data set, and jointly optimizing the optical system parameters and the image restoration super-resolution reconstruction network parameters by using the gradient backpropagation mechanism of deep learning; A storage device for using the optimized image restoration super-resolution reconstruction network to reconstruct the low-resolution blurred image into a high-definition high-resolution image.

[0014] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the super-resolution imaging method for a large field-of-view reflective optical system based on computational imaging described in any one of the above.

[0015] The present invention also provides a computer device, which includes a memory and a processor. A computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes the super-resolution imaging method for a large field-of-view reflective optical system based on computational imaging described in any one of the above.

[0016] The beneficial effects of the present invention are: 1. The present invention takes into account factors such as optical system design and the limitation of the physical resolution of detectors within the design framework, and uses the Wassermann-Wolf design theory to solve the initial structure of a large-field-of-view reflective optical system, which can meet high-resolution imaging while expanding the field of view of the reflective optical system, and is applicable to fields such as aerospace and security monitoring.

[0017] 2. The present invention constructs a dual-branch parallel generative adversarial network for image restoration super-resolution reconstruction, which can extract blur features and reconstruction features specifically, can overcome the problem of the limitation of the basic spatial resolution of sensors, obtain super-resolution information, is beneficial to enhancing image details, and can increase the accuracy of subsequent target recognition.

[0018] The present invention is applicable to computational imaging and super-resolution reconstruction in fields such as aerospace and security monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1 is a flowchart of a super-resolution imaging method for a large-field-of-view reflective optical system based on computational imaging proposed by the present invention; Figure 2 is a schematic diagram of the Wassermann-Wolf curved surface optical system described in the present invention; Figure 3 are two groups of aberration-correcting optical devices for the central field of view and the edge field of view described in the present invention. Among them, Fig. (a) is the optical device for the central field of view of 0°, and Fig. (b) is the optical device for the edge field of view of 1.5°; Figure 4 is the initial structure of a large-field-of-view reflective optical system with aberration correction for each field of view described in the present invention; Figure 5 is the network structure of the optical blur image restoration super-resolution reconstruction proposed by the present invention; Figure 6 is the network structure of the generator described in the present invention; Figure 7 is the network structure of the discriminator described in the present invention; Figure 8 are the super-resolution reconstruction image, the low-resolution blurred image, and the original clear image described in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS The following further elaborates on the specific implementation manners of the present invention in conjunction with the accompanying drawings. The following implementation manners will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several changes and improvements can still be made, and these all fall within the protection scope of the present invention.

[0021] Embodiment 1. Refer to Figure 1 To illustrate this embodiment, this embodiment is proposed to meet the requirements of high-resolution imaging and gradual lightweighting of a large-field-of-view reflective optical system. A super-resolution imaging method for a large-field-of-view reflective optical system based on computational imaging is proposed. The method is as Figure 1 shown and specifically includes the following steps: Step S1: Solve the initial structure of the large-field-of-view optical system using the reflective Wassermann-Wolf equation; Step S2: Model the initial structure of the optical system to obtain an imaging model of the large-field-of-view optical system; Step S3: Obtain the original high-resolution image and perform downsampling. Use the imaging model to perform optical simulation on the downsampled low-resolution image to obtain a low-resolution blurred image degraded by the optical system; Step S4: Construct an image restoration super-resolution reconstruction network; Step S5: Use the high- and low-resolution blurred image pairs as the training data set, construct a loss function to jointly train the data set, and use the gradient backpropagation mechanism of deep learning to jointly optimize the optical system parameters and the image restoration super-resolution reconstruction network parameters; Step S6: Use the optimized image restoration super-resolution reconstruction network to reconstruct the low-resolution blurred image into a high-definition high-resolution image.

[0022] In practical applications of this embodiment, first, the reflective Wassermann-Wolf equation is applied to the establishment of the initial structure of a large field-of-view optical system: Using the design theory of the Wassermann-Wolf equation, a CODE V macro language program is written to control the landing points of the light rays in each field of view on the image plane, making them as close as possible. The surface shape data of the two reflective surfaces of the optical corrector for aberration correction in different fields of view are solved respectively. To obtain the initial structure with aberration correction in different fields of view, the least squares method is used to fit multiple correctors to obtain a unified corrector, minimizing the aberration in each field of view, and thus obtaining the initial structure of the large field-of-view reflective optical system. Then, its optical imaging model and image restoration super-resolution reconstruction model are constructed. The optical imaging model can obtain a simulated blurred image given the basic parameters of the optical system. Subsequently, a super-resolution image free of blur is obtained through the image restoration super-resolution reconstruction network. The imaging model uses the ray tracing method to obtain the differentiable point spread function (PSF) of the optical system. Since the PSF is differentiable, the reconstructed image is differentiable with respect to both the optical system parameters and the restoration algorithm parameters. Therefore, the gradient can be effectively transmitted from the restoration algorithm to the imaging model during optimization, and the calculation and propagation of the gradient can be completed through Pytorch. Therefore, through this model, it is possible to simultaneously optimize the optical parameters and the parameters of the restoration and reconstruction algorithm, achieve the best matching between the imaging system and the reconstruction algorithm, and thus achieve the purpose of simplifying the optical system structure and improving the imaging resolution.

[0023] This embodiment innovatively combines the optical system design with the image restoration super-resolution reconstruction technology, and uses the gradient backpropagation mechanism of deep learning to jointly optimize the lens parameters and the parameters of the image processing algorithm. Among them, the solution of the initial structure of the optical system before joint optimization is particularly important, which affects the network convergence time and the reconstruction effect. The aberrations in each field of view of a large field-of-view optical system are usually inconsistent in type and size, which is not conducive to the convergence of the subsequent joint optimization network. Therefore, this embodiment adopts the Wassermann-Wolf design theory to solve this problem. In the Wassermann-Wolf algorithm, this embodiment obtains the surface shape profiles of the two surfaces of the optical corrector by solving a pair of differential equations. The aspheres obtained can control the incident angles of each ray and the landing points in the image space, and the optical parameters of the surface shapes of the corrector for aberration correction in different fields of view are obtained respectively. Then, the least squares method is used to fuse multiple correctors to obtain a unified corrector, minimizing the aberration in each field of view. This embodiment uses the Wassermann-Wolf design theory to solve the initial structure of the large field-of-view reflective optical system, and then combines it with the super-resolution reconstruction algorithm, which can not only expand the field of view and improve the network training efficiency, but also break through the hardware limitations of the detector, reconstruct the low-resolution image with aberration and limited resolution due to the detector sampling frequency, and obtain a high-resolution image with rich details.

[0024] Embodiment 2. This embodiment specifically describes each step in a super-resolution imaging method for a large field-of-view reflective optical system based on computational imaging proposed in the above embodiment; Step S1: Solve the initial structure of the large field-of-view optical system using the reflective Wassermann-Wolf equation; Specifically: Step S11: By solving the Wassermann-Wolf equation, obtain the optical parameters of the corrector surface shape with aberration correction for different fields of view respectively; Step S12: Use the least squares method to fuse multiple correctors to obtain a unified corrector, thereby obtaining the initial structure of the large field-of-view optical system.

[0025] Step S2: Model the initial structure of the optical system to obtain the imaging model of the large field-of-view optical system; Specifically: Establish an optical imaging model. Its feature is to use python to model and analyze the initial structure of the optical system obtained in the above step S1, track the light rays from the object plane through the optical system to the image plane in the entire field of view, and process the intersection points of a large number of sampled light rays on the image plane, so as to obtain the blur spot formed by the light from infinity on the image plane after passing through the optical system, that is, the point spread function PSF.

[0026] Step S3: Obtain the original high-resolution image and perform downsampling, and use the imaging model to perform optical simulation on the downsampled low-resolution image to obtain the low-resolution blurred image degraded by the optical system; Specifically: S31: According to the imaging model of the optical system, track the light rays from the object plane through the optical system to the image plane in the entire field of view, and process the intersection points of a large number of sampled light rays on the image plane to obtain the point spread function PSF; S32: Simulate the low-resolution blurred image degraded by the optical system according to the point spread function PSF.

[0027] In the actual application of this step, use the obtained PSF point spread function to simulate the low-resolution blurred image degraded by the optical system. Whether the optical system structure is complex or simple, the relationship between the object and the image can be expressed as:

[0028] Where I is the clear scene image, h is the optical system point spread function, n is the random noise, BIt is a blurred image degraded by the optical system. Therefore, after accurately obtaining the PSF of the optical system, the low-resolution blurred image degraded by the optical system can be obtained by convolving the clear image with the PSF and adding noise.

[0029] Step S4: Construct an image restoration super-resolution reconstruction network; Specifically: In order to ensure that the neural network can perform image restoration and super-resolution reconstruction simultaneously, an improved dual-branch generative adversarial network is constructed. The input low-resolution blurred image enters two parallel feature extraction modules respectively for feature extraction, and specifically extracts optical blur features and super-resolution reconstruction features. Then, effective features are further extracted through the feature fusion module, and finally a clear high-resolution image is reconstructed.

[0030] Step S5: Use the high- and low-resolution blurred image pairs as the training dataset, construct a loss function to jointly train the dataset, and adopt the gradient backpropagation mechanism of deep learning to jointly optimize the optical system parameters and the parameters of the image restoration super-resolution reconstruction network; Specifically: Use the high- and low-resolution blurred image pairs obtained in the above step S3 as the training dataset, construct a loss function to jointly train the dataset, and use the gradient backpropagation mechanism of deep learning to jointly optimize the optical system parameters and the parameters of the restoration and reconstruction network. Among them, the optical imaging model uses the ray tracing method to obtain the differentiable point spread function PSF of the optical system. Since the PSF is differentiable, the reconstructed image is differentiable with respect to both the optical system parameters and the restoration algorithm parameters. Therefore, the gradient can be effectively transmitted from the restoration algorithm to the imaging model during the optimization process, and the calculation and propagation of the gradient can be completed by Pytorch. Therefore, through this model, the simultaneous optimization of the optical parameters and the parameters of the restoration and reconstruction algorithm can be realized, and the best matching between the imaging system and the reconstruction algorithm can be achieved.

[0031] Step S6: Use the optimized image restoration super-resolution reconstruction network to reconstruct the low-resolution blurred image into a high-definition high-resolution image.

[0032] Embodiment 3. Refer to Figures 2 to 8 This embodiment is described. This embodiment gives an overall example of a super-resolution imaging method for a large-field reflective optical system based on computational imaging described in the above embodiment; Step 1: Use the reflective Wassermann-Wolf equation to solve the initial structure of the large-field optical system: The Wassermann-Wolf surface is two adjacent reflecting surfaces. Considering the situation of any ray passing through these two surfaces, the optical path schematic diagram at this time is as Figure 2 shown, Figure 2Among them, (hx1, hy1, 0) is the intersection point of the extension line of the incident light and the tangent plane of the first reflecting surface, and (hx2, hy2, 0) is the intersection point of the light after reflection from the second surface and the tangent plane of the second reflecting surface. (x1, y1, z1) and (x2, y2, z2) are the intersection points of the light with the two reflecting surfaces respectively. Among them, for the coordinate systems of (hx1, hy1, 0) and (x1, y1, z1), the origin is located at the vertex of the first reflecting surface, and for the coordinate systems of (hx2, hy2, 0) and (x2, y2, z2), the origin is located at the vertex of the second reflecting surface. The z-axis directions of these two coordinate systems are horizontally to the right, the x-axis is perpendicular to the paper and into the paper, and the y-axis is on the paper and perpendicular to the z-axis and upward. The distance between the two reflecting surfaces is d, and u and u' are the object-space aperture angle and the image-space aperture angle respectively.

[0033] Therefore, the reflective Wassermann-Wolf equation can be expressed as:

[0034] Among them,

[0035]

[0036] Among them, is the direction vector of the incident light to the first reflecting surface; is the direction vector of the incident light after passing through the second reflecting surface, represents the distance in the x-axis direction of the intersection points of the light with the two reflecting surfaces, represents the distance in the y-axis direction of the intersection points of the light with the two reflecting surfaces, The distance in the z-axis direction of the intersection points of the light with the two reflecting surfaces.

[0037] Iteratively solve the above formula to obtain the discrete data information on the two surfaces to be solved respectively. Then use the polynomial surface fitting method to fit the two sets of data points, and the surface equations of the two Wassermann-Wolf reflecting surfaces can be obtained. Using this method, the surface equations of the achromatic optical correctors in multiple different fields of view are obtained respectively. Among them, two sets of optical correctors in the central field of view of 0° and the marginal field of view of 1.5° are as Figure 3 shown.

[0038] The least squares method is used to construct a global optimization problem, and the obtained multiple corrector surface equations are fitted into a unified mathematical model. First, the same data points are sampled from each surface equation, and an error function is established to calculate the deviation between the actual surface and the fitted model. Then, a least squares optimization problem is constructed and solved, and the optimal coefficients are calculated through the normal equations; the fitted surface is made as close as possible to all the original surfaces. Finally, the mean square error is used to evaluate the fitting quality to ensure that the unified surface can accurately describe the original data. Finally, a unified corrector is obtained such as Figure 4 As shown, the initial structure of the large-field reflective optical system with aberration-free field of view is obtained.

[0039] Step 2: Model the optical imaging process and accurately obtain the point spread function (PSF) of the optical system. The specific steps are as follows: The optical surface equation in the Cartesian coordinate system is used together with the incident light to obtain the intersection of the light rays. The light rays are reflected at the intersection. According to the law of reflection, the angle of incidence is equal to the angle of reflection. The direction vector of the incident light ray is known. , the normal direction vector of the mirror surface at the intersection , then the incident angle It can be expressed as:

[0040] From this we can get the direction vector of the reflected light The calculation formula is as follows:

[0041] By using the imaging model to input the basic parameters of the optical system, the coordinates of the intersection of the light and the image plane in any field of view can be obtained. The light of different fields of view can be obtained by changing the angle of the incident light. In order to avoid the problem of non-differentiability in the traditional ray tracing process, the light is Gaussianized and processed. The light reaches the image plane not as a point, but as a diffuse spot. The intensity distribution of each incident light on the image plane can be approximated as a Gaussian distribution. By tracing all incident light in a field of view and accumulating all light intensity matrices, the PSF of the field of view can be obtained.

[0042]

[0043] in, is the normalization factor, Represents the light intensity distribution caused by light reaching the (p,q) pixel.

[0044] Step 3: Use the PSF obtained in the previous step to simulate the degraded image of the optical system. Regardless of whether the optical system structure is complex or simple, the relationship between the object and the image can be expressed as:

[0045] Among them, I is a clear scene image, h is the point spread function of the optical system, n is random noise, B is the blurred image degraded by the optical system. The blurred image degraded by the optical system can be obtained by convolving the clear image with the optical system PSF and adding noise. Among them, when simulating the image captured by the optical system, the standard deviation of the added Gaussian noise is 0.01.

[0046] Step 4: To ensure that the neural network can perform image restoration and super-resolution reconstruction simultaneously, a dual-branch generative adversarial network is constructed. This network can extract optical blur features and super-resolution reconstruction features specifically. The overall structure of the network is as Figure 5 shown. The input low-resolution blurred image enters two parallel feature extraction modules respectively for feature extraction, then further extracts effective features through the feature fusion module, and finally outputs the reconstructed clear image through the reconstruction module.

[0047] Among them, the blur feature extraction module includes two convolutional layers and 6 residual blocks with the same structure. After passing through the first 3×3 convolutional layer, a 64-channel feature map is obtained, then deep feature extraction is performed through the dense residual module, and finally a 64-channel feature map is output through a 3×3 convolutional layer to the fusion module. In addition, the blur feature map passes through another convolutional layer to restore its number of channels to 3, obtaining a deblurred low-resolution image for calculating the deblurring loss function. The reconstruction feature extraction module is basically the same as the blur feature extraction module, using 6 residual blocks as the main structure, and finally outputs a 64-channel feature map to enter the fusion module. The fusion module consists of two convolutional layers, with the convolutional kernel sizes being 3×3 and 1×1 respectively, and the activation function being LeakyRelu. The image reconstruction module includes 2 sub-pixel upsampling layers. Each time passing through an upsampling layer, the image size changes from H×W×C to 2H×2W×C, and finally 4-fold super-resolution reconstruction is achieved. The structure of the generator is as Figure 6 shown.

[0048] The discriminator adopts PatchGAN. It cuts the input image into small pieces for training, which can enhance the local texture details of the image. The discriminator network model is as Figure 7 shown. The number of filter channels gradually increases from 64 output channels to 512 channels, and finally the sigmoid activation function is used to distinguish the probability that different image small pieces are real.

[0049] Step 5: Optical-digital joint optimization, constructing a training dataset and a loss function, and jointly optimizing the basic parameters of the optical system and the parameters of the restoration and reconstruction network using the gradient backpropagation mechanism of deep learning.

[0050] ① First, construct the loss function. Since the network adopts a dual-branch parallel structure, the joint deblurring loss and the reconstruction loss are selected and constrained by the mean square error (MSE) loss:

[0051] Among them, is the MSE loss between the low-resolution clear image and the low-resolution deblurred image, is the MSE loss between the original high-resolution image and the reconstructed super-resolution image.

[0052] The SSIM (structural similarity index) can compare the similarity of images in terms of brightness, color, structure, etc., and is more in line with the human visual system. The ideal structural similarity of the reconstructed image is 1, and the pixel-level SSIM (structural similarity index) is defined as:

[0053] Among them, W and H represent the width and height of the image respectively, y represents the original high-resolution image, G(x) represents the super-resolution image generated by the generator network, represents the structural similarity function.

[0054] The final joint optimization loss function can be expressed as:

[0055] Among them, and are the hyperparameters of and respectively, used to adjust their respective proportions.

[0056] ② Select the original high-resolution images from the public dataset. First, perform downsampling on the original high-resolution images to make them the same size as the detector, and then use the optical system degradation model to perform optical simulation on the downsampled low-resolution images to obtain low-resolution blurred images, and finally obtain a paired high-low resolution blurred image training dataset.

[0057] ③ Use the constructed loss function to train the entire model until the network converges. Save the optical system parameters and the reconstruction network parameters once for each optimized batch, and generate the optical system PSF, degraded images, deblurred images, and super-resolution reconstructed images under the corresponding parameters. During the training process, the ADAM optimizer is used to perform backpropagation on the gradients of the network, and continuously update the optical system parameters and the weights of the restored super-resolution reconstruction network parameters.

[0058] ④ After joint optimization, the reconstruction effect is as shown inFigure 8 As shown, from left to right are: the super-resolution reconstructed image after joint optimization, the blurred image after downsampling and optical degradation, and the clear scene image. The peak signal-to-noise ratio PSNR and SSIM (structural similarity index) of the test images are shown in Table 1.

[0059] Table 1

[0060] It can be seen from Table 1 that the reconstructed image is 0.288 higher than the degraded image in terms of the structural similarity SSIM (structural similarity index), indicating that the similarity between the degraded image and the reconstructed image in terms of structure, brightness, and contrast is higher. And the reconstructed image is 11.12 higher than the degraded image in terms of the peak signal-to-noise ratio PSNR, indicating that the image quality has been improved and the noise or distortion has been reduced.

[0061] Embodiment 4. The super-resolution imaging method of a large-field reflective optical system based on computational imaging described in the above embodiments can all be implemented by computer software. Therefore, in this embodiment, a super-resolution imaging system of a large-field reflective optical system based on computational imaging is proposed. The system specifically includes: a storage device for solving the initial structure of the large-field optical system by using the reflective Wassermann-Wolf equation; a storage device for modeling the initial structure of the optical system to obtain an imaging model of the large-field optical system; a storage device for acquiring the original high-resolution image and performing downsampling, and performing optical simulation on the downsampled low-resolution image by using the imaging model to obtain a low-resolution blurred image degraded by the optical system; a storage device for constructing an image restoration super-resolution reconstruction network; a storage device for using the high- and low-resolution blurred image pairs as a training data set, constructing a loss function to jointly train the data set, and jointly optimizing the optical system parameters and the image restoration super-resolution reconstruction network parameters by using the gradient backpropagation mechanism of deep learning; a storage device for using the optimized image restoration super-resolution reconstruction network to reconstruct the low-resolution blurred image into a high-definition high-resolution image.

[0062] Embodiment 5. This embodiment proposes a computer-readable storage medium on which a computer program is stored. When the computer program is run by a processor, it executes the super-resolution imaging method of a large-field reflective optical system based on computational imaging described in any one of the above Embodiments 1 to 3.

[0063] Embodiment 6. This embodiment provides a computer device, which includes a memory and a processor. A computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes the method for super-resolution imaging of a large field-of-view reflective optical system based on computational imaging according to any one of Embodiments 1 to 3 above.

[0064] For the computer device provided in this embodiment, the hardware device in this part is of a general model and is not shown in the form of a diagram. The system includes a processor and a memory. The processor and the memory can be connected through a bus or other means. The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, as well as corresponding program instructions / modules. The processor runs the non-transitory software programs, instructions, and modules stored in the memory, thereby executing various functional applications and data processing of the processor to implement the method and steps for super-resolution imaging of a large field-of-view reflective optical system based on computational imaging in the above method embodiments.

[0065] The above are only the embodiments of the present invention and do not limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention.

Claims

1. A method for super-resolution imaging of a large field of view reflective optical system based on computational imaging, characterized in that: The method is: S1: Using the reflective Wassermann-Wolf equation to solve the initial structure of the large field of view optical system; S2: Model the initial structure of the optical system to obtain an imaging model of the large field of view optical system; S3: acquiring the original high-resolution image and downsampling it, and using the imaging model to perform optical simulation on the downsampled low-resolution image to obtain a low-resolution blurred image after optical system degradation; S4: Construct image restoration super-resolution reconstruction network; S5: Use high-resolution and low-resolution blurred image pairs as training data sets, and construct a loss function to jointly train the data sets. Use the gradient backpropagation mechanism of deep learning to jointly optimize the optical system parameters and the image restoration super-resolution reconstruction network parameters. S6: The optimized image restoration super-resolution reconstruction network is used to reconstruct the low-resolution blurred image into a high-definition high-resolution image.

2. The method for super-resolution imaging of a large field of view reflective optical system based on computational imaging according to claim 1, characterized in that: S1 is specifically: S11: By solving the Wassermann-Wolf equation, the surface optical parameters of the corrector with different field of view aberration correction are obtained respectively; S12: Multiple correctors are fused using the least squares method to obtain a unified corrector, thereby obtaining the initial structure of the large field of view optical system.

3. The method for super-resolution imaging of a large field of view reflective optical system based on computational imaging according to claim 1, characterized in that: The initial structure of the optical system was modeled using Python.

4. The method for super-resolution imaging of a large field of view reflective optical system based on computational imaging according to claim 3, characterized in that: S3 is specifically: S31: According to the imaging model of the optical system, the light rays from the object plane through the optical system to the image plane in the entire field of view are tracked, and the intersection points of a large number of sampled light rays on the image plane are processed to obtain the point spread function PSF; S32: Simulate a low-resolution blurred image after optical system degradation according to the point spread function PSF.

5. The method for super-resolution imaging of a large field of view reflective optical system based on computational imaging according to claim 4, characterized in that: Low-resolution blurred image after optical system degradation B It is expressed as: in, I For a clear scene image, h is the point spread function of the optical system, n is random noise.

6. The method for super-resolution imaging of a large field of view reflective optical system based on computational imaging according to claim 1, characterized in that: The image restoration super-resolution reconstruction network is an improved dual-branch generative adversarial network. The input low-resolution blurred image enters two parallel feature extraction modules for feature extraction, and then the optical blur features and super-resolution reconstruction features are extracted in a targeted manner. The effective features are further extracted through the feature fusion module to reconstruct a high-definition, high-resolution image.

7. The method for super-resolution imaging of a large field of view reflective optical system based on computational imaging according to claim 1, characterized in that: The loss function is obtained by combining deblurring loss, reconstruction loss and mean square error (MSE) loss.

8. A large-field-of-view reflective optical system super-resolution imaging system based on computational imaging, characterized in that: The system comprises a storage device for executing the method and steps described in claim 1.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the large-field-of-view reflective optical system super-resolution imaging method based on computational imaging as described in any one of claims 1 to 7.

10. A computer device, characterized in that: The device includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes the large-field-of-view reflective optical system super-resolution imaging method based on computational imaging according to any one of claims 1 to 7.

Images