Simple optical system design method based on the consistency constraint of optical transfer function

Abstract

The present invention belongs to the technical field of computational optical imaging, and discloses a simple optical system design method based on the consistency constraint of the optical transfer function, including the following steps: First, design the initial structure of a simple lens, specifically including: calculating the aperture diameter of the lens, calculating the position of the lens aperture, setting the multiple structure of the lens aperture, and setting the evaluation function for optimization; Secondly, control the consistency of the optical transfer function of the simple lens, specifically including: maintaining the above evaluation function, calculating the average optical transfer function, calculating the consistency of the optical transfer function, and adding an evaluation function for optimization. The present invention adopts the above simple optical system design method based on the consistency constraint of the optical transfer function to realize the optimization design of a single-chip computational infrared system based on computational optics, effectively control the consistency of the optical transfer function, and has important guiding significance for the real-time imaging of a single-chip computational optical system.

Description

Technical Field

[0001] The present invention relates to the technical field of computational optical imaging, and in particular to a simple optical system design method based on optical transfer function consistency constraints. Background Art

[0002] In recent years, with the rapid development of space remote sensing, autonomous driving, infrared guidance and other fields, the demand for optical imaging systems with high imaging quality and light weight and miniaturization has been increasing. In order to achieve high-quality imaging effects, traditional optical imaging systems often rely on complex lens groups to correct optical aberrations, which not only makes the system bulky and heavy, but also increases the complexity of design and manufacturing. Computational imaging technology uses image processing algorithms as a virtual aberration correction method to effectively achieve high-quality imaging without increasing or even reducing the number of optical components, thus becoming an ideal solution that takes into account both high imaging quality and light weight and miniaturization. Based on this, single-lens computational imaging systems have received widespread attention and research.

[0003] However, single-lens computational imaging systems face a significant challenge in practical applications: the complexity of image processing algorithms often makes it difficult to meet the demand for real-time video-level output of reconstructed images in practical applications. The huge computing requirements make it difficult for existing algorithms to achieve real-time video-level output of reconstructed images on the neural network chip side, further limiting the practical application potential of single-lens computational imaging systems.

[0004] In single-lens computational imaging systems, there is a close connection between optical design and the selection and complexity of image processing algorithms. A single-lens design method based on subaperture optimization successfully realizes a single-lens system with a spatially invariant point spread function. This design allows the algorithm in the subsequent image processing stage to select a lighter Unet neural network, thereby significantly reducing the computational complexity of the algorithm while maintaining high-performance imaging. Therefore, a lightweight Res-Unet neural network algorithm can be selected in the image processing stage to achieve high-performance imaging. Nevertheless, this method still has limitations in reducing the computational complexity of the algorithm. Such high computing requirements still far exceed the real-time processing capabilities of the neural network chip, limiting the real-time video-level output of the reconstructed image. Therefore, how to further optimize the optical design method and develop a single-lens optical design method that can significantly reduce the complexity of image processing has become a key issue that needs to be solved urgently. The solution to this problem will not only promote the implementation of single-lens computational imaging technology in more practical application scenarios, but will also provide important reference and reference for the design of other high-performance computational imaging systems. Summary of the invention

[0005] The object of the present invention is to provide a simple optical system design method based on the consistency constraint of the optical transfer function to solve the problems of high complexity and poor real-time performance of image processing algorithms in the prior art. By controlling the consistency of the optical transfer function (OTF) of a single-lens system within the full field of view, this method can significantly reduce the complexity of the image processing algorithm, thereby achieving an efficient and real-time imaging effect.

[0006] To achieve the above object, the present invention provides a simple optical system design method based on the consistency constraint of the optical transfer function, including the following steps:

[0007] Step S1, design the initial structure of a simple lens;

[0008] Step S2, control the consistency of the optical transfer function of the simple lens.

[0009] Preferably, in step S1, when designing the initial structure of the simple lens, the specific process is as follows:

[0010] Step S11, calculate the sub-aperture diameter of the lens;

[0011] Step S12, calculate the position of the sub-aperture of the lens;

[0012] Step S13, set the multi-structure of the sub-aperture of the lens;

[0013] Step S14, set the evaluation function for optimization.

[0014] Preferably, in step S2, when controlling the consistency of the optical transfer function of the simple lens, the specific process is as follows:

[0015] Step S21, maintain the evaluation function of step S1;

[0016] Step S22, calculate the average optical transfer function;

[0017] Step S23, calculate the consistency of the optical transfer function;

[0018] Step S24, add the evaluation function for optimization.

[0019] Preferably, in step S11, when calculating the sub-aperture diameter of the lens, the specific process is as follows:

[0020] Based on the criterion that the value of the diffraction-limited MTF corresponding to the sub-aperture at the Nyquist frequency is between 0.1 and 0.2, determine the range of the sub-aperture diameter, as follows: value between 0.1 and 0.2 as the criterion, determine the sub-aperture diameter range, as follows: range, as follows:

[0021] ;

[0022] Wherein, is the central wavelength, is the system focal length.

[0023] Preferably, in step S12, calculate the position of the lens aperture, and the specific process is as follows:

[0024] Assume that the projection of the sub-aperture related to each field of view on the real aperture is uniformly distributed, then calculate the coordinate of the projection aperture in the y direction , as follows:

[0025] ;

[0026] Among them, is the i-th field angle, is the total field angle, is the diameter of the real aperture;

[0027] According to the trigonometric function relationship, calculate the distance between the sub-aperture and the real aperture , as follows:

[0028] .

[0029] Preferably, in step S13, set the parameters of each aperture in the optical design software as a single structure to form an optimized system with a multi-structure.

[0030] Preferably, in step S14, set the lens parameters of the simple optical system and the distance parameters between the sub-aperture and the real aperture as variables, set the evaluation function in the optical design software, use the minimum value of the radius of the blur spot as the evaluation index, and simultaneously constrain the focal length of the system.

[0031] Preferably, in step S22, calculate the average optical transfer function, average the optical transfer functions of all fields of view to obtain the average optical transfer function , as follows:

[0032] ;

[0033] Among them, is the optical transfer function of the i-th field of view, N is the number of fields of view, is the parameter of the simple optical system.

[0034] Preferably, in step S23, calculate the optical transfer function consistency. The sum of the absolute values of the differences between the optical transfer functions of each field of view and the average optical transfer function is defined as the optical transfer function consistency , as follows:

[0035] .

[0036] Preferably, in step S24, an evaluation function for optimization is added. Based on step S1, two operands are added to the evaluation function:

[0037] One is to control the consistency of the optical transfer function , making it close to 0; the other is to control the minimum value of the optical transfer function.

[0038] Therefore, the present invention adopts the above-mentioned simple optical system design method based on the consistency constraint of the optical transfer function, which is applicable to the design of simple lens group optical systems, simple mirror group optical systems, single-lens optical systems, and single-mirror optical systems; the present invention controls the consistency of the optical transfer function of the optical system, thereby reducing the complexity of the image processing algorithm; this design strategy enables the image reconstruction of a simple computational imaging system on a neural network chip to achieve real-time video output, greatly improving the efficiency and response speed of the imaging system.

[0039] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Description of the Drawings

[0040] Figure 1 is a flowchart of the simple optical system design method based on the consistency constraint of the optical transfer function of the present invention;

[0041] Figure 2 is the optical path diagram of the single-lens computational infrared optical system in the embodiment of the present invention;

[0042] Figure 3 is the schematic diagram of the sub-aperture position in the embodiment of the present invention;

[0043] Figure 4 is the OTF diagram of the single-lens computational infrared optical system designed by the traditional design method in the embodiment of the present invention;

[0044] Figure 5 is the OTF diagram of the single-lens computational infrared optical system designed by the design method proposed by us in the embodiment of the present invention;

[0045] Figure 6 is the comparison diagram of the imaging performance (peak signal-to-noise ratio) of the single-lens computational infrared optical system in the embodiment of the present invention;

[0046] Figure 7 is the comparison diagram of the imaging performance (structural similarity) of the single-lens computational infrared optical system in the embodiment of the present invention;

[0047] Figure 8 is the comparison diagram of the imaging performance (simulation image) of the single-lens computational infrared optical system in the embodiment of the present invention. Detailed Embodiments

[0048] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0049] As Figure 1 shown, a simple optical system design method based on the consistency constraint of the optical transfer function includes the following steps:

[0050] Step S1: Design the initial structure of a simple lens. The specific process is as follows:

[0051] Step S11: Calculate the aperture diameter of the lens.

[0052] Step S12: Calculate the position of the lens aperture.

[0053] Step S13: Set the multi-structure of the lens aperture.

[0054] Step S14: Set the evaluation function for optimization.

[0055] Step S2: Control the consistency of the optical transfer function of the simple lens. The specific process is as follows:

[0056] Step S21: Maintain the evaluation function of Step S1.

[0057] Step S22: Calculate the average optical transfer function.

[0058] Step S23: Calculate the consistency of the optical transfer function.

[0059] Step S24: Add the evaluation function for optimization.

[0060] Embodiment 1

[0061] An embodiment of the present invention is a single-lens calculation infrared imaging system. The system has a focal length of 70 mm, an F-number of 1, a field of view of 6.4°×4.8°, a detection spectral range of 8 μm to 12 μm, and uses a non-cooled detector with 12-μm pixels. The detector array size is 640×480. The optical path diagram of this optical system is as Figure 2 shown.

[0062] Step S1: Design the initial structure of a simple lens.

[0063] Step S11: Calculate the aperture diameter of the lens: Based on the criterion that the value of the diffraction-limited optical transfer function (OTF) corresponding to the sub-aperture at the Nyquist frequency is between 0.1 and 0.2, determine the sub-aperture diameter range as follows:

[0064] ;

[0065] Among them, is the central wavelength, is the focal length of the system.

[0066] Here, the diffraction-limited OTF corresponding to the subaperture is taken at the Nyquist frequency The value is about 0.1, and the subaperture diameter is about 36mm.

[0067] Step S12, calculate the position of the lens sub-aperture: Assuming that the projection of each sub-aperture related to the field of view on the real aperture is uniformly distributed, calculate the coordinate of the projected aperture in the y direction , as shown below:

[0068] ;

[0069] in, is the i-th field of view, is the total field of view, is the diameter of the actual aperture; its schematic diagram is as follows Figure 3 shown.

[0070] According to the trigonometric relationship, the distance between the sub-aperture and the real aperture is , as shown below:

[0071] ;

[0072] Here, five uniformly distributed fields of view are selected, namely 0°, 1°, 2°, 3° and 4°, and the corresponding sub-aperture position coordinates ( , ) are (0, 0), (4.25, 243.48), (5.8, 243.41), (12.75, 243.28) and (17, 243.11) respectively.

[0073] Step S13, setting the lens sub-aperture multiple structures: setting the parameters of each aperture as a multiple structure in the optical design software to form a multiple structure optimization system.

[0074] Step S14, setting an evaluation function for optimization: setting parameters such as lens parameters of the simple optical system and the distance between the sub-aperture and the real aperture as variables, setting an evaluation function in the optical design software, taking minimization of the diffuse spot radius as the evaluation index, and at the same time constraining the focal length of the system to 70 mm.

[0075] After multiple rounds of optimization iterations, a single lens design parameter is obtained, and its full-field OTF consistency is poor, such as Figure 4 shown.

[0076] Step S2, controlling the consistency of the optical transfer function of the simple lens, and further controlling the consistency of the optical transfer function of the entire field of view of the system.

[0077] Step S21. Retain the evaluation function of Step S1: Based on the evaluation function set in Step S1, recreate a multi - structure and set the evaluation function to control the consistency of the full - field OTF under the true aperture.

[0078] Step S22. Calculate the average optical transfer function (OTF): Average the optical transfer functions of all fields of view to obtain the average optical transfer function , as follows:

[0079] ;

[0080] where is the OTF of the i - th field of view, N is the number of fields of view, is the parameter of the simple optical system.

[0081] Step S23. Calculate the OTF consistency: The sum of the absolute values of the differences between the OTF of each field of view and the average OTF is defined as the OTF consistency , as follows:

[0082] ;

[0083] Step S24. Add evaluation functions for optimization: Based on Step S1, add two operands to the evaluation function: one is to control the OTF consistency , making it as close to 0 as possible; the other is to control the minimum value of the optical transfer function, making it as large as possible.

[0084] After multiple rounds of iterative optimization, the optimal design parameters of the single - lens are obtained, and the consistency of its full - field OTF is effectively controlled, with good consistency, as Figure 5 shown.

[0085] Embodiment 2

[0086] To verify the superiority of the simple optical system design method based on the OTF consistency constraint proposed in the present invention in reducing the complexity of image - processing algorithms, a simulation comparison experiment is carried out in this embodiment.

[0087] First, a single lens is designed respectively by using the traditional design method and the method proposed in the present invention.

[0088] Next, based on the classic Res - Unet as the basic network, different complexity image - processing algorithms are implemented by adjusting its number of channels.

[0089] On this basis, three new algorithms were further developed: SS-Res-Unet (with only one-fourth of the number of channels of Res-Unet), S-Res-Unet (with only one-half of the number of channels of Res-Unet), and L-Res-Unet (with twice the number of channels of Res-Unet). When these algorithms process degraded images with a resolution of 640×480, their FLOPs are 4.79G (SS-Res-Unet), 19.2G (S-Res-Unet), 76.62G (Res-Unet), and 306.12G (L-Res-Unet), respectively.

[0090] Finally, the parameters of the above two single-lens systems were trained and optimized under four different algorithms, and the quality and performance of the restored images were compared and analyzed.

[0091] Thirty un-trained infrared images were selected as the validation set, and the average peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) before and after the restoration of the images were calculated, as Figure 6 and Figure 7 shown. Under the same algorithm complexity, compared with the traditional design method, the single lens designed by the design method proposed in the present invention has higher PSNR and SSIM. Specifically, the average PSNR is increased by 0.97dB to 1.80dB, and the average SSIM is increased by 0.0085 to 0.0116.

[0092] Finally, S-Res-Unet was selected as the image processing algorithm for the embodiment of the present invention. The reason for the selection is that when the algorithm complexity is further increased, the improvement in the average PSNR and SSIM is very small, while reducing the algorithm complexity will result in a significant decrease in the average PSNR and SSIM. Therefore, S-Res-Unet reaches an ideal balance point between the algorithm complexity and performance.

[0093] In addition, the image reconstruction performance of the embodiment of the present invention is better than that of the traditional design method. Specifically, the average PSNR of the reconstructed image using the S-Res-Unet image processing algorithm is 34.17dB, while even when using the more complex L-Res-Unet image processing algorithm in the traditional design method, the average PSNR of its reconstructed image is only 33.70dB. This advantage is as Figure 8 shown, and the embodiment of the present invention shows better performance in the details of the reconstructed image. Especially in a low-temperature environment of -40°C, the traditional design method cannot effectively restore the detailed information of the image even when using the S-Res-Unet neural network image processing algorithm.

[0094] Although the use of the L-Res-Unet neural network image processing algorithm can improve the detail resolution ability of the restored image, the improvement is limited. In contrast, in the embodiment of the present invention, the details of the restored image using the S-Res-Unet neural network image processing algorithm are clearly visible, and its algorithm complexity is 16 times lower than that of L-Res-Unet, and it can achieve real-time video-level output of the reconstructed image at the neural network chip end.

[0095] Therefore, the simple optical system design method based on the consistency constraint of the optical transfer function adopted by the present invention is applicable to the design of simple lens group optical systems, simple mirror group optical systems, single lens optical systems, and single mirror optical systems; by controlling the consistency of the optical transfer function of the optical system, the present invention reduces the complexity of the image processing algorithm; this design strategy enables the image reconstruction of a simple computational imaging system on a neural network chip to achieve real-time video output, greatly improving the efficiency and response speed of the imaging system.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that they can still modify or equivalently replace the technical solutions of the present invention, and these modifications or equivalent replacements do not make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A simple optical system design method based on consistency constraints of optical transfer functions, characterized in that: The following steps are involved: Step S1: designing a simple lens initial structure. The specific process is as follows: Step S11, calculating the lens sub-aperture diameter; Step S12, calculating the lens sub-aperture position; Step S13, setting the lens sub-aperture multiple structures: setting the parameters of each sub-aperture as a multiple structure in the optical design software to form a multiple structure optimization system; Step S14, setting an evaluation function for optimization: setting the lens parameters of the simple optical system and the distance parameters of the sub-aperture from the real aperture as variables, setting an evaluation function in the optical design software, taking minimization of the diffuse spot radius as an evaluation index, and constraining the focal length of the system at the same time; Step S2: Control the consistency of the optical transfer function of the simple lens. The specific process is as follows: Step S21, maintaining the evaluation function of step S1; Step S22: Calculate the average optical transfer function, average the optical transfer functions of all fields of view, and obtain the average optical transfer function , as shown below: ； in, is the optical transfer function of the ith field of view, N is the number of fields of view, is a simple optical system parameter; Step S23: Calculate the consistency of the optical transfer function. The sum of the absolute values ​​of the differences between the optical transfer function of each field of view and the average optical transfer function is defined as the consistency of the optical transfer function. , as shown below: ； Step S24: adding an evaluation function for optimization. Based on step S1, two operands are added to the evaluation function: One is to control the consistency of the optical transfer function , making it close to 0; the second is to control the minimum value of the optical transfer function.

2. The simple optical system design method based on optical transfer function consistency constraint according to claim 1, characterized in that: In step S11, the lens sub-aperture diameter is calculated, and the specific process is as follows: The diffraction-limited optical transfer function (OTF) corresponding to the subaperture at the Nyquist frequency is The value is between 0.1 and 0.2 as a guideline to determine the subaperture diameter The range is as follows: ； in, is the central wavelength, is the focal length of the system.

3. The simple optical system design method based on optical transfer function consistency constraint according to claim 1, characterized in that: In step S12, the lens sub-aperture position is calculated, and the specific process is as follows: Assuming that the projection of each subaperture related to the field of view on the real aperture is uniformly distributed, calculate the coordinate of the projected aperture in the y direction , as shown below: ； in, is the i-th field of view, is the total field of view, is the diameter of the true aperture; According to the trigonometric relationship, the distance between the sub-aperture and the real aperture is calculated. , as shown below: 。

Images