Free-form surface reflective imaging system generation method

Abstract

The invention provides a free-form surface reflective imaging system generation method, which adopts a method of combining supervised learning and unsupervised learning based on micro ray tracing to train and generate a DNN model of a free-form surface reflective imaging system. The network can generate an optical system conforming to a real optical design model, and the output system has good imaging performance and meets certain design constraint conditions at the same time; for a network obtained by training, system parameters, structure parameters and requirements for light path folding forms are input, and then surface positions and coefficients can be directly output, so that based on specific design requirements, single or multiple free-form surface systems with various different folding forms and different system and structure parameters are directly obtained, and the free-form surface systems can be directly obtained. The efficiency of free-form surface imaging optical design can be remarkably improved, and the labor cost is greatly reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical design, and particularly relates to a method for generating a free-form surface reflective imaging system. Background Art

[0002] Compared with traditional spherical and aspherical surfaces, free-form surfaces break the rotational symmetry, so they can provide higher aberration correction capabilities for asymmetric optical systems. Using free-form surfaces in an optical system can increase the design freedom of the system without increasing the number of optical elements, effectively improve the aberration correction ability of the optical system, and achieve the improvement of imaging quality and the simplification of the optical system structure. In the past few years, free-form surfaces have been continuously applied to various imaging optical systems, such as cameras, head-mounted displays, spectral imagers, and other imaging devices.

[0003] Free-form surfaces are of great significance in novel asymmetric imaging optics. However, the design of free-form surface imaging systems is a highly challenging task. Due to the lack of symmetry, traditional optical design methods are not applicable in the design of free-form surface systems, especially in the stage of constructing the starting point in the initial design. Without a good starting point, the design difficulty will be greatly increased, and the time cost spent will also increase significantly, especially for novice optical designers lacking design experience. To effectively construct the starting point, researchers have proposed some methods, such as the partial differential equation method (PDE), the multi-surface synchronous design method (SMS), and the point-by-point construction iteration method (CI). However, most methods have certain limitations on system parameters (such as the field of view angle) and system structure. In addition, the above methods can only provide a single solution for a specific design task, and this solution is not necessarily the optimal one. If multiple solutions are desired, the entire construction process needs to be repeated, so it is time-consuming and cumbersome. In recent years, deep learning has been successfully applied to different fields, such as computational imaging, metasurface design, and adaptive optics. Et al. obtained a lens database using deep learning, from which a high-quality design starting point for coaxial spherical objective lenses can be obtained. For free-form surface systems, Yang et al. proposed a framework for generating starting points of free-form surface imaging systems based on deep learning. This method was later extended by Chen et al. to apply to a wider range of system specifications. Although significant progress has been made in the automated design of imaging systems using deep learning, most existing methods rely on datasets, so the trained networks lack interpretability and physical meaning, especially with less connection to optical design models. In addition, most methods are limited to generating a single solution for a specific design task. To address this issue, researchers have developed new design frameworks. For example, Mao et al. proposed a more advanced deep learning framework that can generate multiple solutions with different structural parameters for a given input of a specific combination of system parameters. However, the above network models all have limitations, that is, for a specific design task, the optical path folding type of the system output by the model is fixed. For free-form surface imaging systems, especially for complex off-axis reflection systems, there are usually multiple different optical path folding methods. If only a system with a single folding form is generated for a specific system specification, the output system may not necessarily be optimal in terms of structure, image quality, etc., or may not meet the required structural requirements, so its application scenarios are limited. Summary of the Invention

[0004] To solve the above problems, the present invention provides a method for generating a free-form surface reflective imaging system, which can significantly improve the efficiency of free-form surface imaging optical design and greatly reduce the labor cost.

[0005] A method for generating a free-form surface reflective imaging system, which inputs the specific values of system parameters, structural parameters that meet the set requirements, and discriminant parameters corresponding to the optical path folding form into a trained DNN model, and the DNN model outputs a predicted free-form surface reflective imaging system that meets the set requirements;

[0006] wherein, the total loss function L used when training the DNN model total is:

[0007] L total = w data-driven × L data-driven + w design-model × L design-model

[0008] wherein, L data-driven is a data-driven loss function based on supervised learning, and L data-driven is the mean square error between the predicted free-form surface reflective imaging system output by the DNN model and the target free-form surface reflective imaging system, and w data-driven is L data-drivenWeight of; L design-model Is the loss function driven by the optical design model based on unsupervised learning, w design-model Is L design-model Weight of, and there is:

[0009]

[0010] Among them, Is the average imaging performance loss function related to the imaging performance of the predicted free-form surface reflective imaging system output by the DNN model, w image-quality Is Weight of, Is the average structural parameter consistency loss function related to the consistency between the structural parameters input to the DNN model and the structural parameters actually corresponding to the predicted free-form surface reflective imaging system output by the DNN model, w structure Is Weight of, Is the average design constraint loss function related to the constraints that the target free-form surface reflective imaging system needs to meet, w constraint Is Weight of.

[0011] Furthermore, the acquisition method of the training data set used when training the DNN model is:

[0012] S11: First, according to the actual design requirements, select multiple system parameters SYP = [SYP1, SYP2,..., SYP m ,..., SYP M , structural parameters STP = [STP1, STP2,..., STP n ,..., SYP N and surface parameters SUP = [SUP1, SUP2,..., SUP t ,..., SUP T that can completely describe the imaging system; at the same time, use a discrete discrimination parameter G to distinguish different optical path folding forms of the free-form surface reflective imaging system, and each optical path folding form corresponds to a specific G value;

[0013] S12: Determine the spatial range of the system parameters SYP, the spatial range of the structural parameters STP, and the value range of the discrimination parameter G; among them, the free-form surface reflective imaging systems with different optical path folding forms have different spatial ranges of the structural parameters STP, and all the spatial ranges of the system parameters SYP, the spatial ranges of the structural parameters STP, and the value ranges of the discrimination value G are combined to form the input parameter space IPS;

[0014] S13: Randomly sample within the input parameter space IPS to obtain multiple input parameter combinations INP = [SYP, STP, G];

[0015] S14: Use the pre-obstruction evaluation method to perform pre-obstruction determination on all combinations of input parameters, eliminate the combinations of input parameters that do not meet the obstruction requirements, and obtain the final combination of input parameters INP*;

[0016] S15: Generate a free-form surface reflective imaging system corresponding to the combination of input parameters INP*, and obtain the system parameters SYP, structural parameters STP, discrimination parameters G, and surface parameters SUP actually corresponding to the generated free-form surface reflective imaging system as the training data set; among them, use the system parameters SYP, structural parameters STP, and discrimination parameters G as the input parameters of the DNN model, and use the structural parameters STP and surface parameters SUP as the output parameters of the DNN model.

[0017] Furthermore, in step S14, the method for performing pre-obstruction determination on any one input parameter using the pre-obstruction evaluation method is specifically as follows:

[0018] According to the system parameters and structural parameters in the current input parameters, solve all the spherical curvatures of the equivalent coaxial spherical system corresponding to the current input parameters, and respectively use each spherical curvature as the mirror curvature combination value of the free-form surface reflective imaging system corresponding to the current input parameters;

[0019] According to the obtained mirror curvature combination value and the system parameters and structural parameters in the current input parameters, use Gaussian optics and geometric knowledge to calculate the clear aperture of all the mirrors and the image plane of the free-form surface reflective imaging system corresponding to the current input parameters;

[0020] According to the positions of the mirrors, the clear apertures of the mirrors and the image plane of the free-form surface reflective imaging system corresponding to the current input parameters, calculate the distance from the edge point of each mirror or the image plane to the edge of the beam that has not been reflected or passed through itself. If any one of the distances is less than the preset threshold, it is determined that the obstruction is too large, and the current input parameters are not included in the final combination of input parameters INP*.

[0021] Furthermore, the method for generating a free-form surface reflective imaging system corresponding to any one input parameter in the combination of input parameters INP* is as follows:

[0022] Obtain the surface curvatures of each component of the equivalent coaxial spherical system corresponding to the current input parameters;

[0023] Take the surface curvature of each element of the calculated equivalent coaxial spherical system as the actual surface curvature of the corresponding off-axis reflective spherical system, and set the system parameters included in the current input parameters as the actual system parameter values of the off-axis spherical reflective system. Set all the element positions given in the structural parameters included in the current input parameters as the actual element coordinates of the off-axis spherical reflective system, thereby establishing an initial off-axis reflective spherical system;

[0024] Change the surface type of each element of the initial off-axis reflective spherical system to a free-form surface to obtain an initial free-form surface reflective imaging system;

[0025] Control the effective focal length of the initial free-form surface reflective imaging system by the ABCD matrix method, control the distortion of the initial free-form surface reflective imaging system by the ray tracing method, and control the vignetting of the initial free-form surface reflective imaging system by the distance from the edge points of each mirror or image plane to the edge of the beam that is not reflected or passed through from the mirror or image plane itself. Perform multi-parameter optimization on the initial free-form surface reflective imaging system until the imaging quality of the system meets the set requirements and there are no ray tracing errors. Then, take the free-form surface reflective imaging system at this time as the finally generated free-form surface reflective imaging system.

[0026] Further, the system parameters include the field of view FOV, effective focal length EFL, entrance pupil diameter ENPD, and F number;

[0027] The structural parameters include the position and inclination angle of each free-form surface and the image plane in the free-form surface reflective imaging system.

[0028] Further, the constraints that the target free-form surface reflective imaging system needs to meet include focal length constraint, volume constraint, vignetting constraint, and intersection coordinate constraint of the chief ray of the system center field of view with each surface.

[0029] Further, when unique values are set for all system parameters, structural parameters, and discriminant parameters, the DNN model outputs a specific free-form surface reflective imaging system that satisfies the target optical path folding form, system parameters, and structural parameters;

[0030] When unique values are set for all system parameters and structural parameters, and all possible values are set for the discriminant parameters, the DNN model outputs multiple free-form surface reflective imaging systems with different optical path folding forms, and each free-form surface reflective imaging system satisfies the target system parameters and structural parameters;

[0031] When only unique values are set for some system parameters and some structural parameters, while value ranges are set for the remaining system parameters and structural parameters and all possible values are set for the discriminant parameters, the DNN model outputs multiple free-form surface reflective imaging systems with different optical path folding forms, different system parameters, and different structural parameters, realizing the generation of multiple folding forms and multiple structures of the free-form surface reflective imaging system.

[0032] Furthermore, when the free-form surface reflective imaging system to be generated is a free-form surface off-axis three-mirror imaging system without an intermediate image, the determination methods for the system parameters, structural parameters, and discriminant parameters input into the DNN model are as follows:

[0033] Assume that the free-form surface off-axis three-mirror imaging system includes a primary mirror M1, a secondary mirror M2, and a tertiary mirror M3, and the system is symmetric about the YOZ plane;

[0034] Select the half field of view HXFOV in the x direction, the half field of view HYFOV in the y direction, and the entrance pupil diameter ENPD as the system parameters, denoted as SYP = [HXFOV, HYFOV, ENPD];

[0035] Select the inclination angles of all surfaces and the y and z coordinates of the surface vertices as the structural parameters, denoted as STP = [M1 y , M1 z , M3 y , M3 z , IMG y , IMG z , M1 tilt , M2 tilt , M3 tilt , IMG tilt , where M1 y and M1 z are the y and z coordinates of the primary mirror M1 respectively, M3 y and M3 z are the y and z coordinates of the tertiary mirror M3 respectively, IMG y and IMG z are the y and z coordinates of the image plane IMG respectively, M1 tilt is the inclination angle of the primary mirror M1, M2 tilt is the inclination angle of the secondary mirror M2, M3 tilt is the inclination angle of the tertiary mirror M3, IMG tilt is the inclination angle of the image plane IMG;

[0036] There are 8 optical path folding forms corresponding to the free-form surface off-axis three-mirror imaging system, so the value range of G is G = 1, G = 2,..., G = 8.

[0037] Beneficial effects:

[0038] 1. The present invention provides a method for generating a free-form surface reflective imaging system. A DNN model for generating a free-form surface reflective imaging system is trained by combining supervised learning with unsupervised learning based on differentiable ray tracing. This way of introducing an optical design model-driven loss for joint training enables the network to generate an optical system that conforms to a real optical design model. The output system has good imaging performance and meets certain design constraint conditions. For the trained network, after inputting system parameters, structural parameters, and requirements for the optical path folding form, the surface positions and coefficients can be directly output, so as to directly obtain a single or multiple free-form surface systems with various different folding forms and different system and structural parameters based on specific design requirements, which can significantly improve the efficiency of free-form surface imaging optical design and greatly reduce labor costs.

[0039] 2. The present invention provides a method for generating a free-form surface reflective imaging system. For training a deep neural network model with multi-dimensional input parameters, a good data set is crucial. This data set should include high-performance free-form surface systems with various optical path folding forms, system parameters, and structural parameter values. Based on this, the present invention adopts a method for automatically generating a data set, which can randomly sample different parameter combinations in a high-dimensional parameter space and automatically generate corresponding free-form surface imaging systems. According to the obtained data set, a data-driven loss function can be established to support the effective learning of the deep learning model.

[0040] 3. The present invention provides a method for generating a free-form surface reflective imaging system. A training data set containing high-performance free-form surface imaging systems is obtained through an automated data set generation method. A data-driven loss function is established through the data set for the initial training of the network. A differentiable ray tracing is used to calculate an optical design model-driven loss function regarding imaging performance and design constraints. The network is jointly trained by combining the two loss functions (the data-driven loss function and the optical design model-driven loss function).

[0041] 4. The present invention provides a method for generating a free-form surface reflective imaging system. If all input parameter values are defined, the trained model can output a specific system that meets the target optical path folding form, system parameters, and structural parameters. If there are no requirements for the optical path folding form, the trained model can generate multiple systems with various optical path folding forms that meet the design requirements (system parameters and structural parameters). Designers can evaluate the image quality of the output systems or perform rapid automatic optimization as needed, and sort and screen these systems to select a suitable system solution as a good starting point for further optimization. In addition, the network can also be integrated into existing optical design software and cloud servers for the convenience of more designers to use.

[0042] 5. The present invention provides a method for generating a free-form surface reflective imaging system. For the system output by the network, it can be selected for rapid optimization to further obtain a system with better performance. These systems can also be sorted and screened according to preset evaluation indicators, thereby significantly reducing time and labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of automated dataset generation provided by the present invention.

[0044] Figure 2 It is a flowchart of the method for jointly training a neural network by combining a data-driven loss function and an optical design model-driven loss function provided by the present invention.

[0045] Figure 3 It is a schematic diagram of the pre-evaluation blocking method for the design example provided by the present invention.

[0046] Figure 4 It is a schematic diagram of the free-form surface off-axis three-mirror imaging system and the coaxial equivalent system of the design example provided by the present invention.

[0047] Figure 5 It is an optical path diagram of a typical network prediction system when the first set of system parameters is given provided by the present invention.

[0048] Figure 6 It is an optical path diagram of a typical network prediction system when the second set of system parameters is given provided by the present invention.

[0049] Figure 7 It is an optical path diagram of a typical network prediction system and the subsequent optimized system when the third set of system parameters is given provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0051] The present invention proposes a method for generating a free-form surface reflective imaging system, which is applicable to various free-form surface imaging systems with advanced system specifications. The supervised learning based on data-driven loss enables the network to have a certain prediction ability. Then, a method combining supervised learning with unsupervised learning based on differentiable ray tracing is adopted to train the network, and an optical design model-driven loss is introduced for joint training, so that the network can generate an optical system that conforms to the real optical design model. The output system has good imaging performance and meets certain design constraints. After training, based on the input design requirements, the network can immediately output one or more free-form surface systems with different optical path folding forms, different system parameters and structural parameters. The output system can be used as a good starting point for subsequent rapid optimization, or can be used by designers for further ranking and screening. This significantly improves the efficiency of free-form surface imaging optical design and greatly reduces the required labor cost.

[0052] Specifically, the present invention provides a method for generating a free-form surface reflective imaging system. The specific values of system parameters, structural parameters, and discriminant parameters corresponding to the optical path folding form that meet the set requirements are input into the trained DNN model, and the DNN model outputs a predicted free-form surface reflective imaging system that meets the set requirements.

[0053] The following details the training process of the DNN model, including the following steps:

[0054] S1: Obtain a training data set according to the following steps:

[0055] S11: First, according to the actual design requirements, select multiple representative system parameters SYP = [SYP1, SYP2,..., SYP m ,..., SYP M , structural parameters STP = [STP1, STP2,..., STP n ,..., SYP N , and surface parameters SUP = [SUP1, SUP2,..., SUP t ,..., SUP T that can completely describe the imaging system; at the same time, use a discrete discriminant parameter G to distinguish different optical path folding forms of the free-form surface reflective imaging system. Each optical path folding form corresponds to a specific G value, and G will be treated as a classification label in subsequent processing;

[0056] It should be noted that for a free-form surface imaging optical system, representative system parameters include the field of view (FOV), effective focal length (EFL), entrance pupil diameter (ENPD), and F-number (F#). These parameters are not completely independent. Some parameters can be fixed at specific values, and systems with other parameter values can be obtained by scaling. Representative structural parameters include the position and inclination of each free-form surface in the system, and the surface parameters include the coefficients of the mathematical description of the free-form surface. The type of surface is not restricted here.

[0057] Meanwhile, according to different optical path folding forms involved in the design task, all the values of G are determined. For example, for an off-axis three-mirror system without an intermediate image with 8 optical path folding forms, each folding form corresponds to a specific value of G. Therefore, the value range of G is G = 1, G = 2, …, G = 8.

[0058] S12: Determine the spatial range of the system parameter SYP, the spatial range of the structural parameter STP, and the value range of the discrimination parameter G; among them, free-form surface reflective imaging systems with different optical path folding forms have different spatial ranges of the structural parameter STP. All the spatial ranges of the system parameter SYP, the spatial ranges of the structural parameter STP, and the value ranges of the discrimination value G are combined to form the input parameter space IPS;

[0059] S13: Randomly sample within the input parameter space IPS to obtain multiple input parameter combinations INP = [SYP, STP, G];

[0060] S14: Use the pre-obstruction evaluation method to perform pre-obstruction determination on all the input parameter combinations, eliminate the input parameter combinations that do not meet the obstruction requirements, and obtain the final input parameter combination INP*;

[0061] That is to say, the present invention uses the pre-obstruction evaluation method to perform pre-obstruction determination on all the input parameter combinations; if it is determined that there is an obstruction that is difficult to eliminate in a certain parameter combination, the parameter combination is screened out. Only the parameter combinations determined to have no obstruction or only a small obstruction can be left, and finally a specified number of INP are obtained;

[0062] Specifically, a specific input parameter combination INP corresponds to a free-form surface system with specific system parameters, structural parameters, and optical path folding forms; the present invention uses the pre-obstruction evaluation method to perform pre-obstruction determination on any input parameter INP i (1 ≤ i ≤ N, N is the total number of INP) The method for pre-obstruction determination is specifically as follows:

[0063] According to the system parameters and structural parameters in the current input parameters, solve all the spherical curvatures of the equivalent coaxial spherical system corresponding to the current input parameters, and use each spherical curvature as the mirror curvature combination value of the free-form surface reflective imaging system corresponding to the current input parameters; according to the obtained mirror curvature combination value and the system parameters and structural parameters in the current input parameters, use Gaussian optics and geometric knowledge to calculate the clear aperture of all the mirrors and the image plane of the free-form surface reflective imaging system corresponding to the current input parameters; according to the positions of the mirrors, and the clear apertures of the mirrors and the image plane of the free-form surface reflective imaging system corresponding to the current input parameters, calculate the distance from the edge point of each mirror or the image plane to the edge of the beam that has not been reflected or passed through itself. If any one of the distances is less than the preset threshold, it is determined that the obstruction is too large, and the current input parameters are not included in the final input parameter combination INP*. The whole process does not involve the actual system generation, and only needs to complete the pre-obstruction evaluation process through formula derivation and calculation.

[0064] S15: Generate a free-form surface reflective imaging system corresponding to the input parameter combination INP*, and obtain the system parameters SYP, structural parameters STP, discriminant parameter G, and surface parameters SUP actually corresponding to the generated free-form surface reflective imaging system as the training data set; among them, use the system parameters SYP, structural parameters STP, and discriminant parameter G as the input parameters of the DNN model, and use the structural parameters STP and surface parameters SUP as the output parameters of the DNN model.

[0065] Specifically, for any one input parameter INP i (1 ≤ i ≤ N, N is the total number of INPs) in the input parameter combination INP*, the generation method of the corresponding free-form surface reflective imaging system is as follows:

[0066] Obtain the surface curvatures of each component of the equivalent coaxial spherical system corresponding to the current input parameters;

[0067] Use the calculated surface curvatures of each component of the equivalent coaxial spherical system as the actual surface curvatures of the corresponding off-axis reflective spherical system, and set the system parameters included in the current input parameters as the actual system parameter values of the off-axis spherical reflective system, and set all the given component positions in the structural parameters included in the current input parameters as the actual component coordinates of the off-axis spherical reflective system, thereby establishing an initial off-axis reflective spherical system;

[0068] Change the surface types of each component of the initial off-axis reflective spherical system to free-form surfaces to obtain an initial free-form surface reflective imaging system;

[0069] Control the effective focal length of the initial free-form surface reflective imaging system through the ABCD matrix method, control the distortion of the initial free-form surface reflective imaging system through the ray tracing method, control the vignetting of the initial free-form surface reflective imaging system by the distance from the edge points of each mirror or image plane to the edge of the light beam that is not reflected or passed through from the mirror or image plane itself, and optimize multiple parameters of the initial free-form surface reflective imaging system until the imaging quality of the system meets the set requirements and there are no ray tracing errors. Then, use the free-form surface reflective imaging system at this time as the finally generated free-form surface reflective imaging system.

[0070] It should be noted that the generation process of each free-form surface reflective imaging system is independent of each other, and multiple free-form surface reflective imaging systems can be generated in parallel.

[0071] S2: Perform supervised learning on the neural network according to the following steps:

[0072] S21: Input the input parameter combinations in the dataset into the deep neural network DNN to obtain the predicted output of the network. By calculating the difference between the output value and the true value in the dataset, establish a data-driven loss function L data-driven , and then use L data-driven for the supervised learning of the neural network;

[0073] Furthermore, when training the neural network, it is necessary to preprocess the training dataset, including removing abnormal data and normalizing it to ensure that all data with the same parameters are at the same scale; the data-driven loss function Ldata-driven of the supervised learning can be defined as the mean square error between the predicted output and the target output.

[0074] S3: After performing supervised training using the data-driven loss function for a period of time, introduce the optical design model-driven loss function L design-model for unsupervised training, combine supervised learning and unsupervised learning (combine the data-driven loss function and the optical design model-driven loss function) for subsequent neural network training, and finally obtain a DNN model with good performance;

[0075] Furthermore, the unsupervised learning method is as follows:

[0076] Randomly obtain different input parameter combinations in the IPS, normalize them, input them into the DNN neural network and predict the output system, and then trace the system output by the network through the differentiable ray tracing module to construct the optical design model-driven loss function L design-model . For a single system, its unsupervised loss consists of three parts. The first part is related to the imaging performance of the system (such as the diameter of the blur spot, distortion, etc.), which is expressed as L image-quality; The second part of the loss is related to the consistency of the input and output structure parameter values, usually measured by calculating the mean square error between the two, denoted as L structure ; The third part of the loss is related to the design constraints that need to be satisfied (such as constraints on the system focal length, volume, obscuration, etc.), and it is a penalty function based on, for example, quadratic functions, reciprocals, logarithmic functions, high-order power functions, etc., denoted as L constraint . After calculating the loss L i image-quality , L i sturcture , L i constraint for each system, the average imaging performance loss is obtained by summing the losses of the same type in all output systems and then dividing by the total number of systems Average structural parameter consistency loss and average design constraint loss The calculation formulas are as follows:

[0077]

[0078] Adding the above average losses according to the weights, the final optical design model-driven loss function L design-model is obtained, and the calculation formula is as follows:

[0079]

[0080] where w image-quality , w structure and w constraint are weight adjustment factors used to balance the contributions of different losses. The entire prediction and calculation process is completely differentiable, so that the gradients of the loss function L design-model with respect to each parameter in the neural network can be calculated and backpropagated, and the network parameters can be updated in cooperation with the optimization method to improve the performance of the network.

[0081] Furthermore, the calculation formula for the total training loss combining supervised learning and unsupervised learning is as follows:

[0082] L total = w data-driven L data-driven + w design-model L design-model

[0083] By adjusting the weights w data-driven and w design-model respectively, the contributions of the data-driven loss function and the optical design model loss function to the overall network training can be adjusted. Based on L total for network training, a DNN model with good performance can be finally obtained.

[0084] S4: After training is completed, the network can be used to generate the required free-form surface imaging system. By inputting the design requirements (including system parameters, structural parameters, and requirements for the optical path folding form) into the trained network model, one or more systems that meet the requirements can be obtained. These systems can serve as a good starting point for further optimization or can generate better output systems through rapid optimization. These output systems can be evaluated according to one or more specific criteria and then sorted and filtered according to the designer's needs for the designer to use.

[0085] Furthermore, the specific process of obtaining one or more systems that meet the design requirements through the trained DNN model is as follows:

[0086] When unique values are set for all system parameters, structural parameters, and discriminant parameters, the DNN model outputs a specific free-form surface reflective imaging system that meets the target optical path folding form, system parameters, and structural parameters;

[0087] When unique values are set for all system parameters and structural parameters, and all possible values are set for the discriminant parameters (i.e., there is no requirement for the optical path folding form), the DNN model outputs multiple free-form surface reflective imaging systems with different optical path folding forms, and each free-form surface reflective imaging system meets the target system parameters and structural parameters;

[0088] When unique values are set only for some system parameters and some structural parameters, value ranges are set for the remaining system parameters and structural parameters, and all possible values are set for the discriminant parameters, the DNN model outputs multiple free-form surface reflective imaging systems with different optical path folding forms, different system parameters, and different structural parameters, realizing the generation of multiple folding forms and multiple structures of the free-form surface reflective imaging system. It should be noted that the remaining system parameters and structural parameters without fixed values can be randomly assigned values within the corresponding parameter ranges.

[0089] The feasibility of this design method and the effectiveness of the trained DNN model are verified below through the design of a free-form surface off-axis three-mirror imaging system. The training and application process of the DNN model for the off-axis three-mirror free-form surface imaging system with multiple folding forms is described below with reference to the accompanying drawings.

[0090] The central field of view of the selected free-form surface off-axis three-mirror imaging system is (0°, 0°), symmetric about the YOZ plane, and 8 off-axis three-mirror optical path folding forms without intermediate images inside the system are considered. M1, M2, M3, and IMG represent the primary mirror, secondary mirror, tertiary mirror, and image plane respectively, and the aperture stop is located at M2. See Figure 1, first, select representative system parameters and structural parameters. Select the half field of view in the x direction (HXFOV), the half field of view in the y direction (HYFOV), and the entrance pupil diameter (ENPD) as the representative system parameters, denoted as SYP = [HXFOV, HYFOV, ENPD]. The focal length is fixed at 1 mm, and systems with other focal length values can be obtained by scaling. The structural parameters include the inclination angles of all surfaces in the system and the y and z coordinates of the surface vertices. STP = [M1 y ,M1 z ,M3 y ,M3 z ,IMG y ,IMG z ,M1 tilt ,M2 tilt ,M3 tilt ,IMG tilt (M2 y ,M2 z , fixed as the origin of the global coordinate system. In addition, in this example, the chief ray of the system's central field of view intersects with the surface vertex, and the surface inclination angle can be solved by the law of reflection. Therefore, it is not used as an input structural parameter but only as an output structural parameter). At the same time, it is also necessary to determine the surface type and surface parameters. All mirrors adopt the 6th-order XY free-form surface type, which does not contain any spherical and conic terms, nor odd-order terms of x. Therefore, SUP contains a total of 42 parameters. Generally speaking, SYP, STP, and G are used as the inputs of the network together, with a total of 10 inputs; STP and SUP are used as the outputs of the network, with a total of 52 outputs.

[0091] Subsequently, determine the value ranges of the system parameter space, the structural parameter space, and the discriminant parameter. The set system parameter space is as follows: 3° ≤ Half-XFOV ≤ 7.5°, 3° ≤ Half-YFOV ≤ 7.5°, 1 / 6 mm ≤ ENPD ≤ 2 / 3 mm (the corresponding F-number range is from 1.5 to 6). Different structural parameter spaces are divided for off-axis three-mirror systems with different folding forms. For off-axis three-mirror systems with 8 folding forms, the discriminant parameter G takes values from 1 to 8 respectively.

[0092] The next step is to sample multiple input parameter combinations INP in the system parameter and structural parameter spaces for different folding forms. Some input parameter combinations will result in infeasible structures with obstructions that are difficult to eliminate. Use the pre-obstruction evaluation method to screen all parameter combinations. For off-axis three-mirror systems with 8 different foldings, there are multiple regions where light obstruction may occur. Here, the distance from the edge points of a specific mirror or image plane to the edge rays of a specific beam is used to determine whether there is obstruction in a specific region. For a specific illustration, refer to Figure 3 , the L marked in the figure i(1 ≤ i ≤ 8) represent different shielding distances. All shielding distances are distance values with positive and negative signs, and the positive and negative signs depend on the relative positional relationship between the point and the line. In the local coordinate system, when the point is above the line, the value is positive; otherwise, it is negative. To calculate all the shielding distances, the coordinate positions of each surface and the clear aperture need to be obtained.

[0093] The value of INP is denoted as [ω x , ω y , D, y M1 , z M1 , y M3 , z M3 , y IM , z IM , g], where ω x and ω y represent the values of HXFOV and HYFOV respectively, D represents ENPD, and the positions of the vertices of each mirror and the image plane in the global coordinate system are (y M1 , z M1 ), (0, 0), (y M3 , z M3 ), and (y IM , z IM ), and g represents the value of G. The system parameters of the coaxial equivalent system are the same as those of the off-axis system. First, calculate all the curvature values of the coaxial equivalent system. For the specific parameter definitions, refer to Figure 3 . In the coaxial equivalent system, the interval between adjacent elements can be calculated as The curvature of the mirrors in the coaxial system can be obtained based on Gaussian optical theory and third-order aberration theory. The calculation formula is as follows:

[0094]

[0095] Among them, EFL is fixed at 1, so the optical power Φ of the system is 1. Solving the above equations can obtain the curvatures c1, c2, and c3 of M1, M2, and M3. Then, the focal length of M1 can be calculated according to f1' = 1 / (2c1). For the central field of view of (0°, 0°), the beam aperture of M1 is equal to D. According to Figure 3 , by the similar triangle theorem, D2 / D = (f1' - d1) / f1'. Therefore, the full-field clear aperture of M2 is: D2 = D(1 - d1 / f1'). The full-field clear apertures D1 and D3 of M1 and M3 are determined by the heights of the upper and lower marginal rays of the upper marginal field angle ω y and the lower marginal field angle -ω y on the mirror surface. First, for the field of view -ω yRay tracing is performed on the upper edge ray, which intersects the optical axis multiple times, and these intersection points can be regarded as conjugate points of each other. For M1, its "object distance" and "image distance" are denoted as l1 and l1′ respectively, satisfying 1 / l1′ - 1 / l1 = 1 / f1′. The projection height of the ray on M1 can be expressed as -h1 = l1tan(-ω y ). The object distance l1 of the ray with respect to M1 can be calculated as: l1 = f1′h1 / (h1 + f1′tanω y ). Using Figure 3 geometric relationship, we can get l1′ / (d1 - l1′) = -2h1 / D2. Therefore, the projection height of the upper edge ray of the marginal field -ω y with respect to M1 can be obtained as: h1 = (f1′D2 + 2d1f1′tanω y ). Therefore, the full field of view clear aperture of M1 is: D1 = 2(D - h1). The next step is to calculate the full field of view clear aperture of M3. Similarly, ray tracing is performed on the upper edge ray of the field angle -ω y . For the object distance of this ray with respect to M2, there is a relationship l2 = d1 - l1′. The focal length of M2 can be calculated according to f2′ = 1 / (2c2). Similarly, the object distance and image distance of M2 satisfy 1 / l2′ - 1 / l2 = 1 / f2′. Therefore, the image distance of the ray with respect to M2 can be calculated as l2′ = l2 f2′ / (l2 + f2′). From the similar triangle theorem, we can get D3 / D2 = (l2′ - d2) / l2′. Therefore, the full field of view clear aperture of M3 is D3 = D2(l2′ - d2) / l2′. In addition, the full field of view clear aperture of the image plane is D IM = 2tanω y . In summary, the full field of view clear apertures of all the mirrors (M1, M2, and M3) and the image plane (IMG) of the coaxial equivalent spherical system can be calculated by formulas. These aperture sizes can be used as the clear apertures of each mirror and the image plane in the off-axis system. Then, according to the given surface positions and the calculated surface clear apertures, calculate Figure 3 the obscuration distance marked in it, and set the maximum allowable obscuration distance to 0.05 mm. For each INP, if the calculated obscuration value is less than this allowable value, it is determined that this combination has a relatively serious obscuration, and it is deleted from the training dataset. After screening, 5000 input parameter combinations are retained for each folding form, and a total of 40000 corresponding free-form surface systems need to be generated.

[0096] Next, use the method described previously to generate a training dataset. For each INP, first calculate an off-axis spherical system, and then upgrade the surface type of each sphere in the system to an XY free-form surface polynomial up to the 6th order. Next, perform multi-parameter optimization on each system in the optical design software to obtain good imaging quality. Finally, evaluate the imaging performance of the obtained system, and for systems with poor image quality or ray tracing errors, remove them from the dataset. Thus, a complete training dataset is obtained, which contains the system parameters, structural parameters, surface parameters, and discrimination parameters of 39,856 systems in total.

[0097] Next, use the data-driven loss to train the network model. The DNN used in this embodiment has 24 hidden layers. The number of nodes in the hidden layers is gradually increased in the first few layers and then symmetrically decreased in the last few layers, where the number of nodes in the largest layer reaches 360. The network uses the tanh activation function and the Adam optimizer. The learning rate of the optimizer is 1e-4, and the size of one training batch is 1000. In the initial training stage, a total of 216,000 epochs are carried out, and the loss value reaches 7.3×10 -3 .

[0098] Subsequently, combine the data-driven loss with the optical design model-driven loss for training. For the unsupervised learning part, 2000 input parameter combinations are randomly generated for each folding form, and a total of 16,000 fixed inputs are used for unsupervised training. The weight of the data-driven loss is 0.5, and the weight of the optical design model-driven loss is set to 5. At this time, the learning rate is adjusted to 6×10 -5 , and the batch size of the unsupervised training part is 200. This combined training process is carried out for a total of 94 epochs.

[0099] For the already trained DNN network model, use three different input cases to evaluate the prediction effect of the network. In each case, 1600 different inputs are tested respectively. Among them, the predicted systems in the first two cases are not further optimized, and the systems in the third case are quickly optimized after prediction.

[0100] In the first case, all system parameters are given as: HXFOV = 5°, HYFOV = 4° and ENPD = 0.56 mm. The structural parameter values of 8 optical path folding forms are randomly generated in their respective structural parameter spaces. After pre-obscuration evaluation and screening, 200 input parameter combinations are left for each folding form and input into the network. Among the 1600 predicted systems output, 58 systems have obscuration, 5 systems have ray tracing errors, and the average RMS spot diameter of the other 1595 normal systems is 0.0074 mm. Except for a very small number of systems with relatively large aberrations, the output systems can be used as a good starting point for further optimization. SeeFigure 5 , showing 40 typical prediction systems. Each row in the figure represents a system with different structural parameters under the same optical path folding form. The first system in each row has the smallest RMS spot diameter under this folding form.

[0101] In the second case, the given system parameters are HXFOV = 7°, HYFOV = 4° and ENPD = 0.18 mm. Similar to the previous case, all structural parameters are randomly generated within their respective structural parameter spaces. After screening, 1600 INPs are obtained, with 200 for each folding form. Among the systems directly predicted by the network, 6 systems have ray tracing errors and 13 systems have ray blocking. 40 typical systems with different folding forms and structural parameters are shown in Figure 6 .

[0102] In the third case, the given system parameters are HXFOV = 4°, HYFOV = 3° and ENPD = 0.33 mm. The values of all structural parameters are randomly selected, and 1600 INPs are obtained after blocking evaluation. For all prediction systems in this case, perhaps rapid further optimization (taking 1 to 2 seconds for each system) is carried out to obtain good imaging quality for direct use. Similarly, Figure 7 40 typical prediction systems are given.

[0103] It can be seen that the method for generating a free-form surface reflective imaging system provided by the present invention has the following advantages:

[0104] First, within the given system parameter range, it can automatically generate a large number of training data sets with good imaging quality, and use the method of automatic blocking evaluation to screen the data sets, ensuring the feasibility of the imaging system.

[0105] Second, by introducing unsupervised learning based on differentiable ray tracing and combining the optical design model-driven loss with the data-driven loss, it significantly reduces the dependence on a large-scale high-quality data set, so that in the supervised learning stage, only a relatively small-scale initial data set is needed to ensure that the network has good prediction ability in the follow-up.

[0106] Third, after training, after the network inputs the system parameters, structural parameters and optical path folding form requirements, it can quickly generate multiple free-form surface imaging systems, and these systems can be quickly sorted, screened or further optimized according to the preset indicators, greatly reducing the manpower and time investment of the designer.

[0107] Fourth, this method has strong versatility and scalability, is applicable to the free-form surface imaging design of various advanced system parameters and complex structures, and provides an effective guarantee for the design of optical systems with multiple optical path foldings and off-axis structures.

[0108] Fifthly, this method greatly reduces the time and labor costs of designers. At the same time, this method can also be integrated into existing optical design software and cloud servers, making it convenient for more designers to use.

[0109] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can certainly make various corresponding changes and modifications according to the present invention. However, these corresponding changes and modifications should all fall within the protection scope of the claims appended to the present invention.

Claims

1. A method for generating a free-form surface reflective imaging system, characterized in that: The specific values ​​of the system parameters, structural parameters and discrimination parameters corresponding to the optical path folding form that meet the set requirements are input into the trained DNN model, and the DNN model outputs a predicted free-form surface reflective imaging system that meets the set requirements; Among them, the total loss function L used when training the DNN model total for: L total =w data-driven ×L data-driven +w design-model ×L design-model Among them, L data-driven is a data-driven loss function based on supervised learning, and L data-driven is the mean square error between the predicted free-form surface reflective imaging system output by the DNN model and the target free-form surface reflective imaging system, w data-driven For L data-driven The weight of L design-model The loss function for the optical design model based on unsupervised learning, w design-model For L design-model The weight of , and: in, is the average imaging performance loss function related to the imaging performance of the free-form surface reflective imaging system predicted by the DNN model output, w image-quality for The weight of is the average structural parameter consistency loss function related to the consistency between the structural parameters of the input DNN model and the structural parameters of the predicted free-form surface reflective imaging system output by the DNN model, w structure for The weight of is the average design constraint loss function related to the constraints that the target free-form surface reflective imaging system needs to satisfy, w constraint for The weight of .

2. A method for generating a free-form surface reflective imaging system according to claim 1, characterized in that: The method for obtaining the training data set used when training the DNN model is: S11: First, according to the actual design requirements, select multiple system parameters SYP that can fully describe the imaging system = [SYP1, SYP2, ..., SYP m ,…,SYP M ]、Structural parameter STP=[STP1,STP2,…,STP n ,…,SYP N ] and surface parameters SUP=[SUP1,SUP2,…,SUP t ,…,SUP T ]; At the same time, a discrete discrimination parameter G is used to distinguish different optical path folding forms of the free-form surface reflective imaging system, and each optical path folding form corresponds to a specific G value; S12: Determine the spatial range of the system parameter SYP, the spatial range of the structural parameter STP, and the value range of the discrimination parameter G; wherein, the free-form surface reflection imaging systems with different optical path folding forms have different structural parameter STP spatial ranges, and all the system parameter SYP spatial ranges, structural parameter STP spatial ranges, and discrimination value G value ranges are combined together to form an input parameter space IPS; S13: Randomly sample in the input parameter space IPS to obtain multiple input parameter combinations INP = [SYP, STP, G]; S14: using a pre-obstruction evaluation method to pre-determine obstruction for all input parameter combinations, eliminating input parameter combinations that do not meet the obstruction requirements, and obtaining a final input parameter combination INP*; S15: Generate a free-form surface reflective imaging system corresponding to the input parameter combination INP*, and obtain the system parameters SYP, structural parameters STP, discrimination parameters G and surface parameters SUP actually corresponding to the generated free-form surface reflective imaging system as a training data set; wherein the system parameters SYP, structural parameters STP and discrimination parameters G are used as input parameters of the DNN model, and the structural parameters STP and surface parameters SUP are used as output parameters of the DNN model.

3. A method for generating a free-form surface reflective imaging system according to claim 2, characterized in that: In step S14, the method of using the pre-occlusion evaluation method to pre-determine the occlusion of any input parameter is specifically as follows: According to the system parameters and structural parameters in the current input parameters, all spherical curvatures of the equivalent coaxial spherical system corresponding to the current input parameters are solved, and each spherical curvature is used as the combined value of the reflector curvature of the free-form surface reflective imaging system corresponding to the current input parameters; According to the obtained combined value of the curvature of the reflector and the system parameters and structural parameters in the current input parameters, the Gaussian optics and geometric knowledge are used to calculate the clear apertures of all the reflectors and the image plane of the free-form surface reflective imaging system corresponding to the current input parameters; According to the reflector position of the free-form surface reflective imaging system, the reflector and the aperture of the image plane corresponding to the current input parameters, the distance from the edge point of each reflector or image plane to the edge of the light beam that is not reflected from or passes through itself is calculated. If any distance is less than the preset threshold, it is determined that the obstruction is too large and the current input parameters are not included in the final input parameter combination INP*.

4. The method for generating a free-form surface reflective imaging system according to claim 2, characterized in that: The method for generating a free-form surface reflective imaging system corresponding to any input parameter in the input parameter combination INP* is: Get the surface curvature of each element of the equivalent coaxial spherical system corresponding to the current input parameters; The calculated surface curvature of each element of the equivalent coaxial spherical system is used as the actual surface curvature of the element of the corresponding off-axis reflection spherical system, and the system parameters contained in the current input parameters are set as the actual system parameter values ​​of the off-axis spherical reflection system, and all element positions given in the structural parameters contained in the current input parameters are set as the actual element coordinates of the off-axis spherical reflection system, thereby establishing an initial off-axis reflection spherical system; The surface type of each element of the initial off-axis reflection spherical system is changed into a free-form surface to obtain an initial free-form surface reflection imaging system; The effective focal length of the initial free-form surface reflective imaging system is controlled by the ABCD matrix method, the distortion of the initial free-form surface reflective imaging system is controlled by the ray tracing method, the occlusion of the initial free-form surface reflective imaging system is controlled by the distance from the edge point of each reflector or image plane to the edge of the light beam that is not reflected from or passes through each reflector or image plane itself, and the multi-parameter optimization of the initial free-form surface reflective imaging system is performed until the imaging quality of the system meets the set requirements and there is no ray tracing error, and the free-form surface reflective imaging system at this time is used as the final generated free-form surface reflective imaging system.

5. The method for generating a free-form surface reflective imaging system according to claim 1, characterized in that: The system parameters include field of view angle FOV, effective focal length EFL, entrance pupil diameter ENPD and F number; The structural parameters include the position and inclination of each free-form surface and the image plane in the free-form surface reflective imaging system.

6. The method for generating a free-form surface reflective imaging system according to claim 1, characterized in that: The constraints that the target free-form surface reflective imaging system needs to satisfy include focal length constraints, volume constraints, occlusion constraints, and intersection coordinate constraints between the main ray of the system's central field of view and each curved surface.

7. The method for generating a free-form surface reflective imaging system according to claim 1, characterized in that: When unique values ​​are set for all system parameters, structural parameters, and discrimination parameters, the DNN model outputs a specific free-form surface reflective imaging system that satisfies the target optical path folding form, system parameters, and structural parameters; When unique values ​​are set for all system parameters and structural parameters, and all possible values ​​are set for the discrimination parameters, the DNN model outputs multiple free-form surface reflective imaging systems with different optical path folding forms, and each free-form surface reflective imaging system meets the target system parameters and structural parameters; When unique values ​​are set only for some system parameters and some structural parameters, while value ranges are set for the remaining system parameters and structural parameters and all possible values ​​are set for the discrimination parameters, the DNN model outputs multiple free-form surface reflective imaging systems with different optical path folding forms, different system parameters, and different structural parameters, thereby realizing the multi-folding form and multi-structure generation of free-form surface reflective imaging systems.

8. The method for generating a free-form surface reflective imaging system according to claim 1, characterized in that: When the free-form surface reflective imaging system to be generated is a free-form surface off-axis three-mirror imaging system without an intermediate image, the method for determining the system parameters, structural parameters and discrimination parameters input into the DNN model is as follows: Assume that the free-form surface off-axis three-mirror imaging system includes a primary mirror M1, a secondary mirror M2, and a third mirror M3, and the system is symmetrical about the YOZ plane; The half field of view HXFOV in the x direction, the half field of view HYFOV in the y direction and the entrance pupil diameter ENPD are selected as system parameters, which are expressed as SYP = [HXFOV, HYFOV, ENPD]; The inclination angles of all surfaces and the y and z coordinates of the surface vertices are selected as structural parameters, expressed as STP = [M1 y ,M1 z ,M3 y ,M3 z ,IMG y ,IMG z ,M1 tilt ,M2 tilt ,M3 tilt ,IMG tilt ], where M1 y and M1 z are the y and z coordinates of the primary mirror M1, M3 y and M3 z are the y and z coordinates of the three mirrors M3, IMG y and IMG z are the y and z coordinates of the image plane IMG, M1 tilt is the inclination angle of the main mirror M1, M2 tilt is the inclination angle of the secondary mirror M2, M3 tilt is the inclination angle of the three-mirror M3, IMG tilt is the inclination angle of the image plane IMG; There are 8 types of optical path folding forms corresponding to the free-form surface off-axis three-mirror imaging system, and the value range of G is G=1, G=2,…, G=8.

Images