A Fast Generation Method for Multi-Structure in Freeform Surface Imaging Systems Based on Deep Learning

Abstract

This invention provides a method for rapid generation of multiple structures in freeform surface imaging systems based on deep learning. It automatically generates datasets using a system evolution method combining sequential and stochastic approaches. The network is trained by combining supervised learning with a feedback strategy and unsupervised learning based on differential ray tracing, enabling it to predict a wide range of systems and structural parameters. The trained DNN model can rapidly generate multiple systems according to design requirements, and the output systems can be sorted and filtered based on preset evaluation metrics, ultimately selecting a suitable system as a good starting point for subsequent optimization. This invention provides a novel method for designing freeform surfaces or general imaging systems, significantly reducing the time and effort spent on optical design.

Description

Technical Field

[0001] This invention belongs to the field of optical design, and in particular relates to a method for rapid generation of multiple structures in a freeform surface imaging system based on deep learning. Background Technology

[0002] Imaging optical systems play a vital role in scientific and social development. Throughout their long history, imaging systems have primarily consisted of spherical and aspherical elements due to their rotational symmetry and ease of manufacture; however, their aberration correction capabilities are very limited, especially in asymmetric systems. To overcome the limitations of traditional spherical and aspherical systems, non-rotationally symmetric freeform surface optical systems can be employed. These systems can reduce the size and number of components while improving system performance and technical specifications. The use of freeform surfaces is considered a revolution in imaging optical system design. Over the past 15 years, advancements in advanced manufacturing technologies have facilitated the application of freeform surface optics in numerous fields, such as astronomical telescopes, head-up displays, head-up display systems, cameras, off-axis imaging systems, and spectroscopic imagers.

[0003] Advanced freeform surfaces can improve the performance of imaging optical systems, but the complexity of the surface shape and the asymmetry of the system structure, coupled with the scarcity of existing reference systems and the difficulty in understanding freeform surfaces, significantly increase the design difficulty. Traditional optical design methods typically begin by finding a suitable design starting point, followed by multi-parameter optimization. The design starting point is obtained by searching the lens library of optical design software or consulting literature, a time-consuming process that may not yield a feasible starting point for freeform surface systems. Without a good starting point, the design process becomes highly dependent on manpower and design skills, potentially involving a significant amount of time spent on tedious trial and error, especially for optical design beginners with limited knowledge of aberration theory and design experience. Nodal aberration theory has been used to guide the design and optimization of freeform surface imaging systems; direct or point-by-point design methods can construct systems based on given design requirements. However, these methods also have limitations in terms of design efficiency, simplicity, and versatility, especially for systems with advanced system parameters (such as systems with large field of view). All of the above design methods need to be customized for specific design tasks. For other design tasks, these methods may need to be reapplied, and optimization strategies may even need to be adjusted accordingly. Furthermore, the time cost of a single design task is very high. Deep learning can be considered a solution to these problems because it can effectively summarize design knowledge and apply this knowledge to design tasks with different system and structural parameters. In 2019, Researchers used deep learning to obtain a lens database from which high-quality design starting points for coaxial spherical objectives could be derived. This research further introduced more design forms, however, limited to coaxial spherical systems. In 2019, Yang et al. proposed a preliminary design framework for a freeform surface reflection imaging system, and Chen et al. subsequently expanded the range of system parameters. However, this design method has significant limitations because it can only generate one solution, which is not necessarily optimal or meets the design requirements of the system structure. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes a rapid and automatic multi-structure generation method for freeform surface imaging systems based on deep learning. This design method is applicable to generalized off-axis reflection, refraction, and catadioptric systems with multiple freeform surfaces. By inputting design requirements, including system and structural parameters, into a neural network, a multi-structure freeform surface imaging system can be rapidly generated. Furthermore, the output systems can be sorted and filtered according to preset indicators, serving as a good starting point for subsequent optimization. This significantly improves the efficiency of freeform surface imaging optical system design and minimizes the designer's design time and effort.

[0005] Deep learning-based methods for generating multiple structures in freeform surface imaging systems include:

[0006] S1: Obtain the basic dataset, including:

[0007] S11: First, based on the characteristics of the system to be designed, select several representative system parameters SSP = [SSP1, SSP2, ..., SSP...]. m ,…,SSP M Simultaneously, determine the system's structural parameters STP = [STP1, STP2, ..., STP] t ,…,STP T ], and the surface shape parameter SFP = [SFP1, SFP2, ..., SFP v SFP V The system parameter space (SSPS) is determined, and further subdivided into smaller subspaces (SSPS). (i) Each system parameter subspace SSPS (i) Both are associated with a structural parameter subspace STPS (i) Correspondingly, this subspace needs to be determined using a benchmark system, and all subspaces are related to SSPS. (i) -STPS (i) These are combined to form the entire input parameter space; where M, T, and V represent the number of system parameters, structural parameters, and surface parameters, respectively.

[0008] S12: SSPS for each subspace (i) Optimize and generate a benchmark system RSYS (i) Among them, SSPS (i) The central system parameters are used as the reference system RSYS (i) System parameters in RSYS; (i) After generation, its structural parameters need to be obtained. Structural parameters of the reference system As STPS (i) The center of the structure is given, along with the length of the range of values ​​for the structural parameters. Thus, STPS was determined (i) Spatial range;

[0009] S13: Generate a sufficient number of systems based on the benchmark system, and obtain the system parameters, structural parameters and surface parameters of these systems to form a basic dataset; wherein, the system parameters and structural parameters are used as input parameters of the deep neural network (DNN), and the structural parameters and surface parameters are used as output parameters of the deep neural network (DNN);

[0010] S2: Supervised learning of deep neural networks (DNNs):

[0011] S21: Combine the input parameters from the base dataset and input them into the deep learning neural network (DNN) to obtain the predicted output. Calculate the difference between the output value and the true value in the base dataset, i.e., calculate the loss L. super Based on L super Pre-training the deep neural network (DNN) yields a preliminary DNN model.

[0012] S3: After a set period of supervised learning, unsupervised learning is introduced to combine supervised and unsupervised learning and further train the deep neural network (DNN) model.

[0013] S4: After training is complete, the design requirements, including system parameters and structural parameters, are input into the trained deep neural network (DNN) model to obtain one or more systems that meet the requirements.

[0014] Furthermore, in step 2, after each set period of supervised training, a system generation feedback is performed, namely: randomly selecting combinations of system parameters and structural parameters from different subspace pairs as inputs to the deep neural network (DNN); then directly inputting the system predicted by the current deep neural network (DNN) into the optical design software for optimization; and adding the system parameters, structural parameters, and surface parameters of the optimized system that achieves the set imaging quality to the training dataset for further network training.

[0015] Preferably, the SSPS is for each subspace. (i) Optimize and generate a benchmark system RSYS (i) The methods include:

[0016] First, the baseline system with the minimum system parameters is optimized and used as the starting point for evolution. During the evolution process, for each baseline system obtained, the system parameters SSP of that baseline system are calculated. * System parameters SSP of all other benchmark systems to be optimized ** The weighted distance D between them;

[0017] The SSP with the smallest weight distance D from the current benchmark system will be selected. # The corresponding benchmark system becomes the next system to be optimized; and so on, until all benchmark systems are generated.

[0018] Preferably, the method for generating a sufficient number of systems based on the reference system includes:

[0019] Using the reference system RSYS (i) As the initial structure, SSPS is applied in each subspace. (i) -STPS (i) Other systems are generated in the subspace SSPS, and their system parameters are in the subspace SSPS. (i) Randomly selected from RSYS; during the optimization process, RSYS will be used. (i) All surface parameters are set as variables, and their structure parameters are set with a probability P. f Changed to STPS (i) The random value in the data is changed by altering the structure parameters to set the probability P. f1 The structural parameters that are frozen but not changed are set with probability P. f2 Frozen; the same number of systems are generated in parallel in each subspace pair.

[0020] Preferably, the unsupervised learning method includes:

[0021] Random combinations of system parameters and structural parameters from different subspace pairs are used as inputs, and the output system is predicted by a deep neural network (DNN). Then, the differential ray tracing module of the freeform surface imaging system traces the system output by the network to obtain the unsupervised loss L. unsuper ;

[0022] The unsupervised loss L unsuper Including losses related to system imaging performance L performance Loss L related to design constraints constraint .

[0023] Preferably, when combining supervised and unsupervised learning, the loss L of supervised training is reduced. super and the unsupervised loss L unsuper We perform a weighted summation to obtain the total training loss, which is then used to further train the deep neural network (DNN) model.

[0024] Preferably, the supervised learning loss L super It is defined as the mean squared error between the true value and the actual predicted output of the deep neural network (DNN) model.

[0025] Ideally, multiple representative parameters are selected as system parameters, and the position and tilt angle of the freeform surface are selected as structural parameters. The type of surface is not limited.

[0026] Preferably, the method for dividing SSPS into several smaller subspaces is as follows:

[0027] SSP of each element in the system parameter SSP m The range of values ​​is divided into multiple segments according to length, and these segments can be combined arbitrarily to obtain multiple spatial SSPS. (i) .

[0028] Preferably, when training a deep neural network (DNN), the training dataset is preprocessed, and input data with the same parameters are normalized to [-1, 1].

[0029] Preferably, the design requirements, including system parameters and structural parameter requirements, are input into the trained deep neural network (DNN) model:

[0030] For parameters without specified values, values ​​are randomly assigned within the corresponding parameter range. Combined with the given parameters, a certain number of different input parameter combinations are obtained. These combinations are then input into the trained DNN model to generate a series of systems that meet the basic design requirements, thereby realizing the rapid and automatic generation of multiple structures in the freeform surface imaging system.

[0031] Preferably, after obtaining one or more systems that meet the requirements, the output systems are screened and classified based on preset evaluation indicators and related constraints; then the generated systems are further optimized to obtain systems with higher imaging performance.

[0032] The present invention has the following beneficial effects:

[0033] This invention provides a method for rapid generation of multiple structures in a freeform surface imaging system based on deep learning. A basic dataset is automatically generated through a combination of sequential and random freeform surface system evolution methods. This dataset is then used to pre-train a neural network, giving it a certain predictive ability. A feedback strategy is introduced to further enhance network performance. Supervised learning is combined with unsupervised learning based on differential ray tracing to train a high-performance neural network model. For subsequent freeform surface imaging system design, simply inputting the system design requirements (including system parameters and structural parameters) into the trained neural network model will yield a system with one or more structures that meet the design requirements.

[0034] For training deep neural network models with high-dimensional input parameters and a large parameter space, the training set needs to be large enough to obtain good training results. Therefore, this invention uses sequential and random system evolution methods to obtain the basic training set, and then introduces a feedback strategy to supplement the dataset. This not only increases the number of systems in the training set, but also improves the diversity of system structures in the training dataset.

[0035] Unlike supervised learning, unsupervised learning does not require labeled datasets. Therefore, this invention uses a combination of supervised learning and unsupervised learning based on differential ray tracing to train neural networks, which greatly reduces the burden of obtaining a large training dataset during supervised learning.

[0036] For systems that output data over a network, you can choose to quickly optimize them to obtain systems with better performance. You can also sort and filter these systems according to preset indicators, which greatly reduces time and manpower costs.

[0037] This invention provides a method for rapid automatic generation of multiple structures in a freeform surface imaging system based on deep learning. Given the required system and structural parameters as input, the trained model can output a system of one or more structures almost immediately. Designers can choose whether to evaluate the image quality of the output system or perform rapid automatic optimization, and can also sort and filter the output systems to facilitate the selection of suitable systems as a good starting point for further optimization. The network can also be integrated into existing optical design software and cloud servers, making it accessible to more designers. Attached Figure Description

[0038] Figure 1 The flowchart illustrates the supervised learning process for generating the basic dataset and employing a feedback strategy, as provided in this invention.

[0039] Figure 2 The flowchart illustrates the neural network training method that combines supervised and unsupervised learning provided by this invention.

[0040] Figure 3 This invention provides a rapid multi-structure generation process for a freeform surface imaging system based on a trained deep neural network (DNN) model.

[0041] Figure 4 The present invention provides a typical optical path diagram of a network prediction system when both system parameters and structural parameters are given.

[0042] Figure 5 The present invention provides a typical optical path diagram of a network prediction system when system parameters and some structural parameters are given. Detailed Implementation

[0043] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0044] A method for rapid generation of multiple structures in a freeform surface imaging system based on deep learning includes the following steps:

[0045] S1: Obtain the basic dataset by following these steps:

[0046] S11: First, based on the characteristics of the system to be designed, select several representative system parameters SSP = [SSP1, SSP2, ..., SSP...]. m ,…,SSP M Simultaneously, determine the system's structural parameters STP = [STP1, STP2, ..., STP] t ,…,STP T ], and the surface shape parameter SFP = [SFP1, SFP2, ..., SFP v SFP V The system parameter space SSPS is determined, and then further divided into several smaller subspaces SSPS. (i) Each system parameter subspace SSPS (i) Both are associated with a structural parameter subspace STPS (i) Correspondingly, this subspace needs to be determined using a benchmark system, and all subspaces are related to SSPS. (i) -STPS (i) These are combined to form the entire input parameter space;

[0047] S12: Use a system evolution approach for each subspace SSPS (i) Generate a benchmark system RSYS (i) SSPS (i) The central system parameters are used as RSYS (i) System parameters in RSYS.(i) After generation, its structural parameters need to be obtained. Consider the structural parameters of the reference system As STPS (i) The center of the structure is given, along with the length of the range of values ​​for the structural parameters. Therefore, STPS can be determined. (i) The spatial range. Meanwhile, for SSPS with larger system parameters... (i) The corresponding STPS (i) Also bigger;

[0048] S13: A sufficient number of systems are obtained using a stochastic optimization generation method, and the system parameters, structural parameters, and surface parameters of these systems are acquired to form the basic dataset. The system parameters and structural parameters serve as the input parameters of the neural network, while the structural parameters and surface parameters serve as the output parameters of the neural network.

[0049] S2: Perform supervised learning on the neural network according to the following steps:

[0050] S21: Combine the input parameters from the base dataset and input them into the deep neural network (DNN) to obtain the network's predicted output. Calculate the difference between the output value and the true value in the base dataset, i.e., calculate the loss L. super Based on L super The network is pre-trained to obtain a DNN model with some predictive ability.

[0051] S22: Subsequently, a feedback strategy is introduced, in which system-generated feedback is executed once after a period of supervised training. This is repeated multiple times to expand the number and diversity of systems in the dataset, while improving the performance of the network model;

[0052] S3: After a period of supervised learning, unsupervised learning is introduced. Supervised and unsupervised learning are combined for subsequent neural network training, and finally a high-performance DNN model is obtained.

[0053] S4: After training, inputting the design requirements (including system and structural parameter requirements) into the trained DNN model yields one or more systems that meet the requirements. Based on preset evaluation metrics (e.g., various imaging performance metrics and system volume), and other relevant constraints, the output systems can be screened and classified. The systems generated by the network can be directly used as a good starting point for further optimization. Simultaneously, the systems directly output by the network can also be rapidly optimized in parallel, resulting in systems with higher imaging performance and significantly improving the efficiency of optical design for freeform surface systems.

[0054] Furthermore, for freeform surface imaging optical systems, field of view (FOV), effective focal length (EFL), f-number (F#), and entrance pupil diameter (ENPD) are typically used as representative system parameters. In addition, some parameters can be fixed to specific values ​​(e.g., EFL), while other parameter values ​​can be obtained through scaling. The position and tilt angle of the freeform surface can be considered as structural parameters, and the type of surface is unrestricted.

[0055] Furthermore, the method for dividing SSPS into several smaller subspaces is as follows:

[0056] SSP of each element in the system parameter SSP m The range of values ​​for (1≤m≤M) is determined by the length L. m Divided into I m At this point, SSPS is divided into segments. Individual Space SSPS (i) (1≤i≤I).

[0057] Furthermore, the system evolution method is as follows:

[0058] First, the baseline system with the minimum system parameters is optimized and used as the starting point for evolution. During the evolution process, for each baseline system obtained, the system parameters SSP of that baseline system need to be calculated. * System parameters SSP of all other benchmark systems to be optimized ** The weighted distance D between them. The formula for calculating the weighted distance D is as follows:

[0059]

[0060] Where ||·|| is the 2-norm, and W = [W1, W2, ..., W...]. m ,…,W M [ ] is the weight vector that determines the weight of each system parameter. This represents the element-wise multiplication of corresponding vectors;

[0061] SSP with the smallest weighted distance D # The corresponding reference system (i.e., with SSP) * The nearest neighbor of the corresponding system becomes the next baseline system to be optimized and generated. This process is repeated until all baseline systems are generated. If an abnormal system is encountered (e.g., ray tracing errors, occlusion, or poor image quality), the next nearest neighbor system from the already generated systems is used as the starting point for re-evolution and generation.

[0062] Furthermore, the random optimization generation method is as follows:

[0063] Using the reference system RSYS (i) As the initial structure, SSPS is applied in each subspace. (i)-STPS (i) Other systems are generated in the subspace SSPS, and their system parameters are in the subspace SSPS. (i) Randomly selected from RSYS. During the optimization process, RSYS will be used... (i) All surface parameters are set as variables, and their structure parameters are expressed with probability P. f Changed to STPS (i) The random value in the data is changed with probability P. f1 The structural parameters that are frozen but not changed have a probability P. f2 Frozen. The same number of systems are generated in each subspace pair; the generation process can be performed in parallel.

[0064] Furthermore, when training a neural network, the training dataset needs to be preprocessed. Input data with the same parameters are normalized to [-1, 1] using a linear preprocessing method similar to Min-Max normalization.

[0065] Furthermore, the supervised learning loss function L super It can be defined as the mean square error between the target and the network's actual predicted output.

[0066] Furthermore, the feedback strategy is as follows:

[0067] System and structural parameter combinations are randomly selected from different subspace pairs as DNN inputs. The system predicted by the current network is then directly input into optical design software (such as CODE V, Zemax, etc.) for rapid optimization. During optimization, the structural parameters are frozen with probability Ps. If the optimized system has no ray tracing errors or occlusions and exhibits good imaging quality, the system parameters, structural parameters, and surface parameters are recorded and added to the training dataset for further network training. After a period of training, the system generates feedback again, repeating the above process.

[0068] Furthermore, the unsupervised learning method is as follows:

[0069] Random combinations of system parameters and structural parameters from different subspace pairs are used as inputs, and the output system is predicted by a DNN neural network. Then, the differential ray tracing module of the freeform surface imaging system traces the system output by the network to obtain the unsupervised loss function L. unsuper L unsuper It consists of two parts, one part of which is a loss of L performance This is related to the system's imaging performance (e.g., spot diameter, distortion, etc.). Another part of the loss is L. constraintRelated to the design constraints that need to be satisfied (such as constraints on system focal length, volume, occlusion, etc.), it is based on penalty functions such as quadratic functions, reciprocals, logarithmic functions, and higher-order power functions. The entire prediction and calculation process is completely differentiable, thus allowing the calculation of the loss function L. unsuper The gradients of each parameter in the neural network are backpropagated, and optimization methods are used to update the network parameters and improve network performance. GPU computing power and parallel computing are utilized to improve overall computational efficiency. The loss calculation formula for unsupervised training is as follows:

[0070] L unsuper =Σ(ρ w L performance +L constraint )

[0071] Where, ρ w It is used to balance the system parameters of different systems to L performance The weighting factor for the contribution decreases as the system parameters increase.

[0072] Furthermore, the formula for calculating the total training loss combining supervised and unsupervised learning is as follows:

[0073] L total =L super +w unsuper L unsuper

[0074] By adjusting the weight w unsuper This allows adjustment of the contribution of unsupervised learning to the overall network training. Based on L... total The network is trained to obtain a high-performance DNN model.

[0075] Furthermore, the specific process of obtaining one or more structural systems that meet the design requirements through the trained DNN model is as follows:

[0076] For a specific design task, all system and structural parameters can be provided according to the design requirements. In this case, the trained DNN model can immediately output the corresponding single design result based on the input parameters. Alternatively, only some system and structural parameters may be provided based on the actual design needs. In this case, for parameters without specified values, values ​​will be randomly assigned within the corresponding parameter range. Combined with the given parameters, a number of different input parameter combinations can be obtained. These combinations can be fed into the trained DNN model to generate a series of systems that meet the basic design requirements, enabling the rapid and automatic generation of multiple structures in a freeform surface imaging system.

[0077] Example 1

[0078] The feasibility of the design method and the effectiveness of the trained DNN model were verified through the design of a freeform surface off-axis three-mirror imaging system. The following section, with reference to the accompanying figures, illustrates the construction and application process of a DNN model for a freeform surface imaging system with a large parameter range.

[0079] The selected freeform off-axis three-mirror imaging system has a central field of view of (0°, 0°), is symmetrical about the YOZ plane, and the principal ray of the central field of view intersects the vertex of the surface. The optical path folding pattern is a traditional zigzag. M1, M2, M3, and IMG represent the primary mirror, secondary mirror, third mirror, and image plane, respectively, with the aperture stop located at M2. See also Figure 1 First, representative system and structural parameters are selected. For this system, the field of view (XFOV) in the x-direction, the field of view (YFOV) in the y-direction, and the entrance pupil diameter (ENPD) are chosen to describe the system parameters, i.e., SSP = [XFOV, YFOV, ENPD]. The focal length is set to 1mm, a fixed value, and other focal lengths can be obtained by scaling. The structural parameters include the tilt angles of all surfaces in the system and the y and z vertex coordinates. Because the vertex of M2 is the origin of the system's global coordinate system, the y and z coordinates of the M2 vertex are not included in the structural parameter STP. Therefore, STP = [M1 y M1 z M3 y M3 z IMG y IMG z M1 tilt M2 tilt M3 tilt IMG tilt (Note: This embodiment selects a system that is symmetric about the YOZ plane, where the principal ray of the central field of view intersects with the vertex of the surface. Therefore, the surface tilt angle is not used as an input structural parameter, but only as an output structural parameter.) The surface type and surface parameters also need to be determined. The selected freeform surface type is a 6th-order XY polynomial freeform surface. For simplicity, it does not contain any spherical or quadratic surface terms; the mathematical expression of the surface shape consists only of polynomials, and odd-degree terms of x are not used. Therefore, the SFP has 42 parameters. In total, using system parameters and some structural parameters as input parameters to the neural network, there are a total of 9 parameters. Using all structural parameters and surface parameters as output parameters to the neural network, there are a total of 52 parameters.

[0080] The next step is to determine a suitable SSPS. XFOV and YFOV are specified to be in the range of 4-40°, and ENPD in the range of 1 / 6-2 / 3 mm (F-number in the range of 1.5-6). However, because it is difficult to meet the design requirements and achieve good imaging performance while simultaneously achieving a large field of view and a small F-number, only system parameters that satisfy (0.5×XFOV+0.5×YFOV+20×ENPD)≤33.34 (condition 1) are selected. To ensure the network achieves good training results within the specified parameter range, the SSP... m The upper and lower limits are extended by 0.5L m The SSPS is appropriately enlarged and then divided into smaller subspaces. The values ​​of XFOV, YFOV, and ENPD are segmented according to lengths L1 = 2°, L2 = 2°, and L3 = 0.05, respectively, resulting in I1 = 19, I2 = 19, and I3 = 11. The total number of subspace pairs is... After removing subspace pairs whose central system parameters do not meet condition 1, 2598 subspace pairs remain.

[0081] The baseline system RSYS was generated using a system evolution approach. Optimization of the baseline system was performed using the optical design software CODE V. Because the system is symmetric about the YOZ plane, only half of the full field of view was considered in the design. Six sampling points were selected for each system: (0,0), (0,YFOV / 2), (0,-YFOV / 2), (XFOV / 2,0), (XFOV / 2,YFOV / 2), and (XFOV / 2,-YFOV / 2). During system generation, constraints were placed on the system's focal length, occlusion, distortion, and the coordinates of the intersections of the principal ray and the surface in the central region, while allowing for greater aberrations in systems with larger entrance pupil diameters and fields of view. Before generating the baseline system, the following parameters were specified: in It is the reference system RSYS (i) The lateral dimension (along the z-direction), A = [A1, A2, ..., A t ,…,A T Each element in A corresponds to each element in STP, and the value of A is the same for all subspaces. t The values ​​are specified as A1 = A2 = ... = A6 = 1 / 3. With the optimization of the benchmark system, Hdim (i) It is a constantly changing variable that causes R to... (i) It also kept changing until the reference system RSYS (i) Once optimization is complete, R can be determined. (i) The final value. To minimize the occlusion structure in the subspace pair, when optimizing the baseline system, we can use R... (i) The value of R controls the distance between surfaces, ensuring as much as possible that the distance between surfaces is controlled by R.(i) and The defined positions of the various surfaces do not overlap. The evolution direction of the reference system is determined by the weight W, which takes the value of... During the evolution process, the structural parameter values ​​of all reference systems are obtained, thereby determining the range of structural parameters for all subspace pairs.

[0082] After obtaining all the benchmark systems, a complete base dataset is generated using a stochastic optimization generation method. f P f1 and P f2 The values ​​are 0.6, 0.5, and 0.2, respectively. Twelve systems are generated in each subspace pair, resulting in a total of 31,176 systems from 2,598 subspace pairs. The system parameters, structural parameters, and surface parameters of these systems are used to form the basic dataset.

[0083] After obtaining the basic dataset, the DNN is pre-trained using it. In this embodiment, the DNN has 20 hidden layers, with a maximum of 300 nodes per layer. The activation function, optimizer, loss function, and learning rate are hyperbolic tangent, Adam, MSE, and 1e-4, respectively. All data in the training dataset are input into the network. Pre-training is performed 15,000 times. After pre-training, a feedback training strategy is further employed. Each time the system generates feedback, 1000 random parameter combinations are selected from different subspace pairs as DNN inputs, and the system output by the DNN is optimized. S The value was set to 0.5. Systems without ray tracing errors or occlusion and with good image quality were then added to the dataset for further training. After feedback training, supervised training continued for a period, with the learning rate gradually decreasing as training progressed. A total of 147,000 training iterations were performed, increasing the number of systems in the dataset to 109,703. The model's performance was tested using 2000 random inputs. 115 systems exhibited ray tracing errors, 26 systems had occlusion, and the remaining 1859 normal systems had an average root mean square (RMS) speckle diameter of 0.0044 mm.

[0084] Next, we will introduce unsupervised learning. For details on training methods that combine supervised and unsupervised learning, please refer to [link to relevant documentation]. Figure 2 The weights for unsupervised learning are w. unsuper=5, and the input data used for unsupervised training is not fixed. Each training iteration randomly selects 500 pairs from the spatial pairs, and a combination of input parameter values ​​is randomly generated for each spatial pair. These 500 inputs are then used for unsupervised training. A total of 1,210 training iterations combining supervised and unsupervised learning were performed. The model's performance was then tested again using the previous 2,000 inputs, resulting in a reduction of the number of systems with ray tracing errors to 61, and a reduction of the average RMS speckle diameter of the normal systems to 0.0025 mm. It can be seen that the introduction of unsupervised learning significantly improves the performance of the DNN.

[0085] See Figure 3 The resulting DNN model can quickly and automatically generate multi-structure freeform surface imaging systems according to design requirements. By inputting all or part of the system and structural parameters based on design requirements, systems with single or multiple structures can be generated. Users can also choose whether to evaluate system imaging quality and perform further software optimization. Output systems can be filtered or sorted based on selected metrics, such as RMS speckle diameter, maximum absolute distortion of the sampling field of view, system volume, and modulation transfer function (MTF). The filtered or sorted systems are then available for user selection.

[0086] The predictive performance of the trained DNN model was evaluated using two different parameter input methods. In each case, 5000 different inputs were tested. The prediction system's focal length was set to 1mm, and no further optimization was performed.

[0087] In the first case, all system and structural parameters are given (all parameter combinations are random values ​​in the parameter space). A set of input parameters corresponds to one output system. Of the 5000 predicted systems, 138 systems have ray tracing errors, 43 systems have occlusion, and the average RMS blur diameter of the remaining 4819 normal systems is 0.0026 mm. Except for a very few systems with relatively large aberrations, the output systems can serve as a good starting point for further optimization. See also... Figure 4 Nine typical prediction systems were demonstrated.

[0088] The second scenario involves providing all system parameters and some structural parameters. The given parameters are: XFOV = 24°, YFOV = 16°, ENPD = 1 / 3 mm (F# = 3.00), M1 y = 2.38mm (Note M2) y =0). Other structural parameters were set to random values ​​within their respective subspaces. Among the predicted systems, three systems exhibited ray tracing errors, and one system was unobstructed. The average RMS speckle diameter for the other 4997 normal systems was 0.0010 mm. Nine typical systems with different structural parameters are listed below. Figure 5 . Figure 5 (a), (b) and (c) show systems with the smallest RMS speckle diameter, the smallest distortion and the smallest volume, respectively.

[0089] Therefore, the fast and automatic generation method for multi-structure freeform surface imaging systems based on deep learning provided in this embodiment has the following advantages: First, it can quickly generate a large number of high-quality basic datasets within a wide range of system parameters. Feedback strategies can further supplement the datasets, increasing both the number and diversity of systems, which helps the network train better and more fully. Data acquisition can be automated. Second, unsupervised learning is equivalent to directly optimizing the system output by the network. Its introduction greatly reduces the pressure of acquiring large datasets during supervised learning, thus significantly reducing the dataset size during supervised training. Third, after obtaining a high-performance DNN model, only design requirements including system and structural parameters need to be input. The network can then quickly generate multi-structure freeform surface imaging systems. Furthermore, the output systems can be sorted and selected according to given criteria, serving as a good starting point for final optimization. Simultaneously, the systems directly output by the network can also be rapidly optimized in parallel, resulting in systems with higher imaging performance. Fourth, this method provides a general, rapid, and automatic method for generating multiple structures in freeform surface imaging systems, applicable to a wide range of system parameters and advanced system parameters, as well as various types or applications, offering insights into the generation of initial structures for other freeform surfaces. Fifth, this method significantly reduces the time and manpower costs for designers, and it can also be integrated into existing optical design software and cloud servers, making it convenient for more designers to use.

[0090] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A multi-structure generation method for a freeform surface imaging system based on deep learning, characterized in that, include: S1: Obtain the basic dataset, including: S11: First, based on the characteristics of the system to be designed, select several representative system parameters SSP = [SSP1, SSP2, ..., SSP...]. m ,…,SSP M Simultaneously, determine the system's structural parameters STP = [STP1, STP2, ..., STP] t ,…,STP T ], and the surface parameters SFP = [SFP1, SFP2, ..., SFP v SFP V The system parameter space (SSPS) is determined, and further subdivided into smaller subspaces (SSPS). (i) Each system parameter subspace SSPS (i) Both are associated with a structural parameter subspace STPS (i) Correspondingly, this subspace SSPS (i) It needs to be determined using a benchmark system, and all subspaces are relative to SSPS. (i) -STPS (i) These parameters are combined to form the entire input parameter space; where M, T, and V represent the number of system parameters, structural parameters, and surface parameters, respectively; representative system parameters include field of view, effective focal length, F-number, and entrance pupil diameter. S12: SSPS for each subspace (i) Optimize and generate a benchmark system RSYS (i) Among them, SSPS (i) The central system parameters are used as the reference system RSYS (i) System parameters in RSYS; (i) After generation, its structural parameters need to be obtained. Structural parameters of the reference system As STPS (i) The center of the structure is given, along with the length of the range of values ​​for the structural parameters. Thus, STPS was determined (i) Spatial range; S13: Generate a sufficient number of systems based on the benchmark system, and obtain the system parameters, structural parameters and surface parameters of these systems to form a basic dataset; wherein, the system parameters and structural parameters are used as input parameters of the deep neural network (DNN), and the structural parameters and surface parameters are used as output parameters of the deep neural network (DNN); S2: Supervised learning of deep neural networks (DNNs): S21: Combine the input parameters from the base dataset and input them into the deep learning neural network (DNN) to obtain the predicted output. Calculate the difference between the output value and the true value in the base dataset, i.e., calculate the loss L. super Based on L super Pre-training the deep neural network (DNN) yields a preliminary DNN model. S3: After a set period of supervised learning, unsupervised learning is introduced to combine supervised and unsupervised learning and further train the deep neural network (DNN) model. S4: After training is complete, the design requirements, including system parameters and structural parameters, are input into the trained deep neural network (DNN) model to obtain one or more systems that meet the requirements.

2. The method for rapid generation of multiple structures in a freeform surface imaging system based on deep learning as described in claim 1, characterized in that, In S2, after each set period of supervised learning, a system generation feedback is performed, namely: randomly selecting a combination of system parameters and structural parameters from different subspace pairs as input to the deep neural network (DNN); then directly inputting the system predicted by the current deep neural network (DNN) into the optical design software for optimization; and adding the system parameters, structural parameters, and surface parameters of the optimized system that achieves the set imaging quality to the training dataset for further network training.

3. The method for rapid generation of multiple structures in a deep learning-based freeform surface imaging system as described in claim 1 or 2, characterized in that, The above refers to each subspace SSPS (i) Optimize and generate a benchmark system RSYS (i) The methods include: First, the baseline system with the minimum system parameters is optimized and used as the starting point for evolution. During the evolution process, for each baseline system obtained, the system parameters SSP of that baseline system are calculated. * System parameters SSP of all other benchmark systems to be optimized ** The weighted distance D between them; The SSP with the smallest weight distance D from the current benchmark system will be selected. # The corresponding benchmark system becomes the next system to be optimized; and so on, until all benchmark systems are generated.

4. The method for rapid generation of multiple structures in a deep learning-based freeform surface imaging system as described in claim 1 or 2, characterized in that, The method for generating sufficient systems based on the benchmark system includes: Using the reference system RSYS (i) As the initial structure, SSPS is applied in each subspace. (i) -STPS (i) Other systems are generated in the subspace SSPS, and their system parameters are in the subspace SSPS. (i) Randomly selected from RSYS; during the optimization process, RSYS will be used. (i) All surface parameters are set as variables, and their structure parameters are set with a probability P. f Changed to STPS (i) The random value in the data is changed by altering the structure parameters to set the probability P. f1 The structural parameters that are frozen but not changed are set with probability P. f2 Frozen; the same number of systems are generated in parallel in each subspace pair.

5. The method for rapid generation of multiple structures in a deep learning-based freeform surface imaging system as described in claim 1 or 2, characterized in that, The unsupervised learning methods include: Random combinations of system parameters and structural parameters from different subspace pairs are used as inputs, and the output system is predicted by a deep neural network (DNN). Then, the differential ray tracing module of the freeform surface imaging system traces the system output by the network to obtain the unsupervised loss L. unsuper ; The unsupervised loss L unsuper Including losses related to system imaging performance L performance Loss L related to design constraints constraint .

6. The method for rapid generation of multiple structures in a freeform surface imaging system based on deep learning as described in claim 5, characterized in that, When combining supervised and unsupervised learning, the supervised learning loss L super and the unsupervised loss L unsuper We perform a weighted summation to obtain the total training loss, which is then used to further train the deep neural network (DNN) model.

7. The method for rapid generation of multiple structures in a freeform surface imaging system based on deep learning as described in claim 6, characterized in that, The supervised learning loss L super It is defined as the mean squared error between the true value and the actual predicted output of the deep neural network (DNN) model.

8. The method for rapid generation of multiple structures in a deep learning-based freeform surface imaging system as described in claim 1 or 2, characterized in that, Several representative parameters are selected as system parameters, and the position and tilt angle of the freeform surface are used as structural parameters. The type of surface is not restricted.

9. The method for rapid generation of multiple structures in a deep learning-based freeform surface imaging system as described in claim 1 or 2, characterized in that, The method for dividing SSPS into smaller subspaces is as follows: SSP of each element in the system parameter SSP m The range of values ​​is divided into multiple segments according to length, and these segments can be combined arbitrarily to obtain multiple subspaces SSPS. (i) .

10. The method for rapid generation of multiple structures in a deep learning-based freeform surface imaging system as described in claim 1 or 2, characterized in that, When training a deep neural network (DNN), the training dataset is preprocessed, and input data with the same parameters are normalized to [-1, 1].

11. The method for rapid generation of multiple structures in a deep learning-based freeform surface imaging system as described in claim 1 or 2, characterized in that, The design requirements, including system parameters and structural parameter requirements, are input into the trained deep neural network (DNN) model: For parameters without specified values, values ​​are randomly assigned within the corresponding parameter range. Combined with the given parameters, a certain number of different input parameter combinations are obtained. These combinations are then input into the trained DNN model to generate a series of systems that meet the basic design requirements, thereby realizing the rapid and automatic generation of multiple structures in the freeform surface imaging system.

12. The method for rapid generation of multiple structures in a deep learning-based freeform surface imaging system as described in claim 1 or 2, characterized in that, After obtaining one or more systems that meet the requirements, the output systems are screened and classified based on preset evaluation indicators and related constraints; then the generated systems are further optimized to obtain systems with higher imaging performance.

Images