Universal lens intelligent core adjustment method and system based on lens library and optical simulation

Abstract

The present invention discloses a universal lens intelligent core adjustment method and system based on a lens library and optical simulation, belonging to the field of optical lens core adjustment. Different simulated lenses are added to a simulated optical path system, and clear extended images are used as input in each field of view to calculate the point spread function matrix of each field of view of the simulated lens under different misalignment amounts, and the misalignment amount is used as a label; at the same time, the illumination distribution matrix and distortion distribution matrix corresponding to each misalignment amount are obtained; a series of ideal degraded images under different misalignment amounts are calculated, and the simulated degraded images are optimized using the illumination distribution matrix and the distortion distribution matrix; the universal lens intelligent core adjustment model is trained using the simulated degraded images and misalignment amount labels, and after fine-tuning, the intelligent core adjustment of lenses with the same optical structure can be realized. The present invention uses the aberration characteristics of a large number of simulated lenses to pre-train the core adjustment model a priori, improves the generalization of the model to different lenses, and achieves the purpose of quickly adjusting any lens using only one model.

Description

Technical Field

[0001] The present invention relates to the field of optical lens core adjustment, and in particular to a universal lens intelligent core adjustment method and system based on a lens library and optical simulation. Background Art

[0002] In recent years, due to the continuous improvement of optical design and lens manufacturing technology, optical lenses have developed rapidly and are widely used in various fields such as mobile phones, automotive systems, and security. The resolution of image sensors has increased from 480P in the past to 4K or even 8K today, which puts higher requirements on the imaging performance of optical lenses. The production of high-quality optical lenses depends on excellent optical design, precision optical manufacturing, and precise optical assembly. High-definition optical lenses are usually composed of multiple optical elements, and the precise geometric arrangement of each element is crucial to achieving imaging quality equivalent to the optical design.

[0003] Traditional alignment methods use passive alignment through mechanical structures, but as the tolerance requirements for lens design become increasingly stringent, the manufacturing accuracy requirements for mechanical structures become increasingly high, and costs are difficult to control. At the same time, due to the manufacturing and processing errors of optical components themselves, their optimal imaging position is not always completely consistent with the optical design.

[0004] In order to solve this problem, active alignment technology came into being, that is, using the optical feedback after the light source passes through the lens to predict the deviation of the optical element and align it to the optimal imaging position. The current mainstream active alignment technology, for example, uses the cross slit image to calculate the MTF value under different fields of view during the alignment of the mobile phone lens, obtains the multi-field defocus curve under a certain spatial frequency through defocus scanning, and calculates the misalignment amount based on this. Such a method usually requires multiple defocus scans, which is relatively time-consuming and seriously limits the efficiency of large-scale lens production. The intelligent core adjustment method based on deep learning uses the cross image pictures with eccentricity labels collected by the core adjustment equipment as training data, and sends them to the deep learning model for training. It can be used in the actual assembly process to reduce the number of defocus scans, thereby significantly improving the core adjustment efficiency. However, due to the differences in the aberrations introduced by lenses with different optical structures, the model trained with a certain structure lens is not applicable to the core adjustment of lenses with other structures. At this time, a series of tasks such as data set preparation and model training need to be re-performed for the new lens. The actual acquisition of lens data requires a lot of time and resources, so this field needs a universal intelligent core adjustment method that can be applied to different lens core adjustments. The difficulty in realizing this technology lies in:

[0005] 1) There are many structures and types of imaging lenses, and lenses of different structures have different optical properties, so it is difficult to cover all types of aberrations with one data set; and there are large differences between simulation data and actual data, so simulation data cannot be used in the actual core adjustment process.

[0006] 2) When the dataset contains lenses with multiple aberrations, the model also needs to have strong generalization and be able to make accurate core alignment predictions for images with different aberration degradations. Summary of the invention

[0007] In order to solve the problems of low efficiency of the above-mentioned traditional lens alignment method and high data preparation cost of the alignment method based on deep learning, the present invention proposes a universal lens intelligent alignment method and system based on lens library and optical simulation. The alignment model is pre-trained by using the aberration characteristics of a large number of simulated lenses to improve the generalization of the model to different lenses, and the purpose of quickly aligning any lens using only one model is achieved.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0009] In a first aspect, the present invention proposes a universal lens intelligent core adjustment method based on a lens library and optical simulation, comprising:

[0010] Build a simulation lens database;

[0011] Adding the lens design file in the simulation lens database into the simulation optical path system; the simulation optical path system is provided with a plurality of simulated industrial cameras under the field of view, and the simulated industrial camera includes a focusing lens and an imaging sensor arranged at the focus of the focusing lens;

[0012] Under each field of view, the clear expanded image is used as input, and the point spread function matrix of the imaging sensor of each field of view of the simulation lens under different misalignment is calculated, and the corresponding misalignment is used as the label of the point spread function matrix; at the same time, the illumination distribution matrix and distortion distribution matrix corresponding to each misalignment are obtained;

[0013] A series of ideal degraded images under different misalignment amounts are calculated based on the clear extended image and the point spread function matrix; the ideal degraded image is optimized based on the post-compensation algorithm using the illumination distribution matrix and the distortion distribution matrix to obtain a simulated degraded image;

[0014] Use simulated degraded images and misalignment labels to train a universal lens intelligent alignment model;

[0015] The real data of the lens to be aligned is used to fine-tune the universal lens intelligent alignment model, and the fine-tuned model is used to perform intelligent alignment of lenses with the same optical structure.

[0016] As a preferred embodiment of the present invention, the simulation lens database construction process includes:

[0017] An automatic lens design algorithm is used to generate candidate simulated lens structures under different design specifications, wherein the automatic lens design algorithm takes the sampled design specifications and the randomly generated lens parameters as input and takes the simulated lens structure under the corresponding design specifications as output;

[0018] Eliminate candidate simulation lens structures that do not meet the lens processing constraints, screen the remaining candidate simulation lens structures based on the sampling strategy, and build a simulation lens database;

[0019] The sampling strategy described meets the following requirements:

[0020] The sampled simulated lens samples cover the severity of the lens sample aberrations that are evenly distributed, and the difference between the lens samples is greater than a set threshold.

[0021] As a preferred embodiment of the present invention, the severity of the lens sample aberration refers to the average value of the root mean square radius of the point diagram of each field of view of the simulated lens structure, and the difference between the lens samples refers to the absolute value difference between the point spread function matrices of the simulated lens structure.

[0022] As a preferred embodiment of the present invention, the method of taking a clear expanded image as input in each field of view, calculating the point spread function matrix of each field of view imaging sensor of the simulation lens under different misalignment amounts, and taking the corresponding misalignment amount as a label of the point spread function matrix includes:

[0023] The simulated lens structures are divided into two groups according to the tolerance analysis results of each simulated lens structure in the simulated lens database;

[0024] Under each field of view of the simulated optical path system, the pupil sampling frequency, image plane sampling frequency and image plane sampling interval matching the performance of the imaging sensor are preset;

[0025] Add an offset between two groups of the simulated lens structure, take a clear extended image as input, and calculate the point spread function matrix of each field of view imaging sensor of the simulated lens under the offset through a ray tracing algorithm; traverse the imaging sensor performance range and the simulated lens structure to obtain the point spread function matrix of each field of view corresponding to the offset under different performance parameters of the simulated lens structure;

[0026] The misalignment range is traversed to obtain a series of point spread function matrices of each field of view at each misalignment, wherein the labels of the point spread function matrices are the corresponding misalignments.

[0027] As a preferred embodiment of the present invention, the clear expanded images under different fields of view are different.

[0028] As a preferred embodiment of the present invention, the method of calculating a series of ideal degraded images under different misalignment amounts according to the clear extended image and the point spread function matrix includes:

[0029] Under each misalignment, a series of point spread function matrices of each field of view are convolved with the clear extended image under the field of view to obtain a series of ideal degraded images under the misalignment, and the labels of the series of ideal degraded images are the corresponding misalignments.

[0030] As a preferred embodiment of the present invention, the post-compensation algorithm is specifically as follows:

[0031] The ideal degraded image is multiplied by the illumination distribution matrix and the distortion distribution matrix under the corresponding misadjustment to optimize the ideal degraded image.

[0032] As a preferred embodiment of the present invention, the lens intelligent core-adjusting model adopts a convolutional neural network structure, takes a simulated degraded image as input, and predicts the misalignment amount.

[0033] As a preferred embodiment of the present invention, the misalignment amount is a multi-degree-of-freedom value.

[0034] In a second aspect, the present invention proposes a universal lens intelligent core-adjusting system based on a lens library and optical simulation, which is used to implement the universal lens intelligent core-adjusting method mentioned above.

[0035] The beneficial effects of the present invention are:

[0036] The present invention proposes a universal lens intelligent core-adjusting method and system based on a lens library and optical simulation, wherein the lens structure in the designed simulation lens database is loaded into a simulation optical path system, misalignment is added to various lens structures, and the point spread function matrix of each field imaging sensor of the simulation lens under different misalignment is calculated with a clear extended image as input under each field of view, and an ideal degraded image is further calculated and optimized based on a post-compensation algorithm, thereby providing a large number of reliable training data sets for the intelligent core-adjusting model; since the lens structure in the simulation lens database meets the actual processing requirements and covers a large number of aberration characteristic distributions, the aberration characteristics of a large number of lenses are used to improve the generalization of the intelligent core-adjusting model to different lenses, and the model is fine-tuned through a small number of actual pictures in the actual core-adjusting process, so that any lens can be quickly core-adjusted using one model. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The block diagram is a general lens intelligent core adjustment method based on lens database and optical simulation;

[0038] Figure 2 It is a simplified schematic diagram of a universal lens intelligent core adjustment method based on lens database and optical simulation;

[0039] Figure 3 This is a schematic diagram of the structure of the universal lens intelligent core adjustment model;

[0040] Figure 4 Schematic diagram of the simulation optical system. DETAILED DESCRIPTION

[0041] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention may be combined accordingly without conflicting with each other.

[0042] The accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0043] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0044] like Figure 1 As shown, the universal lens intelligent core tuning method proposed in the present invention includes a simulation lens database construction part, an optical simulation part, an intelligent core tuning model training and fine-tuning part, and a new lens core tuning part. First, a simulation lens database is constructed; the lens design file in the simulation lens database is added to the simulation optical path system, and the point spread function matrix, ideal degraded image and simulated degraded image of each field of view imaging sensor under different misalignment are obtained in turn by using an optical simulation algorithm, and image-misalignment data pairs covering all lenses in the lens library under different misalignments are obtained as the core tuning data set, and the intelligent core tuning model is trained, and the intelligent core tuning model is fine-tuned based on a small amount of new lens actual data to obtain a core tuning model adapted to the new lens; the actual pictures collected during the assembly process of the new lens are used as input, and the core tuning model adapted to the new lens is used to quickly predict the misalignment.

[0045] In this embodiment, a universal lens intelligent core adjustment method based on a lens database and optical simulation mainly includes the following steps:

[0046] S1, using an automatic lens design algorithm to generate candidate simulated lens structures under different design specifications, the automatic lens design algorithm uses sampled design specifications and randomly generated lens parameters as input, and uses simulated lens structures under corresponding design specifications as output.

[0047] S2, screening the candidate simulated lens structures generated by S1 to construct a simulated lens database; in the screening process, based on the aberration distribution characteristics of the lens samples and the lens processing constraints, a preset number of simulated lens structures are sampled from the candidate simulated lens structures to construct a simulated lens database.

[0048] S3, constructs an intelligent core-tuning dataset, takes the clear extended image Igt as input, generates the degraded image Idg with misalignment and the misalignment label truth corresponding to each simulated lens in the simulated lens database through the simulation algorithm, forms an image-label data pair {Idg, truth}, and outputs the core-tuning dataset.

[0049] 4) Train a universal lens intelligent alignment model, and use the trained lens intelligent alignment model to add a small amount of real-time images to quickly align any lens during the actual alignment process.

[0050] like Figure 2 As shown, the present invention obtains a universal core-tuning model by training a simulation data set of several lenses. After fine-tuning the universal core-tuning model using a small amount of actual data of lens X, the lens X core-tuning model can be obtained, which can be applied to batch core-tuning of lens X.

[0051] In the above step S1, the present invention proposes an automatic lens design algorithm to generate candidate simulation lens structures for constructing a simulation lens database. The algorithm can generate optical imaging lens samples with various structures that meet actual processing requirements, covering a large number of aberration characteristic distributions.

[0052] In a specific implementation of the present invention, the process of implementing the automatic lens design algorithm includes:

[0053] 1.1) Randomly initialize the lens group P according to the design specifications. The lens group P includes several simulation lenses. The design specifications of each simulation lens include the field of view angle, F number, aperture position and the number of lenses. The value of the design specification is obtained by sampling from a preset range, wherein the field of view sampling range is 20 degrees to 40 degrees, the sampling interval is 2 degrees, the F number sampling range is 2.0 to 5.0, the sampling interval is 0.3, the aperture position sampling is all possible positions of the aperture stop under the given number of lenses, and the number of simulation lenses is set to 1 to 6. Through the above given range, sampling from the range can generate the design specification parameters for initializing the lens group P; in addition, it is necessary to randomly generate the physical parameters of each lens in each simulation lens of the lens group P and the air spacing between two adjacent lenses. The physical parameters of the lens include curvature, glass thickness, air spacing, material refractive index and material Abbe number; the number of initialization iterations k=0.

[0054] In this embodiment, a lens group is obtained by normalizing the value range of the physical parameters of the lens and the air spacing between two adjacent lenses, and randomly generating a parameter vector group in the range of 0 to 1. A simulated lens is initialized by combining the field of view angle, F number, aperture position, etc. in the design specifications. In this step, a number of simulated lenses are randomly initialized and defined as a lens group P, and each lens group constitutes a simulated lens.

[0055] 1.2) Globally optimize the current lens group P to obtain a globally optimized lens group P*. In this embodiment, any one of the heuristic global search algorithms such as simulated annealing algorithm, particle swarm algorithm, ant colony algorithm, etc. is used to achieve global optimization. The global optimization refers to optimizing the physical parameters of each lens in each simulated lens and the air spacing between two adjacent lenses. The number of simulated lenses in the lens group P* after global optimization remains unchanged.

[0056] 1.3) A group of simulation lenses with higher evaluation function values ​​are selected from the lens group P* after global optimization to form the lens group Q. In this embodiment, the simulation lenses in the lens group P* are sorted by the evaluation function values, and a preset number of simulation lenses are selected from high to low to form the lens group Q.

[0057] 1.4) Locally optimize the lens group Q to obtain a locally optimized lens group Q*. In this embodiment, any one of the damped least squares method, ADAM algorithm, quasi-Newton method, etc. is used to achieve local optimization, and the local optimization refers to optimizing the physical parameters of each lens in each simulated lens and the air spacing between two adjacent lenses, and the number of simulated lenses in the locally optimized lens group Q* remains unchanged.

[0058] 1.5) Update the number of iterations k=k+1, determine whether the number of iterations meets the preset number, if yes, output the locally optimized lens group Q*; otherwise, mutate the locally optimized lens group Q* to obtain a lens group M with the same number of simulated lenses, merge Q* and M to replace the current lens group P, and return to step 1.2). In this embodiment, the mutation operation randomly changes one or more parameters of the physical parameters of each lens in the simulated lens of the lens group Q* and the air spacing between two adjacent lenses, wherein the sum of the glass thickness and the space spacing remains unchanged before and after the mutation.

[0059] In a specific implementation of the present invention, the evaluation function used to achieve global optimization and local optimization can be the same as the evaluation function used in step 1.3). The evaluation function is used to evaluate the quality of the current simulated lens. The evaluation function of the simulated lens can be implemented using existing methods in the field, which will not be described in detail in the present invention. The goals of local optimization and global optimization are the same, and they are implemented using global optimization algorithms and local optimization algorithms respectively. The difference is that the core idea of ​​local optimization is to use the gradient descent algorithm to handle differentiable optimization problems, while global optimization can handle nonlinear, non-convex and non-differentiable optimization problems and jump out of the local optimal solution.

[0060] The lens sample aberration distribution characteristics in the above step S2 include the severity of the lens sample aberration and the degree of difference between the lens samples, specifically:

[0061] a) The average value of the RMS (Root Mean Square) radius of the spot diagram of each field of view of the simulated lens sample, indicating the severity of the aberration of the lens sample;

[0062] b) The absolute value difference between the point spread function arrays of each simulated lens sample, indicating the degree of difference between the lens samples;

[0063] The sampling strategy needs to meet the following requirements: the sampled simulated lens samples cover the severity of the evenly distributed lens sample aberrations, and the difference between the lens samples is greater than the set threshold.

[0064] Before sampling, the simulated lens structures that do not meet the lens processing constraints are first eliminated from the lens group Q*.

[0065] The clear extended image Igt in the above step S3 is a commonly used extended image for imaging detection, such as a cross slit graticule, a chessboard, etc., which serves as the basis for subsequent imaging simulation.

[0066] In a specific implementation of the present invention, the specific process of the simulation algorithm is:

[0067] 3.1) Constructing a simulated optical path system, adding the lens design file in the lens database into the simulated optical path system; the simulated optical path system is provided with simulated industrial cameras under multiple fields of view, the simulated industrial camera includes a focusing lens and a sensor arranged at the focus of the focusing lens, and the sensor is used for imaging.

[0068] The present invention designs corresponding sensors in different fields of view for imaging, thereby expanding the field of view range.

[0069] Figure 4This is a schematic diagram of the principle of the simulation optical path system. After the image of the cross-reticle passes through the simulation lens, the cross-reticle lines corresponding to different fields of view are obtained as the clear extended image Igt of the field of view, and the lens group is controlled by six-degree-of-freedom precision displacement.

[0070] 3.2) Divide each simulated lens into two groups according to the tolerance analysis results of each simulated lens in the simulated lens database, add an offset between the two groups, and preset the pupil sampling frequency, image plane sampling frequency and image plane sampling interval according to the sensor performance in the simulated industrial camera so that the pupil sampling frequency, image plane sampling frequency and image plane sampling interval match the sensor performance;

[0071] The clear expanded image Igt is used as input in each field of view. The point spread function matrix of each field of view of each simulated lens in the simulation lens database under different misalignment is calculated through the ray tracing algorithm, and the corresponding misalignment is used as the common label truth of each field of view point spread function matrix; at the same time, the corresponding illumination distribution matrix and distortion distribution matrix under each misalignment are obtained.

[0072] In this step, the value range of pupil sampling frequency, image plane sampling frequency and image plane sampling interval can be preset according to the actual situation, and the value range must conform to the actual situation of the industrial camera. Randomly sample each parameter and modify the sensor performance of the corresponding simulated industrial camera. Under each simulation lens and each misalignment, multiple groups of point spread function matrices with different fields of view can be obtained. The point spread function matrices of multiple groups of different fields of view are fused according to the corresponding field of view to obtain a group of point spread function matrices with different fields of view under each simulation lens and each misalignment. Here, within the above-mentioned preset reasonable range, the influence of different sensor performance on the point spread function matrix is ​​introduced as noise into the subsequent training of the intelligent core tuning model to improve the generalization performance of the intelligent core tuning model. This embodiment adopts a method for batch acquisition of simulation data under different misalignments, first presets the value range of the misalignment under each degree of freedom, and then samples and generates a combination of misalignments under different degrees of freedom according to the actual situation in each value range as a misalignment scheme. Here, the misalignment is a plurality of degrees of freedom, and only 2 degrees of freedom are taken as an example in the figure. The sampling method in each value range may be any one or more combinations of random sampling, uniform sampling, Gaussian distribution sampling, and sampling with a greater sampling density when the offset is smaller.

[0073] 3.4) Through the fast convolution algorithm, a series of clear extended images Igt* at different fields of view are convolved with the point spread function matrix at the corresponding fields of view under different misalignment amounts to obtain the ideal degraded image;

[0074] In this step, since each field of view corresponds to a clear expanded image Igt*, and each field of view corresponds to a series of point spread function matrices under each misalignment, the clear expanded image Igt* of each field of view is convolved with each point spread function matrix of the field of view under a fixed misalignment A to obtain a series of ideal degraded images under each field of view. This series of ideal degraded images uses the misalignment A as a shared label. By traversing the simulation data under different misalignments, a large number of data pairs of ideal degraded images and misalignments can be generated.

[0075] 3.5) Through the post-compensation algorithm, the ideal degraded image is multiplied by the corresponding illumination distribution matrix and distortion distribution matrix under the corresponding misalignment to obtain a series of simulated degraded images Idg* that are close to the actual acquisition, and the image-label data pair {Idg*, truth} is formed with its misalignment label truth as the core adjustment data set.

[0076] The universal lens intelligent core adjustment model described in step S4 above includes a degradation feature extraction module and a misalignment amount prediction module, such as Figure 3 As shown, with the simulated degraded image as input, the image features are first obtained through the feature extraction network, and then the misalignment corresponding to the simulated degraded image is predicted through the misalignment prediction network, and the parameters of the degradation feature extraction module and the misalignment prediction module are updated by back propagation according to the loss between the predicted misalignment and the misalignment label. In this embodiment, the degradation feature extraction module adopts a convolutional neural network structure, such as ResNet, MobileNet, etc., to extract the degradation features of the degraded image; the misalignment prediction module adopts an MLP network to map the degradation features to multi-dimensional misalignment parameters.

[0077] The training of the universal lens intelligent core tuning model includes two stages: pre-training and fine-tuning. The process is as follows:

[0078] 4.1) Pre-training of the universal lens intelligent core adjustment model

[0079] 4.1.1) Initialize model parameters;

[0080] 4.1.2) Batch read in the core adjustment data set, including the image-misalignment amount data pairs of lenses with different aberration characteristics in the lens library under misalignment state, and perform corresponding preprocessing on them; in this embodiment, the preprocessing includes cropping the region of interest of the read image;

[0081] 4.1.3) Send the simulated degradation image Idg to the degradation feature extraction module to obtain its high-dimensional degradation features, and then send it to the misalignment prediction module to output the misalignment result predicted by the model;

[0082] 4.1.4) Calculate the loss function, that is, calculate the root mean square loss between the misalignment prediction and the misalignment label;

[0083] 4.1.5) Perform reverse gradient propagation according to the loss function calculated in 4.1.4) to update the network parameters of the degradation feature extraction module and the misalignment prediction module, and determine whether the current number of iterations has reached the set maximum number of iterations. If so, end the training. If not, determine whether the prediction accuracy of the model on the validation set has not improved after the set number of iterations. If so, end the training in advance. If not, return to step 4.1.2) to continue training.

[0084] 4.2) Fine-tuning of the universal lens intelligent core-alignment model

[0085] The lens to be aligned is placed on the alignment device, and a small number of clear expanded images (such as cross slits) are collected as degraded images through the lens and the misalignment labels are marked to build a fine-tuning training set for the lens. The pre-trained general lens intelligent alignment model is fine-tuned for the second time using the fine-tuning training set. After fine-tuning, the model can quickly adapt to the aberration distribution of the new lens, thereby accurately predicting its misalignment. After that, the lens of this structure can be aligned and produced on a large scale.

[0086] The present invention uses an automatic lens design algorithm to construct a lens database, generate optical imaging lens samples with various structures and meeting actual processing requirements, and cover a large number of aberration characteristic distributions. An optical simulation algorithm is used to add misalignment to various lens samples in the lens database, and a ray tracing method is used to calculate the point spread function with the misalignment label, and then a fast convolution algorithm and a compensation algorithm are used to obtain an extended image with corresponding aberration degradation (such as a cross slit, a checkerboard, etc.). The optical simulation algorithm used in the present invention provides a large number of reliable training data sets for the intelligent core tuning model, including extended images close to actual data and corresponding misalignment label data pairs. The present invention also proposes a universal intelligent core tuning model, which uses the aberration characteristic priors of a large number of lenses to improve the generalization of the model to different lenses, and fine-tunes the model through a small number of actual pictures in the actual core tuning process, so as to achieve rapid core tuning of any lens using only one model.

[0087] Based on the same inventive concept, a universal lens intelligent core adjustment system based on a lens library and optical simulation is also provided in this embodiment, including:

[0088] A simulation lens database module, which is used to construct a simulation lens database;

[0089] An optical simulation preprocessing module, which is used to add the lens design file in the simulation lens database into the simulation optical path system; the simulation optical path system is provided with a plurality of simulated industrial cameras under the field of view, and the simulated industrial camera includes a focusing lens and an imaging sensor arranged at the focus of the focusing lens;

[0090] The optical simulation module is used to calculate the point spread function matrix of the imaging sensor of each field of view of the simulation lens under different misalignment amounts with a clear expanded image as input under each field of view, and use the corresponding misalignment amount as the label of the point spread function matrix; at the same time, obtain the illumination distribution matrix and distortion distribution matrix corresponding to each misalignment amount;

[0091] A simulated degradation calculation module is used to calculate a series of ideal degraded images under different misalignment amounts according to the clear extended image and the point spread function matrix; the ideal degraded image is optimized based on the post-compensation algorithm using the illumination distribution matrix and the distortion distribution matrix to obtain a simulated degraded image;

[0092] A training module, which is used to train a universal lens intelligent core-tuning model using simulated degraded images and misalignment labels;

[0093] The intelligent core-tuning module is used to fine-tune the universal lens intelligent core-tuning model using the actual data of the lens to be tuned, and use the fine-tuned model to perform intelligent core-tuning of lenses with the same optical structure.

[0094] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be repeated here. The system embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0095] The embodiments of the system of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The system embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the corresponding computer program instructions in the non-volatile memory are read into the memory by the processor of any device with data processing capabilities and run.

[0096] The above-mentioned embodiments only express several implementation modes of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. For those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A universal lens intelligent core alignment method based on lens library and optical simulation, characterized in that: include: Build a simulation lens database; Add the lens design file in the simulation lens database into the simulation optical path system; The simulated optical path system is provided with simulated industrial cameras under multiple fields of view, and the simulated industrial camera includes a focusing lens and an imaging sensor arranged at the focus of the focusing lens; Under each field of view, the clear expanded image is used as input, and the point spread function matrix of the imaging sensor of each field of view of the simulation lens under different misalignment is calculated, and the corresponding misalignment is used as the label of the point spread function matrix; at the same time, the illumination distribution matrix and distortion distribution matrix corresponding to each misalignment are obtained; A series of ideal degraded images under different misalignment amounts are calculated based on the clear extended image and the point spread function matrix; the ideal degraded image is optimized based on the post-compensation algorithm using the illumination distribution matrix and the distortion distribution matrix to obtain a simulated degraded image; Use simulated degraded images and misalignment labels to train a universal lens intelligent alignment model; The real data of the lens to be aligned is used to fine-tune the universal lens intelligent alignment model, and the fine-tuned model is used to perform intelligent alignment of lenses with the same optical structure.

2. The universal lens intelligent core alignment method based on lens library and optical simulation according to claim 1, characterized in that: The simulation lens database construction process includes: An automatic lens design algorithm is used to generate candidate simulated lens structures under different design specifications, wherein the automatic lens design algorithm takes the sampled design specifications and the randomly generated lens parameters as input and takes the simulated lens structure under the corresponding design specifications as output; Eliminate candidate simulation lens structures that do not meet the lens processing constraints, screen the remaining candidate simulation lens structures based on the sampling strategy, and build a simulation lens database; The sampling strategy described meets the following requirements: The sampled simulated lens samples cover the severity of the lens sample aberrations that are evenly distributed, and the difference between the lens samples is greater than a set threshold.

3. The universal lens intelligent core alignment method based on lens library and optical simulation according to claim 2, characterized in that: The severity of the lens sample aberration refers to the average value of the root mean square radius of the point diagram of each field of view of the simulated lens structure, and the difference between the lens samples refers to the absolute value difference between the point spread function matrices of the simulated lens structure.

4. The universal lens intelligent core alignment method based on lens library and optical simulation according to claim 1, characterized in that: The method of taking the clear expanded image as input in each field of view, calculating the point spread function matrix of the imaging sensor in each field of view of the simulation lens under different misalignment amounts, and taking the corresponding misalignment amount as the label of the point spread function matrix includes: Divide the simulated lens structures into two groups according to the tolerance analysis results of each simulated lens structure in the simulated lens database; Under each field of view of the simulated optical path system, the pupil sampling frequency, image plane sampling frequency and image plane sampling interval matching the performance of the imaging sensor are preset; Add an offset between two groups of the simulated lens structure, take a clear extended image as input, and calculate the point spread function matrix of each field of view imaging sensor of the simulated lens under the offset through a ray tracing algorithm; traverse the imaging sensor performance range and the simulated lens structure to obtain the point spread function matrix of each field of view corresponding to the offset under different performance parameters of the simulated lens structure; The misalignment range is traversed to obtain a series of point spread function matrices of each field of view at each misalignment, wherein the labels of the point spread function matrices are the corresponding misalignments.

5. The universal lens intelligent core alignment method based on lens library and optical simulation according to claim 4, characterized in that: The clear extended images are different in different fields of view.

6. The universal lens intelligent core alignment method based on lens library and optical simulation according to claim 5, characterized in that: The method of calculating a series of ideal degraded images under different misalignment amounts according to the clear extended image and the point spread function matrix includes: Under each misalignment, a series of point spread function matrices of each field of view are convolved with the clear extended image under the field of view to obtain a series of ideal degraded images under the misalignment, and the labels of the series of ideal degraded images are the corresponding misalignments.

7. The universal lens intelligent core alignment method based on lens library and optical simulation according to claim 1, characterized in that: The post-compensation algorithm is specifically: The ideal degraded image is multiplied by the illumination distribution matrix and the distortion distribution matrix under the corresponding misadjustment to optimize the ideal degraded image.

8. The universal lens intelligent core alignment method based on lens library and optical simulation according to claim 1, characterized in that: The lens intelligent core adjustment model adopts a convolutional neural network structure, takes a simulated degraded image as input, and predicts the misalignment amount.

9. The universal lens intelligent core alignment method based on lens library and optical simulation according to claim 1, characterized in that: The misalignment values ​​described are multi-degree-of-freedom values.

10. A universal lens intelligent core adjustment system based on lens library and optical simulation, characterized in that: include: A simulation lens database module, which is used to construct a simulation lens database; An optical simulation preprocessing module, which is used to add the lens design file in the simulation lens database into the simulation optical path system; the simulation optical path system is provided with a plurality of simulated industrial cameras under the field of view, and the simulated industrial camera includes a focusing lens and an imaging sensor arranged at the focus of the focusing lens; The optical simulation module is used to calculate the point spread function matrix of the imaging sensor of each field of view of the simulation lens under different misalignment amounts with a clear expanded image as input under each field of view, and use the corresponding misalignment amount as the label of the point spread function matrix; at the same time, obtain the illumination distribution matrix and distortion distribution matrix corresponding to each misalignment amount; A simulated degradation calculation module is used to calculate a series of ideal degraded images under different misalignment amounts according to the clear extended image and the point spread function matrix; the ideal degraded image is optimized based on the post-compensation algorithm using the illumination distribution matrix and the distortion distribution matrix to obtain a simulated degraded image; A training module, which is used to train a universal lens intelligent core-tuning model using simulated degraded images and misalignment labels; The intelligent core-tuning module is used to fine-tune the universal lens intelligent core-tuning model using the actual data of the lens to be tuned, and use the fine-tuned model to perform intelligent core-tuning of lenses with the same optical structure.

Images