Method for aligning lens system and device for aligning lens system
Patent Information
- Application Number
- JP2022178650
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-11-09
- Filing Date
- 2022-11-08
- Publication Date
- 2025-11-17
AI Technical Summary
Existing methods for aligning lens systems in optical systems are time-consuming due to the need for fine quantization and extensive evaluation of alignments, exacerbated by manufacturing tolerances, which make it difficult to efficiently determine proper alignment.
A method utilizing machine learning systems to reduce the number of alignments required by training a machine learning system to predict suitable alignments based on characteristic values of refracted optical signals, employing iterative and optimization techniques to converge quickly on the optimal alignment.
This approach significantly reduces the time needed to align lens systems by minimizing the number of alignments evaluated, allowing for faster and more efficient alignment processes.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method for determining the alignment of a lens system, an apparatus for aligning a lens system, a computer program, and a machine-readable storage medium.
Background Art
[0002] Advantages of the Invention During the manufacture of an optical system such as an optical sensor, a telescope or a microscope, it is repeatedly a problem that a lens system, for example one or more lenses, should be properly aligned in the objective lens of the optical system. For example,it may be necessary that the lens system is aligned such that its focus is located at a point where it can be preset in advance, and / or that the focal length of the optical system reaches a preset value.
[0003] When both the lens system and other components of the optical system are affected by manufacturing tolerances, finding the proper alignment of the lens system becomes a difficult problem. Depending on the corresponding tolerances, situations arise where it cannot be generally assumed that the proper alignment of the first lens system is the proper alignment of the second lens system. Therefore, when a large number of optical systems are to be manufactured, a method for aligning the lens system to fit the optical system is desired (in the sense that the characteristics vary variously among products of the same manufacturing process).
[0004] In particular, when the lens systems of multiple optical systems need to be aligned, it is desirable that the lens system alignment process be performed in the shortest possible time. For example, when aligning multiple pre-configurable alignments, it is possible to determine whether each alignment of the lens system is suitable for the optical system. For this purpose, the possible alignments of the lens system may be quantized in equally spaced steps, and the lens system may be aligned in accordance with these quantized alignments. This approach is also known as grid search. [Overview of the Initiative] [Problems that the invention aims to solve]
[0005] However, the drawback of such an exhaustive approach is that the quantization must usually be extremely fine and "not skipped" so that a suitable alignment is at least approximately captured by the quantization. Therefore, this form of finding the alignment of a lens system usually takes a lot of time because the lens system needs to be aligned and evaluated according to all the alignments of the grid.
[0006] It is desirable to minimize the number of alignments that need to be inspected. This is a difficult problem because, due to the tolerances mentioned above, it is only possible to limit the alignment to one that is suitable for the corresponding lens system within a predetermined range.
[0007] A method having the features of independent claim 1 can determine the appropriate alignment of a lens system based on a machine learning system. The inventors were able to confirm that by appropriately incorporating a machine learning system, the number of alignments that need to be evaluated is significantly reduced compared to known methods. [Means for solving the problem]
[0008] Disclosure of the invention In a first embodiment, the present invention relates to a method for determining the alignment of a lens system, the method being a. A step of aligning the lens system according to the first alignment provided, b. A step of determining a first refracted optical signal by the refraction of a first emitted optical signal in a lens system aligned according to a first alignment, c. A step of determining a first characteristic value that represents the characteristics of the first refracted light signal, d. A step of training a first machine learning system in relation to a first alignment and a first determined characteristic value, wherein the machine learning system is configured to determine an output representing the characteristics of the alignment. e. A step of determining the alignment of the lens system based on the output of the first machine learning system, Includes.
[0009] A lens system can be understood as a single lens. Selectively, a lens system can also be understood as multiple lenses. For example, a single objective lens can be understood as a single lens system.
[0010] The alignment of a lens system can be understood as the relative position of the lens system with respect to a predetermined point. For example, the lens system may be aligned with respect to a desired focal point and / or a desired focal length. In other words, alignment represents the degree of freedom to which the lens system can be aligned in three-dimensional space accordingly.
[0011] Alignment can be understood, in particular, as the orientation and positioning of a lens system in three-dimensional space. Specifically, alignment can be represented by a six-dimensional vector, in which case the vector represents the position along three axes in three-dimensional space, and, alongside this, one rotation about each of these axes. Optionally, alignment can also be represented by a quaternion. Optionally, alignment can also be represented by Euler angles.
[0012] The method for determining alignment can be understood, in particular, as a method implemented in a computer. That is, the steps described above are performed by a computer. In this case, the step of aligning the lens system can be understood as the step in which the computer determines the drive control signals that appropriately align the lens system.
[0013] This method can be understood as a method for determining a first characteristic value for a first alignment, and then determining, based on the first characteristic value, whether or not this alignment is appropriate with respect to that characteristic. For this purpose, an optical signal is transmitted through a lens system aligned to correspond to the first alignment (transmitted optical signal). The transmitted optical signal is refracted by this lens system. The thus refracted optical signal can then be received, for example, in a receiving unit. Subsequently, the characteristic value of the refracted signal can be determined.
[0014] Preferably, the transmitted optical signal is transmitted by a transmitting unit. Preferably, the transmitting unit and the receiving unit are further fixed in position and orientation, so that the first characteristic value for the lens system is affected only by the first alignment.
[0015] The characteristics may represent, for example, the focusing, intensity, or position of the refracted light signal, as measured by a receiving unit. Since the refracted light signal is primarily determined by a first alignment, the determined first characteristic value can be understood as a gauge for the first alignment. Generally, from here on, characteristic values relating to alignment are understood to be characteristic values determined when a lens system is aligned according to this alignment and the refracted light signal is evaluated with respect to the characteristic value. In particular, for example, if multiple characteristics of the alignment are to be examined or should be included in the optimization of the alignment of the lens system, multiple first characteristic values may be determined by a first machine learning system.
[0016] The objective of the method may preferably be to align a lens system such that one first characteristic value lies within a pre-settable range, or that multiple first characteristic values lies within each of the pre-settable ranges. Advantageously, for this purpose, an appropriate alignment of the lens system is estimated based on the output of a first machine learning system. For this purpose, the lens system is first aligned according to a first alignment, and one or more first characteristic values are determined. Subsequently, the first machine learning system may be trained to predict one or more first characteristic values with respect to the first alignment. In other words, the first machine learning system may be trained to learn, specifically for each lens system, which alignment leads to which characteristic value. If the method is used to align multiple lenses, one first machine learning system specific to each lens system may be trained.
[0017] Preferably, the method may provide a plurality of first alignments, for each of these first alignments one or more first characteristic values may be determined. The output of the first machine learning system can be understood as an estimate of one or more characteristic values of the alignments passed to the first machine learning system.
[0018] To determine one or more characteristic values with respect to alignment, the lens system is aligned in accordance with this alignment, and only the first machine learning system needs to be evaluated to estimate one or more characteristic values. The inventors were able to confirm that this method significantly reduces the number of second alignments that need to be determined until the alignment of the lens system is appropriate, and therefore significantly speeds up the alignment process of the lens system.
[0019] The remainder of the specification describes embodiments relating, in particular, to the determination of characteristic values and the output of characteristic values. It should be understood throughout that it is obvious to those skilled in the art that multiple characteristic values can also be determined or output.
[0020] Advantageously, this method requires only a relatively small number of primary alignments to be adjusted and evaluated for the lens system. The estimation process can be understood as a virtual alignment and evaluation of the lens system. This allows for a significantly faster evaluation of numerous alignments compared to the actual alignment and evaluation of the lens system.
[0021] In particular, it can be understood that the alignment of the lens system according to the alignment is that the lens system is aligned on the test stand. In this case, the evaluation of the alignment characteristics is that on the test stand, an optical signal is transmitted through the lens system, the optical signal is refracted in the lens system, and subsequently, in order to determine the characteristic value representing the characteristics, the refracted optical signal is processed by the receiving unit.
[0022] In various preferred embodiments of this method, the step of determining the alignment of the lens system based on the output of the first machine learning system is f. determining a second alignment such that the output of the first machine learning system determined for the second alignment is within a pre-set value range; g. determining a second refracted optical signal by refraction of the second transmitted optical signal in the lens aligned according to the second alignment; h. determining a second characteristic value representing the characteristics of the second refracted optical signal; i. repeating steps d., f., g., and h. when the second characteristic value for the second alignment is not within the pre-set value range for the second characteristic value, where the second alignment is used as an additional first alignment for training the first machine learning system, providing the second alignment as the alignment of the lens system when the second characteristic value for the second alignment is within the pre-set value range for the second characteristic value; may be included.
[0023] The above embodiments can be understood as an iterative method. In each iteration, one alignment is determined, the machine learning system is trained by this alignment, and subsequently, a better alignment of the lens system is determined based on the output of the machine learning system. It can be understood that the estimation of the machine learning system regarding the proper alignment of the lens system is actually inspected regarding the alignment of the lens system and regarding the determination of the characteristic values regarding the alignment. Thereby, the machine learning system converges more and more in the direction of estimating the actual characteristic values regarding the alignment more accurately. The inventors have advantageously been able to confirm that this convergence starts extremely quickly, and thus, the proper alignment of the lens system is determined after only a few iterations at an early stage.
[0024] Advantageously, the pair of the second alignment and the second characteristic value determined in an iteration can be added to the training data of the first machine learning system in a subsequent iteration. Thereby, more knowledge regarding the relationship between the alignment and the characteristic values regarding this alignment is transmitted to the first machine learning system in each iteration. Thereby, this method converges more quickly.
[0025] Such a technique can also be understood as a form of active learning. An iteration point is determined by the first machine learning system (the second alignment), and for this iteration point, in actuality, the values to be specified by the first machine learning system (the second characteristic values) are determined respectively. The pairs determined in this way can then be used for further training of the first machine learning system.
[0026] Therefore, the laborious calculation of a suitable alignment can be advantageously determined by optimizing an approximation, which is determined by a machine learning system. Subsequently, the alignment obtained by the optimization of the approximation may be tested, which is done by aligning the lens system in accordance with the obtained alignment. Through this iterative method, the approximation becomes increasingly accurate, thereby advantageously finding a suitable alignment after only a few iterations.
[0027] The second alignment can be considered appropriate if the second characteristic value falls within a pre-set range. The pre-set range for the second characteristic value may, in particular, be the same as the pre-set range for the first characteristic value.
[0028] To identify the second alignment, often as an optimization problem, the alignment is provided as the second alignment, and the corresponding second characteristic value is maximized or minimized with respect to it.
[0029] Preferably, in this method, the second alignment can also be determined on an optimization basis, in which case the constraints represent adherence to at least one boundary of a pre-set range of values for the second characteristic value.
[0030] Optimization is preferably performed using the following formula
number
[0031] In all forms of this method, it is possible, in principle, for more than one characteristic to be examined for a single alignment. In cases where the second characteristic value is directly optimized, Pareto optimization may be performed when there are multiple characteristics to be examined (i.e., when there are multiple first characteristic values or multiple second characteristic values). In the case of optimization under constraints, each characteristic may be given a set of constraints regarding a predetermined range of values.
[0032] Preferably, the first machine learning system may be pre-trained first. In this case, the subsequent training of the first machine learning system can be understood as fine-tuning of the first machine learning system. This pre-training enables the first machine learning system to determine sufficiently accurate estimates after only a few iterations. Pre-training may be performed, for example, by approaching one or more lens systems with different first alignments on a test stand and determining the corresponding first characteristic values. The pairs of first alignments and first characteristic values thus determined may then be used as the training dataset for pre-training the first machine learning system.
[0033] The pre-training of the first machine learning system may also include pruning of the first machine learning system, which, for example, involves removing trainable parameters from the first machine learning system. The parameters to be pruned can be determined, in particular, based on a validation dataset.
[0034] Preferably, the first machine learning system may include a polynomial model, which is configured to determine an output that represents a characteristic for alignment purposes.
[0035] The polynomial model may be configured to estimate one or more first characteristic values with respect to a first alignment. The machine learning system may also include multiple polynomial models, and in particular, one polynomial model for the first characteristic value to be predicted. The advantage of using a polynomial model is that it can be trained very quickly. This further reduces the time required to determine the alignment of the lens system.
[0036] In a preferred form of the method, it is also possible to provide multiple first alignments in step a. based on a Bayesian optimization method.
[0037] Advantageously, this method allows us to determine multiple first alignments that represent at least one initial constraint on a suitable alignment. This further accelerates the convergence of the approximation of the first machine learning system.
[0038] It is also possible to selectively determine the first alignment based on a second machine learning system, in which case the second machine learning system is configured to identify alignment changes that lead to an appropriate alignment based on the alignment.
[0039] For example, it is possible to randomly identify a provisional alignment, in which case the provisional alignment is preferably iterated over multiple iterative steps and improved based on a second machine learning system. In each iteration, for example, a change to the provisional alignment may be determined by the second machine learning system, the provisional alignment is adjusted according to the determined change, and the adjusted alignment is provided as the provisional alignment for the next iteration. Subsequently, one or more provisional alignments may be used in this method as one or more first alignments. Advantageously, one or more first alignments may be appropriately limited as early as before the implementation of this method, thereby allowing the method to converge more rapidly and thus enabling faster determination of the alignment of the lens system.
[0040] The second machine learning system may include, in particular, a neural network configured to predict appropriate changes for alignment.
[0041] Preferably, the second machine learning system may be trained using a reinforcement learning method.
[0042] The input of a reinforcement learning method defines states and actions for a second machine learning system. Alignment, in particular, can be used as a state. Thus, the second machine learning system may be configured to process alignment. Selectively or additionally, one or more characteristic values determined for alignment can also be used as states. The actions determined by the second machine learning system may, in particular, be changes indicating how the alignment passed to the machine learning system should be modified.
[0043] A reward function is used to train a reinforcement learning method. In this method, the reward function may be the cumulative quality of each determined provisional alignment over multiple actions of the second machine learning system. For example, to determine the training data for the second machine learning system, multiple further provisional alignments may be determined based on randomly selected provisional alignments, which is done by determining a provisional alignment at each iteration step based on the preceding provisional alignment. Selectively or additionally, multiple provisional alignments may also be used as training data for the second machine learning system, which may be, for example, a predetermined number of alignments along a grid in the alignment space. For each of the multiple provisional alignments, a corresponding characteristic value may be determined on the test stand. Furthermore, for these determined characteristic values, it may be determined how far these characteristic values are from the corresponding boundary of a predetermined range of characteristic values, and simultaneously within this range, where the distance from the boundary can be understood as a quality gauge for each provisional alignment. The reward for the multiple provisional alignments thus determined may be the sum of the quality gauges. Subsequently, a second machine learning system may be trained, preferably using a policy gradient method.
[0044] The advantage of training a second machine learning system is that the second machine learning system is trained so that each change determined by the second machine learning system results in the best possible improvement to the corresponding provisional alignment. Thus, by efficiently restricting one or more first alignments to appropriate values, subsequent determination of the lens system alignment becomes possible in an even shorter time.
[0045] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. [Brief explanation of the drawing]
[0046] [Figure 1] This diagram schematically shows the flow of the method for determining lens alignment. [Figure 2] This is a schematic diagram showing an apparatus for implementing a method for determining lens alignment. [Modes for carrying out the invention]
[0047] Description of the Examples Figure 1 shows a flowchart representing the flow of a method for determining the alignment (A) of a lens. In the first step (101) of the method, a first alignment is initially provided. This alignment may be provided, for example, based on a second machine learning system. Selectively, the first alignment may be determined randomly based on possible numerical values of the alignment. For example, the alignment may be represented by a six-dimensional vector, in which case a six-dimensional random vector may be drawn to provide the first alignment. Preferably, in the first step (101), multiple first alignments may be provided.
[0048] For one or more first alignments, one first characteristic value is determined in the second step (102). Selectively, multiple first characteristic values may be determined for one or more first alignments. For a first alignment, one or more first characteristic values may be determined as follows: First, the lens system is aligned according to the first alignment. Next, an optical signal, for example, an optical signal in the visible region of light, is transmitted through the lens system. The optical signal is refracted by the lens system. Subsequently, the thus refracted optical signal may be received by a receiving unit. The receiving unit may include, for example, an image sensor, which may detect the refracted optical signal.
[0049] Based on the detected signal, one or more first characteristic values are then determined in a third step (103) of method (100). The first characteristic values may represent, for example, a focusing gauge, the intensity of a refractive signal, or the position of a refractive signal with respect to an image sensor.
[0050] If a plurality of first alignments are provided in the first step (101), preferably one or more first characteristic values may be determined for each first alignment.
[0051] Next, all determined pairs of first alignments and one or more first characteristic values are used in the fourth step (104) to train a first machine learning system. The first machine learning system preferably includes a polynomial model configured to predict the corresponding first characteristic values based on the first alignments. The polynomial model may preferably be pre-trained in pre-training preceding method (100). Pre-training may particularly include tuning the coefficients of the polynomial model such that an appropriate degree of the polynomial model is selected. In particular, this may be done by cross-validation. Subsequently, the preferably pre-trained polynomial model may be refined based on these pairs.
[0052] If, for a given alignment, multiple first characteristics should be predicted by the first machine learning system, this polynomial model may be configured to predict all of the first characteristics. Optionally, the first machine learning system may also include a single intrinsic polynomial model to predict one first characteristic each.
[0053] Next, in the fifth step (105), a second alignment is determined based on the first machine learning system trained in the fourth step (104). For this purpose, the optimization problem may be solved such that the characteristic values estimated by the first machine learning system for the second alignment fall within a pre-settable range of values. The pre-settable values may be represented in particular by a lower boundary and / or an upper boundary, where the optimization objective function preferably represents the distance of the estimated characteristic values to the upper boundary and / or the lower boundary. The objective of the optimization can be understood as determining the second alignment such that the characteristic values estimated for the second alignment have the longest possible distance to the lower boundary and / or the upper boundary. To solve the optimization problem, known methods from the domain of nonlinear optimization may be used, for example, nonlinear programming.
[0054] Next, in the sixth step (106), the lens system is aligned in accordance with the second alignment. Subsequently, an optical signal is transmitted through the lens system and the refracted optical signal is determined.
[0055] For the refracted light signal thus determined, a characteristic value may be determined in the seventh step (107) of method (100), as in the case of the first alignment. Such a characteristic value may be provided here as a second characteristic value. Next, it may be checked whether the second characteristic value is within a preset range. If the second characteristic value is within a preset range, the second alignment may be provided as the alignment (A) of the lens system, and the method may be terminated.
[0056] If the second characteristic value is not within a pre-settable range, steps 4(104) to 7(107) of method (100) may be repeated. Preferably, in this case, pairs of second alignments and second characteristic values included in the training set may be provided as further first alignments and further first characteristic values. Steps 4(104) to 7(107) of method (100) may be repeated repeatedly until the second characteristic value is within a pre-settable range.
[0057] Due to physical conditions or an improper first alignment, the method may be unable to determine the alignment, resulting in one or more first characteristic values falling within their respective pre-settable ranges. In such cases, the method may be interrupted after a pre-set number of iteration steps. The pre-set number of iteration steps may be defined in particular based on an expected number of iteration steps, which typically represents the number of steps after the method has identified the alignment such that one or more first characteristic values fall within their respective pre-settable ranges. The pre-set number of iteration steps may, for example, be a multiple of the expected number of iteration steps. Selectively, if the pre-set number of iteration steps is reached or exceeds a pre-set number of iteration steps, the method may be restarted for a new lens system with a new first alignment.
[0058] Figure 2 shows an apparatus (200) configured to carry out method (100). The apparatus (200) includes a control unit (40) configured to carry out the steps of method (100). The control unit (40) controls an actuator (10) which can align the lens system (L) according to a preset alignment. The actuator (10) may be a motor that can mechanically align the lens system (L).
[0059] To determine characteristic values related to alignment, the control unit (40) can drive and control the actuator (10) so that the lens system (L) is aligned according to this alignment. Next, the control unit (40) can drive and control the transmitting unit (U1) so that the transmitting unit transmits an optical signal (E). The optical signal (E) is refracted in the lens system (L), and the refracted optical signal (G) thus determined is received by the receiving unit (U2). The receiving unit (U2) may preferably include an image sensor, by which the refracted optical signal (G) is measured.
[0060] Next, the signal (G) received by the receiving unit (U2) may be evaluated to determine a characteristic value. The characteristic value thus determined may then be transmitted back to the control unit (40). Selectively, the measurement of the image sensor itself may be transmitted to the control unit (40), and subsequently the control unit (40) may determine the characteristic value.
[0061] In another preferred embodiment, the control unit (40) includes at least one processor (45) and at least one machine-readable storage medium (46) which stores instructions for causing the control unit (40) to perform the method (100) when executed on the at least one processor (45).
[0062] The term "computer" includes any device that processes pre-configurable computational rules. Such computational rules may exist in the form of software, in the form of hardware, or in a hybrid form of software and hardware.
[0063] Generally, a “multiple” can be understood as an indexed entity. That is, each element of the “multiple” is assigned a unique index, preferably by assigning a consecutive integer to the elements contained within the “multiple.” Preferably, if the “multiple” contains N elements, then N is the number of elements in the “multiple,” and these elements are assigned integers from 1 to N.
Claims
1. A method (100) for determining the alignment (A) of a lens system (L), comprising: a. aligning (101) the lens system (L) according to a provided first alignment; b. determining (102) a first refracted optical signal (G) by refraction of a first transmitted optical signal (E) in the lens system (L) aligned according to the first alignment; c. determining (103) a first characteristic value representative of a characteristic of said first refracted optical signal (G); d. Training (104) a first machine learning system in association with the first alignment and the determined first feature value, the machine learning system configured to determine an output for the alignment that represents a feature of the alignment; e. determining the alignment (A) of the lens system (L) based on the output of the first machine learning system; A method (100) comprising:
2. pre-training the first machine learning system in a step preceding the method (100); The method (100) of claim 1.
3. The step of determining the alignment (A) of the lens system (L) based on the output of the first machine learning system comprises: f. determining 105 a second alignment such that the output of the first machine learning system determined for the second alignment falls within a preconfigurable range of values; g. determining (106) a second refracted optical signal (G) by refraction of the second transmitted optical signal (E) in the lens system (L) aligned according to the second alignment; h. determining (107) a second characteristic value representative of a characteristic of the second refracted optical signal (G); i. if the second feature value for the second alignment does not lie within a preconfigurable range of values for the second feature value, repeating steps d, f, g, and h, wherein the second alignment is used as an additional first alignment for training the first machine learning system; providing the second alignment as an alignment (A) of the lens system (L) if the second characteristic value for the second alignment is located within a preconfigurable range of values for the second characteristic value; Including, The method (100) of claim 1.
4. determining the second alignment based on optimization; the optimization constraint expresses compliance with at least one boundary of the predefinable value range; The method (100) of claim 3.
5. the first machine learning system includes a polynomial model, the polynomial model configured to determine an output representing the characteristic for alignment. The method (100) of claim 1.
6. said first alignment being provided based on a Bayesian Optimization method, The method (100) of claim 1.
7. determining the first alignment based on a second machine learning system; the second machine learning system is configured to identify, based on the alignment, a change in the alignment. The method (100) of claim 1.
8. training said second machine learning system by a reinforcement learning method, The method (100) of claim 7.
9. the lens system (L) is part of an optical sensor; The method (100) of claim 1.
10. An apparatus (200) for aligning a lens system (L), comprising: An apparatus (200) configured to perform the method according to any one of claims 1 to 9.
11. A computer program configured to perform the method of any one of claims 1 to 9 when executed by a processor (45).
12. A machine-readable storage medium (46) having stored thereon the computer program of claim 11.