Aberration estimating method, aberration estimating device, and program

Abstract

To provide an aberration estimating method and the like that can accurately estimate an aberration of an optical system.SOLUTION: An estimating method for estimating an aberration of an optical system 103 by using a plurality of images acquired by causing an image pick-up device 104 to receive an optical image of a subject 101 formed through the optical system at a plurality of defocus positions different from each other, includes: a step S3 of acquiring the plurality of images; a step S5 of composing the plurality of images to create a composite image; and a step S6 of estimating the aberration of the optical system on the basis of the composite image.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The disclosure of this specification relates to an aberration estimation method, an aberration estimation device, and a program for estimating the aberration of an optical system using a light intensity distribution. [Background technology]

[0002] In optical instruments such as cameras and telescopes, the aberration of optical systems such as lenses is measured to evaluate and guarantee the performance of the instrument. Because aberration measurement requires measuring the phase of light, interferometers and Shack Hartmann sensors have traditionally been used. However, these measurement devices require dedicated optical modules, which are costly and require large equipment.

[0003] Patent Document 1 discloses a method for estimating the aberration of an optical system by optimizing calculations based on light intensity distributions measured at multiple defocus positions, without using a dedicated optical module. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-51039 Summary of the Invention [Problem to be solved by the invention]

[0005] The method described in Patent Document 1 acquires images at multiple defocus positions and estimates the aberration of the optical system based on the images. In the method described in Patent Document 1, there is a risk that sufficient accuracy may not be obtained depending on the image sensor used to acquire the images. [Means for solving the problem]

[0006] An estimation method according to one embodiment of the present invention is a method for estimating the aberration of an optical system using multiple images acquired by having an image sensor receive an optical image of a subject formed through the optical system at multiple defocus positions that are different from each other, and is characterized by having the steps of acquiring the multiple images, synthesizing the multiple images to generate a composite image, and estimating the aberration of the optical system based on the composite image. [Effects of the Invention]

[0007] According to the above-described estimation method, it is possible to provide an aberration estimation method that can estimate the aberration of an optical system with high accuracy. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram of an aberration estimation system according to each embodiment. [Figure 2] 10A and 10B are schematic diagrams of modified examples of the aberration estimation system according to the embodiments. [Figure 3] 10A to 10C are diagrams illustrating an aberration estimation method according to each embodiment. [Figure 4] FIG. 10 is a diagram showing a change in a point image due to defocusing for an aberration-free lens. [Figure 5] FIG. 10 is a diagram showing the change in a point image due to defocusing for a lens with aberration. [Figure 6] FIG. 10 is a diagram showing the relationship between a defocused point image and a sampling position. [Figure 7] 10A and 10B are diagrams illustrating the amount of change in defocus position in each example. [Figure 8] 10 is a flowchart illustrating an aberration estimation method according to each embodiment. [Figure 9] 10 is a flowchart illustrating an example of a method for combining images. [Figure 10] FIG. 10 is a diagram illustrating an example of a light intensity distribution. [Figure 11] FIG. 10 is a diagram illustrating an example of a corrected image. [Figure 12] FIG. 10 is a diagram illustrating an example of a composite image. [Figure 13] FIG. 1 is a diagram showing the estimation results of Example 1. [Figure 14] FIG. 10 is a diagram showing the estimation results of Example 2. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same components are designated by the same reference numerals, and redundant explanations will be omitted.

[0010] First, an aberration estimation system 1 in this embodiment will be described with reference to Fig. 1. Light emitted from an object 101 is incident on a device under test 102. The device under test 102 has an imaging optical system 103 and an image sensor 104, and the light emitted from the object 101 is imaged on the image sensor 104 by the imaging optical system 103. The object 101 is placed on a driving device (motorized stage) 105. The driving device 105 is controlled by a computer (control unit) 106, and moves the object 101 in a direction along the optical axis (optical axis direction) to a specified position.

[0011] The subject 101 is a component that emits light from an aperture. For example, it may be a pinhole, but is not limited to this. It may also be the end of an optical fiber or a distant celestial body. It may also be a white circle painted on a black flat substrate or a small piece of luminous paint. Anything that transmits, scatters, or emits light within a certain area functions as the subject 101. The desired size of the area that transmits, scatters, or emits light is determined by the imaging optical system 103. The size of the image of the subject 101 formed by the imaging optical system 103 should be small compared to the amount of blur of the image caused by defocus. For example, the size of the image is approximately half or less of the amount of blur, preferably 1 / 10 or less. The subject 101 may be configured to include a light source, or a light source (not shown) that illuminates the subject 101 may be used.

[0012] The image sensor 104 is controlled by the computer 106, and acquires a light intensity distribution by receiving an optical image of the object 101 at each position to which the object 101 moves. That is, the image sensor 104 is configured to be able to acquire a plurality of light intensity distributions with different defocus amounts. Alternatively, the light intensity distributions acquired directly may be stored in the computer 106 or a data storage device (not shown). The image sensor 104 may also be controlled via the device under test 102. The image sensor 104 may be, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal-Oxide Semiconductor) sensor.

[0013] In this embodiment, the device under test 102 has a configuration in which the imaging optical system 103 and the image sensor 104 are integrated, but this is not limited to this. The imaging optical system 103 may be any optical system that forms an image of the subject 101.

[0014] In this embodiment, the optical system and the image sensor may be separated from each other instead of being integrated. This configuration makes it possible to measure an object that does not include an image sensor. For example, this is an interchangeable lens for an interchangeable lens camera or a telescope for astronomical photography.

[0015] The driving device 105 is a device that can be driven in one dimension, such as an electric stage. Note that the driving method is not limited to this, as long as it can change the amount of defocus during imaging. For example, a focus adjustment mechanism included in the imaging optical system 103 may be used, or as shown in FIG. 2, a collimator lens 108 that collimates light emitted from the subject 101 may be driven. Furthermore, a spatial phase modulator or the like may be placed in the optical path to control the phase of the light and provide defocus.

[0016] The computer 106 performs post-processing on the acquired light intensity distribution to estimate the aberration of the device under test 102. In this case, the computer 106 functions as a calculation unit. The post-processing may be performed by the computer 106 or by a separate calculation device. Alternatively, the post-processing may be performed by a calculation device (not shown) present on a cloud via a network.

[0017] The display unit 107 displays the estimated aberration. The estimated aberration may be stored in the computer 106 without being displayed on the display unit 107, or the stored aberration may be used for analytical processing such as performance analysis. The display unit 107 is, for example, a liquid crystal display or a projector.

[0018] The aberration acquired as the aberration of the device under test 102 includes not only the aberration of the imaging optical system 103 but also the influence of all optical elements of the image sensor 104, such as the cover glass. The aberration acquired as the aberration of the device under test 102 also includes aberration caused by assembly errors and distortions that occur during the assembly process of integrating the imaging optical system 103 and the image sensor 104. In this case, the cover glass of the image sensor 104 may also be considered to be part of the imaging optical system 103, and errors during assembly can also be considered to be errors occurring in the imaging optical system. Therefore, hereinafter, the aberration of the imaging optical system (optical system) refers to either the aberration of the imaging optical system 103 alone or the aberration of a device in which the imaging optical system 103 and the image sensor 104 are integrated.

[0019] The principle on which this embodiment functions and an aberration estimation method based on this principle will be described below. As an example, the aberration (wavefront aberration) W(ξ,η) generated by the imaging optical system will be used as the aberration to be calculated. Note that (ξ,η) are Cartesian coordinates in pupil space normalized by the pupil radius.

[0020] In this embodiment, it is also possible to calculate the aberration of the entire device that occurs when the imaging optical system and the image sensor are integrated. Once the wavefront aberration W(ξ,η) is obtained, the amount of transverse aberration and longitudinal aberration can be calculated by simple calculation. By expanding the aberration W(ξ,η) using Zernike polynomials, it can be converted into Seidel aberration. It can also be converted into chromatic aberration by measuring at different wavelengths.

[0021] In this embodiment, the aberration W(ξ,η) is estimated by the computer 106 from the light intensity distribution of the optical image of the subject 101 measured. Here, a method using the transport of intensity equation is exemplified as a method for estimating the aberration W from the light intensity distribution. The transport of intensity equation is expressed by the following equation 1.

[0022]

number

[0023] Here, x and y are two-dimensional Cartesian coordinates on a plane perpendicular to the optical axis, z is the coordinate in the optical axis direction, and I(x, y; z) is the light intensity distribution in a plane perpendicular to the optical axis at position z in the optical axis direction. Also, Φ(x, y; z) is the light phase distribution in a plane perpendicular to the optical axis at position z in the optical axis direction. ∇ ⊥ is the differential operator in the x and y directions, and λ is the wavelength.

[0024]

number

[0025] is the position in the optical axis direction at which the phase distribution Φ is calculated.

[0026] If we consider the case where the light intensity distribution I is uniform, the left-hand side becomes the second-order derivative of the light phase distribution Φ. In this case, equation 1 shows that the change in intensity due to light propagation (right-hand side) is proportional to the second-order change in the light phase distribution (left-hand side). By utilizing this property in reverse, we can reconstruct the light phase distribution by measuring the change in intensity due to light propagation.

[0027] In this embodiment, the aberration of the imaging optical system is estimated based on the transport of intensity equation using light intensity distributions measured at two defocus positions, one positively and one negatively spaced from the focal position. Figure 3 shows an aberration estimation system for measuring light intensity distributions at two defocus positions, one positively and one negatively spaced from the focal position. The dashed-dotted line in Figure 3 indicates the focal position of the device under test 102.

[0028] FIG. 3(a) shows an example of a defocus position that is positively spaced from the focus position, and FIG. 3(b) shows an example of a defocus position that is negatively spaced from the focus position.

[0029] If the defocus is sufficiently large and symmetrical in both positive and negative directions, the measured light intensity distribution will be equivalent to the light intensity distribution measured in pupil space at positions symmetrical about the pupil plane and roughly equidistant from it. Therefore, the solution φ0 to the transport of intensity equation will be the phase distribution on the pupil plane, i.e., the aberration W(ξ,η).

[0030] In this case, the coordinates ξ and η can be converted from the coordinates x and y, the defocus amount z0, and the numerical aperture and imaging magnification of the imaging optical system 103. The differential equation for the position z on the right side of equation 1 can be approximated by the difference value between the light intensity distribution Im1 acquired with the arrangement of Figure 3(a) and the light intensity distribution Im2 acquired with the arrangement of Figure 3(b). Furthermore, the light intensity distribution I on the left side of equation 1 can be approximated by the average value of Im1 and Im2.

[0031] There are various methods for solving the differential equations, for example, by expanding the phase distribution and light intensity distribution in an orthogonal function system. In this case, the computational load can be reduced by selecting an easily handled function such as a Fourier basis or Zernike polynomials as the orthogonal function system.

[0032] As described above, in order to estimate the aberration of an imaging optical system from the light intensity distribution using the transport of intensity equation, it is necessary to acquire light intensity distributions at multiple defocus positions. Note that, due to the characteristics of the transport of intensity equation, it is preferable to use light intensity distributions at two or more defocus positions, at least one at a positive defocus position and one at a negative defocus position.

[0033] In this embodiment, the light intensity distribution is acquired by the image sensor 104. As described above, when estimating the aberration of a device in which an imaging optical system and an image sensor are integrated, the image sensor is optimized for its original purpose, such as photography, and may not be suitable for acquiring the light intensity distribution required for aberration estimation. Therefore, in this embodiment, aberration is estimated from a composite image obtained by combining multiple images. With this configuration, it is possible to accurately estimate the aberration of the optical system without relying on the image sensor in the aberration estimation system.

[0034] Furthermore, in this embodiment, in order to speed up the image acquisition time, the light intensity distribution is acquired from an image with reduced resolution. By combining multiple images, a high-resolution image can be obtained from a low-resolution image. As a result, the processing time for aberration estimation can be reduced while minimizing the impact on accuracy.

[0035] Note that any known method can be used to reduce the image resolution. Below, we will explain the method using downsampling, which is the fastest method. This method reduces the amount of data by outputting data every few pixels, rather than transferring and saving all of the data with the same number of pixels as the sensor.

[0036] Although the present embodiment illustrates a method of solving the transport of intensity equation as an example of a calculation method for estimating aberration, the present invention is not limited to this. Machine learning or the like may also be used as a calculation method for estimating aberration from the light intensity distribution.

[0037] The principle of restoring a high-resolution image from multiple low-resolution defocused images will be described. In the optical system shown in Figure 1, the image i of an object formed on the sensor surface can be expressed by Equation 2, where o is the light intensity distribution of the object and PSF is the point spread function of the imaging optical system 103. Note that * represents a convolution operation. x and y are orthogonal coordinates with respect to the optical axis on the sensor surface. In Equation 2, o is defined in object space, but by reducing the coordinate by the reduction magnification of the imaging optical system, it is expressed as a convolution operation in image space. Unless otherwise specified, imaging is assumed to be at 1x magnification, but this is not limited to this. i(x,y)=o(x,y)*PSF(x,y)...Equation 2

[0038] In Equation 2, the effect of defocus is expressed as a change in the PSF. For simplicity's sake, if o is a point light source, Equation 2 becomes Equation 3, and the image i of the subject becomes the same as the PSF. i(x,y)=PSF(x,y) Equation 3

[0039] Next, the relationship between defocus and point image formation will be explained using Figures 4 and 5. Figure 4 shows a point image formation when there is no aberration. Figure 5 shows a point image formation when there is aberration of 0.3λ in the 7th term of Zernike, 0.3λ in the 15th term, and 0.2λ in the 16th term.

[0040] Figure 4(a) shows a point image at the focal position (defocus amount on the image plane is approximately 0 μm). Figure 4(b) shows a point image with a defocus amount of 150 μm on the image plane. Figure 4(c) shows a point image with a defocus amount of 1300 μm on the image plane. Finally, Figure 4(d) shows a point image with a defocus amount of 1330 μm on the image plane. In Figure 4, the F-number of the optical system is 3, and the wavelength of the light source is 0.5 μm.

[0041] Also, Figure 5(a) shows a point image at the focal position (defocus amount on the image plane is approximately 0 μm). Figure 5(b) shows a point image with a defocus amount on the image plane of 150 μm. Figure 5(c) shows a point image with a defocus amount on the image plane of 1300 μm. And Figure 5(d) shows a point image with a defocus amount on the image plane of 1330 μm. In Figure 5, the F-number of the optical system is 3, and the wavelength of the light source is 0.5 μm.

[0042] As shown in Figures 4(a) and 5(a), the point image becomes roughly a point near the focal position, but when defocus is applied, it spreads in accordance with the amount of defocus, as shown in Figures 4(b) and 5(b). In this case, if there is no aberration in the imaging optical system as shown in Figure 4(b), the effect of defocus appears as a spread of the point image, but if there is aberration, a point image with varying shades is obtained as shown in Figure 5(b).

[0043] Next, Figures 4(c) and 5(c) show point images obtained when a certain degree of defocus is applied compared to Figures 4(b) and 5(b). The shading of the light intensity distribution caused by aberrations is more clearly visible in Figures 4(c) and 5(c). The light intensity distribution of the point image that appears in this way roughly coincides with the second derivative of the wavefront aberration W. Therefore, the effect of defocus can be estimated using the transport of intensity equation based on the light intensity distribution of the point image spread.

[0044] 4(d) and 5(d) show point images obtained by slightly changing the defocus position of Fig. 4(c) and Fig. 5(c). In the point images expanded to the point images shown in Fig. 4(c) and Fig. 5(c), the light intensity distribution does not change, but the shading of the light intensity distribution is enlarged or reduced.

[0045] In reality, the light intensity distribution for a point image is calculated by convolving o with the PSF, as shown in Equation 2, so the captured image does not coincide with the point image. However, if the size of o in image space, i.e., the size of the image of subject 101 multiplied by the reduction magnification, is smaller than the spread (amount of blur) of a defocused point image, the captured image will be approximately equivalent to the point image. In this case, the size of o in image space needs to be less than half the spread of the PSF, and preferably less than 1 / 10.

[0046] By utilizing the above-described characteristics, a high-resolution defocused point image can be restored from a low-resolution defocused point image.

[0047] Next, an example of a case where a low-resolution point image is acquired at an arbitrary defocus position will be described with reference to Fig. 6. The figure shows an example of a spread point image and an example of sampling positions, and the positions indicated by circles in the figure indicate the sampling positions. Note that the number of sampling positions is shown as an example and is not limited to that shown in Fig. 6.

[0048] Figure 6(b) shows a point image when the defocus amount is increased compared to Figure 6(a). As mentioned earlier, when the defocus is slightly changed from the defocus amount z0, which is enough to spread the point image, the light intensity distribution remains the same, and only the spread of the point image changes. Therefore, the images obtained in Figures 6(a) and 6(b) are equivalent to those obtained when the same point image is acquired at different sampling positions. The expansion / reduction ratio of the point image when the defocus is changed by Δz is expressed by Equation 4. M=(z0+Δz) / z0...Equation 4

[0049] In other words, by changing the defocus amount by Δz, it is possible to sample a point image similar to that obtained when the sampling position is multiplied by M in the radial direction. An example of the expanded point image and the sampling position in this case is shown in Figure 6(c). Therefore, acquiring multiple images while changing the defocus amount multiple times is equivalent to acquiring multiple images sampled at different positions. Therefore, by combining these images, it is possible to increase the resolution of the obtained point image.

[0050] An example of a configuration for obtaining a plurality of images while changing the defocus amount a plurality of times as described above is shown in Fig. 7. Note that the dashed dotted line in Fig. 7 indicates the focal position of the device under test 102.

[0051] FIG. 7(a) shows an example in which a point image at a defocus position ±z0, which is an arbitrary defocus amount in the positive and negative directions, is used as a reference, and an image is acquired at that position and at a position where the defocus is changed by a small change amount Δz from the defocus position ±z0. In this case, it is preferable that images are acquired multiple times in both the positive and negative directions at positions where the defocus is changed by the small change amount Δz. Note that it is not necessary to use defocus positions (±z0) that are symmetrical in the positive and negative directions as the reference, and the defocus positions do not have to be symmetrical. It is sufficient to acquire images at multiple positions that are shifted by small changes from a position where a certain amount of defocus is given, at least in either the positive or negative defocus direction.

[0052] Note that, although FIG. 7(a) shows an example in which light intensity distributions are acquired at the same number of positive and negative defocus positions, this is not limiting. For example, as shown in FIG. 7(b), aberrations can be estimated in the same way even when two point images are acquired at a position with a positive defocus amount relative to the focal position and four point images are acquired at a negative defocus position. In general, it is sufficient to acquire n and m images near the positive and negative reference defocus positions, respectively. Here, n and m are integers equal to or greater than 0, and n + m is at least 2.

[0053] A preferred range of the change amount Δz in this embodiment will be described below. Note that, in the following, the distance between a first position and a second position, which is a defocus position adjacent to the first position, among a plurality of different defocus positions is defined as the change amount Δz [μm].

[0054] Here, if the pixel pitch is p and the downsampling rate is α, the image can be expressed as an optical image sampled at intervals of αp. The downsampling rate indicates how many pixels are selected at intervals, with α=5 meaning that the image was acquired using pixels every fifth pixel. Furthermore, if the resolution is reduced by a method other than downsampling, the downsampling rate may be the ratio between the original resolution and the resolution of the image after the resolution is reduced.

[0055] Wavefront aberration W tends to change sharply near the pupil edge, and as mentioned above, the intensity distribution of a defocused point image roughly corresponds to the second derivative of wavefront aberration W. As a result, a defocused point image often has large intensity changes near its contour. For this reason, being able to accurately measure the area near the contour of a defocused point image is an important requirement for highly accurate aberration estimation. If the amount of change in diameter D of the point image when defocus is changed by Δz is ΔD, the relationship between the two can be expressed by Equation 5, using the F-number of the imaging optical system. Δz / ΔD=F Equation 5

[0056] The change in the sampling position due to the expansion or contraction of the point image is half of ΔD, so the movement amount Δp of the sampling position is given by Equation 6. Δp=ΔD / 2=Δz / (2F) Equation 6

[0057] On the other hand, point images are sampled at intervals of αp. Therefore, it is desirable that the amount of movement Δp of the sampling position is greater than about 1 / 8 of αp. This can be written as Equation 8. Δp≧αp / 8...Equation 7

[0058] Next, by substituting Equation 6 into Equation 7, Equation 8 is obtained. Δz≧αpF / 4...Equation 8

[0059] From the above and formula 8, the amount of change Δz increases as the downsampling rate α increases, i.e., the number of pixels in the acquired image decreases compared to the number of pixels in the image sensor. In other words, it is preferable that adjacent defocus positions are spaced apart enough to satisfy formula 8.

[0060] It is also preferable to set the amount of change Δz approximately proportional to the downsampling rate α. It is also preferable that the amount of change Δz increases as the F-number of the imaging optical system 103 increases, and that the amount of change Δz is set approximately proportional to the F-number.

[0061] Furthermore, the larger the pixel size p of the image sensor 104, the larger the amount of change Δz, and it is preferable that the amount of change Δz be determined to be approximately proportional to the pixel size p.

[0062] Here, the size of an imaging element used in a camera or the like is about 3 to 6 μm, and the F-number of an optical system used for imaging is about 1 to 11. Furthermore, when the downsampling rate α is set to 2, it is preferable that the amount of change Δz [μm] satisfies the following conditional expression (1a): 1.50≦Δz≦550 (1a)

[0063] Furthermore, when the downsampling rate α is set to 7 in order to further enhance the effect of the speed increase, it is preferable that the amount of change Δz [μm] satisfies the following conditional expression (1b). 5.25≦Δz≦440 (1b)

[0064] If the upper limit of each conditional expression is exceeded, the aberration estimation system becomes large.

[0065] Although the above example illustrates downsampling and other processes for speeding up the processing of the aberration estimation method, downsampling and other processes are not essential components of this embodiment. Furthermore, when using a sensor with a large pixel pitch p of the image sensor, a configuration in which the distance αp between image pixels is large and the change amount Δz is appropriately set based on Equation 8 can similarly achieve a speed-up effect. Furthermore, the change amount Δz does not need to be constant. In other words, when there are first, second, and third defocus positions in order from the focal position, the change amount between the first position and the second position does not need to be the same as the change amount between the second position and the third position. [Example]

[0066] The aberration estimation method in this embodiment will be described with reference to Fig. 1, Fig. 8 and Fig. 9. The aberration estimation method in this embodiment is executed by the aberration estimation system 1 in Fig. 1. Fig. 8 is a flowchart showing the aberration estimation method.

[0067] Here, we will supplement the configuration of the aberration estimation system 1 in this embodiment. This includes parameters used to derive an example of an analysis result by a simulation, which will be described later, and does not limit the configuration of this embodiment.

[0068] The subject 101 is a pinhole that emits light with a wavelength of 0.5 μm from its aperture. The device under test 102 uses a CMOS as the image sensor 104. The imaging optical system 103 has an F-number of 3, and when expanded using the Fringe-Zernike polynomial, it has aberrations with coefficients of 30 nm in the fifth term, -20 nm in the eighth term, and 150 nm in the ninth term. This was determined by measuring wavefront aberration.

[0069] The driving device 105 moves to a position z in the optical axis direction instructed by the computer 106, and acquires the light intensity distribution using the image sensor 104. The pixel size of the image sensor 104 is 5 μm. The imaging optical system is set to a unity magnification.

[0070] First, in step S1, a downsampling rate α is determined. α may be a value determined in advance, or may be selected by the user. To allow the user to input the downsampling rate α, a user interface may be provided that allows the user to specify the downsampling rate α. At this time, the interface is not limited to one that allows the user to directly input α, and options such as speed priority or image quality priority may be displayed and selected by the user.

[0071] However, the present invention is not limited to this, and a configuration may be adopted in which the user can select the defocus position. In this case, the computer 106 determines the downsampling rate α and other parameters that satisfy the above-mentioned formula in accordance with the defocus position. The downsampling rate α and the defocus position may also be determined in advance in accordance with the imaging optical system 103, the image sensor 104, and the like.

[0072] In this embodiment, it is assumed that a high-speed imaging method has been selected by the user, and the downsampling rate α is set to 7. In other words, images are output at intervals 7 times the pixel pitch of the image sensor 104.

[0073] Next, in step S2, the computer 106 determines the drive position of the drive device 105. In other words, the drive position of the drive device 105 is the reference defocus position z. The computer 106 determines the defocus interval Δz based on the downsampling rate α determined in S1 in addition to the F-number and pixel size p input in advance.

[0074] In this embodiment, as an example, the reference defocus amount z0 is set to 1300 μm, and Δz is set to Δz=αpF=105 μm, which is four times the value when the equality sign in Equation 8 is established.

[0075] Next, in step S3, the light intensity distribution I(x, y, z) at a plurality of defocus positions is calculated. j) is measured. j is the number of the captured image, and in the following explanation, it is assigned in order from the negative side of the defocus position z. At this time, the computer 106 in the aberration estimation system 1 controls the driving of the motorized stage 105 and the imaging by the image sensor 104. Furthermore, under the control of the computer 106, the driving by the driving device 105 and the measurement of the light intensity distribution by the image sensor 104 are repeated until the acquisition of images showing the light intensity distribution I at a plurality of defocus positions is completed. The acquired light intensity distribution I(x, y, z j ) is stored in a data storage device or in temporary memory of the computer 106.

[0076] Subsequently, in step S4, the computer 106 reads the image showing the light intensity distribution I(x, y, zj) stored in the memory. j ) is stored in the temporary memory of computer 106, this step may be skipped.

[0077] In step S5, the computer 106 calculates the light intensity distribution I(x, y, z j ), a synthesized image showing a synthesis intensity distribution Is(x, y, ±z-0) is generated. In this embodiment, multiple images are acquired near the positive and negative reference defocus positions ±z0, and these are synthesized to generate synthesized images near each of the positive and negative reference positions. In this embodiment, generating synthesized images near each of the positive and negative reference positions can further improve the accuracy of aberration estimation. However, the present invention is not limited to this, and the effect of improving the accuracy of aberration estimation can be achieved by using at least one synthesized image.

[0078] In step S6, the computer 106 solves the transport of intensity equation based on the composite image to estimate the aberration of the imaging optical system 103. If necessary, the estimated aberration may be sent to the display unit 107 and displayed.

[0079] An example of the synthesis process executed by the computer 106 in step S5 in this embodiment will now be described with reference to Fig. 9. Fig. 9 is a flowchart showing the synthesis process method.

[0080] First, in step S51, the computer 106 performs image interpolation processing on an image showing a light intensity distribution I(x, y, zj) to obtain an interpolated intensity distribution Ii(x, y, z j In this embodiment, bicubic interpolation is performed as the interpolation process, and the number of pixels in the image is increased by four times. With this configuration, even if the resolution of the image showing the light intensity distribution I(x, y, zj) is low due to the performance of the imaging element, or even if the image is acquired with a small number of pixels to reduce the processing load, it is possible to acquire an image with a high number of pixels.

[0081] Next, in step S52, the computer 106 performs an enlargement or reduction process on one or more of the interpolated images to generate a corrected intensity distribution Ic(x, y, z j ) to generate a corrected image.

[0082] The relationship between the interpolated image and the corrected image is expressed by Equation 9. In this case, the spread of the point image changes in each of the images acquired by changing the defocus by a small amount from the reference defocus position ±z0, and the amount of change can be expressed as the magnification M in Equation 4. Ic(x,y,zj)=Ii(Mx,My,zj)...Equation 9

[0083] In this embodiment, based on Equation 4, the image enlargement magnification (enlargement ratio) is 1.08, and the image reduction magnification (reduction ratio) is 0.91.

[0084] Next, in step S53, the computer 106 generates a composite image based on the multiple corrected images. In this embodiment, the composite image is generated by averaging the multiple corrected images. Note that the method for generating the composite image is not limited to this. If necessary, the multiple corrected images may be averaged after being assigned different weights. In this embodiment, two corrected images are generated: one image obtained by combining multiple corrected images obtained at a positive defocus position, and the other image obtained by combining multiple corrected images obtained at a negative defocus position.

[0085] 10 to 12 show examples of images obtained by the estimation method of this embodiment. FIG. 10 shows images showing point images obtained at different defocus positions. FIG. 11 shows an interpolated image obtained by the interpolation process in step S52. FIG. 12 shows a composite image in this embodiment. Note that FIGS. 10 and 11 show images at positions separated by the positive and negative defocus amount z0, and images at positions where the defocus is changed by a small amount based on the ±z0 positions.

[0086] As shown in Figure 10, the sampling position of the point image changes due to changes in defocus, as described above, and therefore the variation in brightness seen around the contour of the point image changes for each image.

[0087] The multiple images in Fig. 11 correspond to the multiple images in Fig. 10, respectively. In this embodiment, interpolation processing is performed after trimming only the area containing point images from the entire image, so the overall size of the image is the same in Fig. 10 and Fig. 11, which show the relationship before and after interpolation processing. By the processing in this embodiment, it is possible to obtain images in which the size of the point images is approximately the same through enlargement / reduction processing, while maintaining the roughness of the image quality as in the images showing point images in Figs. 10 and 11. The composite image shown in Fig. 12 has improved image quality compared to the image in Fig. 11.

[0088] By performing the above process in step S5, in a method for estimating the aberration of an optical system based on a plurality of images with different defocus amounts, it is possible to accurately estimate the aberration from a low-resolution point image. As a result, the load of the aberration estimation process can be reduced, and the process can be sped up.

[0089] 9 is an example and is not limited to this. For example, non-equidistant composite data allocated from each defocus point image may be created. Alternatively, instead of performing the averaging process in S53, the derivative of z and the light intensity distribution I appearing in the transport of intensity equation may be approximated from multiple images using linear or polynomial fitting. In either method, the aberration of the optical system can be estimated with high accuracy even if multiple images with different defocus amounts acquired near the reference position are low-resolution images.

[0090] Here, with reference to FIG. 13 showing the estimation results of Example 1, the aberration W estimated by the estimation method of this example, its true value, and the aberration estimated by the conventional estimation method will be compared. The aberration W is shown as coefficients expanded in a Fringe Zernike polynomial. In FIG. 13, the horizontal axis represents the Zernike term, and the vertical axis represents the coefficient. The dashed line representing the estimation results of this example is closer to the solid line of the true value than the dotted line of the conventional example. In other words, this example enables highly accurate estimation. [Example]

[0091] Next, the aberration estimation method in this embodiment will be described with reference to Fig. 14. In the aberration estimation method in this embodiment, the steps shown in Fig. 8 are executed by the aberration estimation system 1 in Fig. 1. Note that in this embodiment 2, the parameters used to derive the example of the analysis result by simulation are different from those in the first embodiment.

[0092] The simulation results are shown in Figure 14. The vertical axis of Figure 14 shows the value obtained by dividing Δz by the F-number, and the vertical axis shows the residual RMS of the aberration and the true value. The residual RMS is defined by the following equation 10. In this case, the estimated aberration is West, the true aberration is W0, and the number of data points within the pupil aperture area is

[0093]

number

[0094] and summation is performed within the pupil aperture region.

[0095]

number

[0096] The smaller the residual RMS value, the more accurate the estimation result. According to Equation 8, Δz is proportional to the F value, so in order to compare results with different F values, the horizontal axis is the value obtained by dividing Δz by F.

[0097] The parameters used to calculate the data marked with circles in Fig. 14 are such that the F-number of the imaging optical system 103 is 2, and when expanded using a Fringe Zernike polynomial, the aberration coefficients are −300 nm in the eighth term and 300 nm in the ninth term. The reference defocus amount z0 is 900 μm, and the downsampling rate α is 7. The parameters used to calculate the data marked with crosses in Fig. 14 differ from the parameters used to calculate the data marked with circles in Fig. 14 in that the downsampling rate α is 7.

[0098] 14, the parameters used to calculate the data indicated by * are that the F-number of the imaging optical system 103 is 3, and when expanded using a Fringe Zernike polynomial, the fifth term has aberrations of 30 nm, the eighth term has a coefficient of −20 nm, and the ninth term has aberrations of 150 nm. The reference defocus amount z0 is 1300 μm, and the down-sampling rate α is 7. The above conditions are the same as those in Example 1.

[0099] 14, when Δz / F is 0, that is, when the present invention is not implemented, the estimation residual is large. On the other hand, as Δz / F increases, the estimation residual becomes smaller. Δz / F at which the estimation residual is minimum varies depending on the downsampling rate α, and the larger the downsampling rate α, the larger the tendency for the estimation residual to be minimum when Δz / F is large.

[0100] Furthermore, the value of Δz / F where the estimated residual is about half the minimum value of the conventional example (Δz / F=0) is approximately 10 μm when the downsampling rate is α=7, and approximately 15 μm when the downsampling rate is α=10. This roughly matches the equality condition in Equation 9.

[0101] Here, if the change amount Δz is further increased, the estimation error increases. This is thought to be because the point image spreads further as Δz increases, and spreads to the same position as the next sampling point. However, because the sampling position changes as if expanding or contracting in the radial direction, the amount of movement of the sampling position in response to a change in the change amount Δz differs near the outline and near the center of the point image. Because the intensity distribution of the point image depends on the aberration of the imaging optical system, the tendency of the estimation residual for changes in Δz also changes.

[0102] Note that the data indicated by ◯ in Fig. 14 show large estimation errors when Δz / F is approximately 50 μm, while the data indicated by * in Fig. 14 show large estimation errors when Δz / F is approximately 70 μm. To achieve highly accurate estimation for various aberrations, it is preferable that Δz / F be at least 70 μm or less, and it is more preferable that Δz / F be 50 μm or less.

[0103] Modifications of each embodiment will be described. This embodiment can be effective even when image quality is reduced by compression processing or the like. In this case, data acquisition can be accelerated by substituting video capture for image acquisition. This can be achieved by continuously moving the drive device 105 to capture video in step S3. Furthermore, an image at a required defocus position is extracted from the video obtained in step S4, and a synthesis process is performed in step S5.

[0104] With this configuration, even if the image quality of an image extracted from a moving image has been degraded by compression processing, the aberration can be estimated with high accuracy by processing to combine multiple images with different defocuses.

[0105] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0106] Although the embodiments and examples of the present invention have been described above, the present invention is not limited to these embodiments and examples, and various combinations, modifications, and changes are possible within the scope of the gist of the present invention.

[0107] The embodiments of the present invention include the following methods and configurations.

[0108] (Method 1) An estimation method for estimating an aberration of an optical system using a plurality of images acquired by receiving optical images of a subject formed through the optical system at a plurality of mutually different defocus positions by an image sensor, comprising: acquiring the plurality of images; generating a composite image by combining the plurality of images; and estimating the aberration of the optical system based on the composite image.

[0109] (Method 2) The estimation method according to Method 1, wherein the step of generating a composite image comprises averaging the plurality of images to generate the composite image.

[0110] (Method 3) 3. The estimation method according to method 1 or 2, further comprising the step of generating a plurality of interpolated images having a larger number of pixels than the plurality of images by interpolating the plurality of images.

[0111] (Method 4) 4. The method of any one of methods 1 to 3, further comprising the step of enlarging one or more of the images.

[0112] (Method 5) 5. The estimation method according to method 4, wherein in the step of enlarging the image, the enlargement ratio for the image is determined based on the defocus position.

[0113] (Method 6) 4. The estimation method according to any one of methods 1 to 3, further comprising the step of downscaling one or more of the images.

[0114] (Method 7) The estimation method according to Method 6, wherein in the step of reducing the image, a reduction ratio for the image is determined based on the defocus position.

[0115] (Method 8) the plurality of mutually different defocus positions include a first position and a second position that is a defocus position adjacent to the first position, When the amount of change between the first position and the second position is Δz [μm], 1.50≦Δz≦550 8. The estimation method according to any one of Methods 1 to 7, wherein the following condition is satisfied:

[0116] (Method 9) The estimation method according to any one of Methods 1 to 8, wherein the plurality of defocus positions are determined based on at least one of a downsampling rate or a pixel pitch of the imaging element.

[0117] (Method 10) 10. The aberration estimation method according to Method 9, further comprising the step of acquiring at least one of the downsampling rate or the pixel pitch of the imaging element.

[0118] (Configuration 1) An estimation device that estimates aberration of an optical system using a plurality of images acquired by receiving optical images of a subject formed through the optical system at a plurality of different defocus positions on an image sensor, a driving means for changing the plurality of defocus positions; a calculation unit that estimates the aberration of the optical system, The estimation device is characterized in that the calculation unit generates a composite image by combining at least two images from among the plurality of images, and estimates the aberration of the optical system based on the composite image.

[0119] (Program 1) A program causing a computer to execute the estimation method according to any one of Methods 1 to 10.

[0120] (Configuration 2) A storage medium storing the program described in Program 1. [Explanation of symbols]

[0121] 101 Subject 103 Imaging Optical System 104 Image sensor S3 Steps to acquire multiple images S5: Generate a composite image S6: Estimating the aberration of the optical system

Claims

1. An estimation method for estimating an aberration of an optical system using a plurality of images acquired by receiving optical images of a subject formed through the optical system at a plurality of mutually different defocus positions by an image sensor, comprising: acquiring the plurality of images; generating a composite image by combining the plurality of images; and estimating the aberration of the optical system based on the composite image.

2. 2. The method according to claim 1, wherein the step of generating a composite image comprises averaging the plurality of images to generate the composite image.

3. 2. The estimation method according to claim 1, further comprising the step of generating a plurality of interpolated images having a larger number of pixels than the plurality of images by interpolating the plurality of images.

4. The method of claim 1 further comprising the step of enlarging one or more of the plurality of images.

5. 5. The estimation method according to claim 4, wherein in the step of enlarging the image, an enlargement ratio for the image is determined based on the defocus position.

6. The method of claim 1, further comprising the step of reducing one or more of the plurality of images.

7. 7. The estimation method according to claim 6, wherein in the step of reducing the image, a reduction ratio for the image is determined based on the defocus position.

8. the plurality of mutually different defocus positions include a first position and a second position that is a defocus position adjacent to the first position, When the amount of change between the first position and the second position is Δz [μm], 1.50≦Δz≦550 8. The estimation method according to claim 1, wherein the following condition is satisfied:

9. The estimation method according to claim 1 , wherein the plurality of defocus positions are determined based on at least one of a downsampling rate and a pixel pitch of the image sensor.

10. 10. The aberration estimation method according to claim 9, further comprising the step of acquiring at least one of the downsampling rate and the pixel pitch of the image sensor.

11. An estimation device that estimates aberration of an optical system using a plurality of images acquired by receiving optical images of a subject formed through the optical system at a plurality of different defocus positions on an image sensor, a driving means for changing the plurality of defocus positions; a calculation unit that estimates the aberration of the optical system, The estimation device is characterized in that the calculation unit generates a composite image by combining at least two images from among the plurality of images, and estimates the aberration of the optical system based on the composite image.

12. A program causing a computer to execute the estimation method according to any one of claims 1 to 7.

13. A storage medium storing the program according to claim 12.

Images