System and method for digital optical aberration correction and spectral imaging

Abstract

A system and method for digital optical aberration correction and spectral imaging is provided. An optical system can include an optical imaging unit to form an optical image near an image plane of the optical system, a wavefront imaging sensor unit located near the image plane to provide raw digital data of a light field and image output near the image plane, and a control unit to process the raw digital data and the image output to provide a deblurred image output, wherein the control unit includes a storage unit storing instructions and a processing unit executing the instructions to receive the image input and the raw digital data of the light field incident to the wavefront imaging sensor and generate the deblurred image based on an analysis of an optical mutual coherence function at the image plane.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 023,287, filed May 12, 2020, and Israel Patent Application No. 276922, filed August 25, 2020, both of which are incorporated herein by reference. Technical Field

[0003] This invention belongs to the fields of optical imaging, digital deblurring, digital aberration correction, digital adaptive optics, and spectral imaging. Background Technology

[0004] Optical aberrations are a measure of non-ideal imaging, leading to a decrease in image quality and sharpness. In geometrical optics, ideal imaging is typically described as the convergence of rays from every point in object space through the imaging system at a single corresponding point on the image plane. In practice, the wave nature of light must be considered, which precludes focusing light to an infinitesimally small point. Conversely, in physical wave optics, ideally converging rays produce a finite-sized spot due to diffraction. A typical representative diameter of such a spot is given by the Airy disc 1.22·λ / NA, where λ is the typical wavelength and NA is the numerical aperture of the converging ray cone. Ideal imaging is often referred to as the "diffraction limit," and in ray optics, a sufficient criterion is that rays converge within the Airy disc.

[0005] Optical aberrations can be caused by a variety of sources. Some are inherent design compromises of optical imaging systems due to conflicting demands between imaging requirements and system complexity. On the one hand, operational requirements such as field of view, working distance, magnification, aperture size, and spectral range add additional constraints to optical design. These typically manifest as increased system complexity, such as an increased number and type of optical elements, materials used, surface profiles (spherical / aspherical), and alignment and manufacturing tolerances. On the other hand, forces driving simplification, such as cost, size, weight, readily available materials and processes, relaxed manufacturing tolerances, and robustness to environmental factors, can lead to compromised image quality. The use of multiple complex aspherical surfaces to reduce aberrations is known in the art.

[0006] Additional aberrations are also a result of manufacturing and alignment tolerances during the construction of optical imaging systems. Manufacturing and alignment tolerances cause the parameters of the actual imaging system to deviate from their nominal design values. Drift / variation of the optical system due to environmental factors (such as thermal changes, pressure changes, shocks and / or vibrations) can also lead to other aberrations.

[0007] Optical aberrations can also occur due to factors outside the imaging system. Through turbulent media such as atmospheric disturbances, imaging can lead to time-varying optical aberrations, thus reducing image quality—for example, the "seeing" effect in ground-based astronomy and surveillance. In medical ophthalmological imaging, aberrations present in the subject's eye reduce imaging resolution, and in many cases, suppress the diffraction-limited resolution in imaging the microscopic cellular structures of the retina. Imaging of large biological tissues is also limited by aberrations caused by translucent tissues.

[0008] In the prior art, adaptive optics systems are used to compensate for aberrations. Adaptive optics systems are particularly used to compensate for aberrations that depend on the imaging medium external to the imaging system. Adaptive optics systems are further used to compensate for time-dependent variations in optical system parameters. These adaptive optics systems consist of optical elements capable of dynamically modulating the wavefront, such as deformable mirrors or spatial light modulators; feedback wavefront sensors, such as Shack-Hartman sensors; and a control unit that uses data from the wavefront sensor to control the wavefront modulation elements using closed-loop feedback.

[0009] Adaptive optics systems are difficult to integrate into existing imaging systems. Integrating them requires modifying the optical path to combine the wavefront sensor and wavefront modulation element, typically using additional optical relays and beam splitters. Furthermore, many adaptive optics systems are designed so that the wavefront sensor and wavefront modulation element are located at or near the pupil plane of the imaging system, thus sampling and correcting optical wavefront aberrations uniformly across the entire imaging field. Spatially dependent adaptive optics correction is often performed in scanning optical imaging systems, further complicating the overall system design. In many cases, an additional "guide-star" illumination system is required to provide sufficient feedback to the wavefront sensor, further complicating the system and limiting its practicality and operating range.

[0010] On the one hand, the conflicting demands for simple optical design and high-quality imaging necessitate effective aberration correction schemes. On the other hand, physical correction of the optical wavefront using adaptive optics is highly complex, with limited use cases and high costs. Therefore, the idea of ​​digital aberration correction (or digital adaptive optics), which involves performing digital algorithms to deblur the captured imaging data, is very compelling.

[0011] In image processing, aberrations manifest as a blurring kernel applied to a nominally ideal image, resulting in a blurred image. In many cases, this blurring kernel is spatially dependent, meaning different blurring kernels are applied to different parts of the image. In many situations, the actual blurring kernel is unknown, especially when optical aberrations are caused by factors outside the optical imaging system, such as a cloudy imaging medium, or when optical imaging changes due to environmental factors such as temperature and pressure.

[0012] Deconvolution deblurring of blurred images yields limited results for several reasons. First, in many cases, actual optical aberrations severely degrade the modulation transfer function (MTF), making digital recovery of it introduce unacceptable levels of noise. Furthermore, in many cases, the blur kernel is unknown, leading to a series of blind deconvolution / deblurring algorithms with limited effectiveness.

[0013] Another known technique for digital deblurring and aberration correction is light field imaging, which uses a plenoptic camera, angle-sensitive pixels, and variations thereof to capture light field data. Light field data is a geometric description of the trajectories of light rays propagating from a scene to an imaging device. Traditional imaging records the number of rays that hit each sensor pixel, regardless of the angle of incidence of each ray. Light field imaging, however, aims to record the position (i.e., the pixel it hits) and angle of incidence of each ray, or angle-sensitive pixel, using a microlens array in a plenoptic camera configuration. Conceptually, light field imaging provides data on the position and angle of light rays in the imaging scene as they propagate from the imaging object through the main imaging aperture to the image sensor. Theoretically, this ray angular position data can then be used for virtual ray tracing, propagation, and manipulation of rays to infer the distance between the object and the camera, and to digitally correct aberrations in rays that have not converged to the ideal focus due to aberrations or simply because they are too far from the focal plane.

[0014] As is well known, light field imaging cannot achieve digital deblurring and image quality restoration to diffraction-limited ideal imaging. Due to its inherent angular-spatial resolution trade-off, the image resolution achievable by light field imaging is a multiple (typically 3-10) of the diffraction limit and Nyquist sampling limit given by Δx = λ / 4NA. This is because the entrance pupil of an all-optical system, being only a small portion of the main imaging lens's entrance pupil, produces a much lower optical resolution. The logic behind this configuration is that each such pupil segment represents a different ray angle. Without segmenting the main imaging pupil, the all-optical system cannot recognize the angular information of the rays passing through the system's imaging, rendering the entire virtual ray tracing and propagation scheme ineffective.

[0015] Therefore, the promise that light field imaging can be used to correct optical aberrations remains unfulfilled because standard light field imaging operates at the optical diffraction limit, far from the principal imaging aperture. The inherent image resolution and quality of the light field are far lower than the effective resolution reduction caused by typical aberrations in diffraction-limited optics, so they yield no benefit.

[0016] Other known techniques are based on sparse sampling and underdetermined reconstruction, such as coded aperture and compressed light field techniques. However, these methods rely on heuristic assumptions about the imaging scene, thus creating numerous imaging artifacts for scenes containing elements not covered by the algorithm's training set. Furthermore, the required aperture-coded and compressed light field masks block most of the light, reducing the signal reaching the detector sensor array and lowering the system's overall signal-to-noise ratio (SNR). These types of imaging systems are designed to operate far from the diffraction limit of the main imaging aperture, similar to light field imaging.

[0017] Using hardware-based adaptive optics systems to correct aberrations is highly complex and expensive, may have limited effectiveness, and can impose restrictive constraints on the operational range of the imaging system. Digital aberration correction based on deblurring and deconvolution techniques using blurred original images is often limited by noise and the inability to know the aberration blur kernel in advance. Furthermore, light field-based methods cannot perform diffraction-limited digital aberration correction due to inherent spatial and angular resolution limitations. Sparse sampling techniques are known to introduce unacceptable imaging artifacts, reduce the system's signal-to-noise ratio, and typically operate far from the diffraction limit.

[0018] Traditional color imaging typically uses color filter arrays, such as RGB Bayer filter arrays and their variants. This results in a significant loss of light reaching the sensor. Typically, up to 50-70% of the light is filtered out. Using dichroic prisms can overcome this light loss, but at the cost of increased image sensor and system complexity. Other diffraction-dependent multispectral or grating-based hyperspectral imagers are generally limited to use as one-dimensional line scan cameras with narrow entry slits.

[0019] A system and method are needed to perform digital aberration correction up to the diffraction limit and optionally without light loss, producing a deblurred output image at maximum signal-to-noise ratio, regardless of the input scene being imaged. This needs to be achieved using 2D imaging arrays and with minimal light loss for RGB color, multispectral, or hyperspectral imaging, possibly combined with digital aberration correction. Summary of the Invention

[0020] According to an embodiment of the present invention, an optical system is provided, comprising: an optical imaging unit for forming an optical image near an image plane of the optical system; a wavefront imaging sensor unit located near the image plane for providing raw digital data on the light field and an image output near the image plane; and a control unit for processing the raw digital data and the image output to provide a deblurred image output; wherein the control unit includes a storage unit for storing instructions and execution instructions for receiving an image input and raw digital data of the light field illuminating the wavefront imaging sensor, and generating a deblurred image based on the analysis of the optical coherence function of the image plane.

[0021] The control unit can also: calculate the field characteristics of the light field; identify point sources in the image output based on the coherence and superposition information in the field characteristics; estimate the blur degree of each identified point source; and reconstruct the deblurred image into a composite of deblurred point sources.

[0022] The field property can be a Wigner distribution or an equivalent entity related to the Wigner distribution through mathematical transformation.

[0023] The wavefront imaging sensor unit may further include a color filter array with at least two color bands, and the control unit is used to calculate the field characteristics corresponding to each color band, generate multiple color field attributes, and reconstruct the deblurred image based on the combination of color field characteristics.

[0024] The wavefront imaging sensor unit may include at least two image sensors, each image sensor being associated with a spectral filter, and the control unit is used to calculate field characteristics corresponding to each of the at least two image sensors, thereby generating multiple color field characteristics, and reconstructing a deblurred image based on a combination of color field characteristics.

[0025] According to an embodiment of the present invention, the wavefront imaging sensor unit includes: an optical modulator unit located near the image plane; and an image sensor unit located downstream of the optical modulator unit relative to the general propagation direction of the input light field through the system, for acquiring raw digital image output, wherein the optical modulator unit uses at least one of a phase modulation and an amplitude modulation to modulate the light field.

[0026] The optical modulator unit may include multiple unit cells, and the image sensor unit may include a sensor unit array; the sensor unit array defines multiple sensor subarray units, each sensor subarray corresponding to one of the multiple unit cells of the optical modulator unit; and the optical modulator unit is used to premodulate the input light collected by the image sensor unit, and each unit cell of the optical modulator unit guides a portion of the collected input light incident on it to its corresponding sensor subarray unit and one or more adjacent sensor subarray unit cells in a predetermined proximity region.

[0027] According to an embodiment of the present invention, the original number of pixels N of the plurality of sensor subarray units of the image sensor unit is... R The number N of Nyquist sampling points of the optical modulator unit is equal to or greater than the number of Nyquist sampling points of the optical modulator unit. N .

[0028] According to other embodiments, the original number of pixels N of the plurality of sensor subarray units of the image sensor unit R and the number N of the Nyquist sampling points of the optical modulator unit N Follow the following relationship: in And among them This indicates the spatial variability of dynamic aberrations.

[0029] According to an embodiment of the present invention, the control unit is used to calculate the field characteristics corresponding to each unit lattice, generate multiple color field characteristics, and reconstruct at least one of the following groups based on the combination of the color field characteristics: a full-color RGB image, a hyperspectral image without using spectral filters or color filters.

[0030] According to other embodiments, the control unit is used to calculate the field characteristics corresponding to each unit lattice, generate multiple color field characteristics, and reconstruct a field with N... O Each output image pixel and N C The output image of each color field characteristic, where N O Below the Nyquist sampling limit N N and N C x N O ≤NR That is, the original number of pixels.

[0031] According to one aspect of the invention, when the light field comprises multiple wavelengths, either different or consecutive, the control unit performs one or both of the following operations: (1) estimating the spectral distribution of each identified point source; and (2) reconstructing the spectral distribution map of the image.

[0032] The control unit further performs one or more of the following: (1) estimating the aberration intensity of each identified point source; (2) estimating the depth based on the dynamic aberration intensity estimated for each identified point source, generating a spatial map of photometric aberration intensity, and reconstructing a depth map based on the spatial map of photometric aberration intensity; (3) restoring the diffraction-limited imaging resolution of the defocused portion of the image based on the depth map.

[0033] At least the unit lattice of the optical modulator unit and the sensor unit array of the image sensor unit can be manufactured as one of the following groups: monolithic integrated units; and units such that the unit lattice is part of a process stack for manufacturing the sensor unit array; and individual units.

[0034] The design of the optical modulation unit can be one of the following: a binary design with one disk unit per unit lattice; a multilayer design consisting of concentric rings with equidistant radii; a multilayer design consisting of at least two concentric rings with unequal radii; a binary design with two disks per unit lattice arranged at the edge of the unit lattice; or a multi-ring design with two sets of rings per unit lattice arranged at the edge of the unit lattice.

[0035] The unit lattice of the optical modulation unit will apply phase modulation, amplitude modulation, or both phase and amplitude modulation.

[0036] The optical imaging unit may be one of the following groups: a refractive optical imaging unit, a reflective optical imaging unit, a reflective-refractive optical imaging unit, a diffractive optical imaging unit, and combinations thereof.

[0037] The optical imaging unit may include an objective lens and at least one of the group consisting of a tube lens, a relay optics, and a telescope optics, for forming an image on the image plane. The optical imaging unit can be used as a camera lens and is one of the group consisting of a wide-angle lens, a standard lens, a telephoto lens, and a zoom lens. The optical imaging unit may be one of the group consisting of a refracting telescope system, a reflecting telescope system, or a reflecting-refracting telescope system.

[0038] The wavefront imaging sensor unit can be one of the following groups: 2D region sensor, line scan sensor, multi-line scan sensor and TDI sensor.

[0039] According to one aspect of the present invention, a method for digital optical aberration correction of an image formed by an imaging unit near an image plane of an optical system is provided, comprising: providing raw digital data of a light field illuminating a wavefront imaging sensor unit located near the image plane and an image output formed near the image plane; and processing the raw digital data and the image output by a control unit to provide a deblurred image output based on an analysis of optical coherence functions on the image plane.

[0040] In one embodiment, the wavefront imaging sensor unit includes an optical modulator unit located near the image plane and an image sensor unit located downstream of the optical modulator unit relative to the general propagation direction of the input light field through the system, for acquiring raw digital image output, and the method further includes modulating the light field using at least one of phase modulation and amplitude modulation.

[0041] In another embodiment, the light modulator unit includes a plurality of unit lattices, and the image sensor unit includes a sensor unit array; the sensor unit array defines a plurality of sensor subarray units, each sensor subarray corresponding to one unit lattice of the plurality of unit lattices of the light modulator unit; and the method further includes: premodulating input light collected by the image sensor unit by the light modulator unit; and guiding a portion of the collected input light incident thereon by each unit lattice of the light modulator unit to its corresponding sensor subarray unit and one or more adjacent sensor subarray units in a predetermined neighborhood. The original number of pixels N of the plurality of sensor subarray units of the image sensor unit is... R The number N of Nyquist sampling points of the optical modulator unit is equal to or greater than the number of Nyquist sampling points of the optical modulator unit. N The original number of pixels N of the plurality of sensor subarray units of the image sensor unit. R and the number N of the Nyquist sampling points of the optical modulator unit N Follow the following relationship: in And among them This indicates the spatial variability of dynamic aberrations.

[0042] In one embodiment of the present invention, the method further includes: calculating the field characteristics of the light field; identifying point sources in the image output based on the coherence and superposition information in the field characteristics; estimating the blurring degree of each identified point source; and reconstructing the deblurred image as a synthesis of the deblurred point sources. The field characteristics are a Wigner distribution or an equivalent entity related to the Wigner distribution through mathematical transformation.

[0043] In one embodiment of the present invention, the wavefront imaging sensor unit further includes a color filter array having at least two color bands, and the method further includes calculating color field characteristics corresponding to each color band to generate multiple color field characteristics, and reconstructing a deblurred image based on a combination of color field characteristics.

[0044] In another embodiment, the wavefront imaging sensor unit includes at least two image sensors, each image sensor being associated with a spectral filter, and the method further includes calculating color field characteristics corresponding to each of the at least two image sensors, generating a plurality of color field characteristics, and reconstructing a deblurred image based on a combination of the color field characteristics.

[0045] In yet another embodiment, the original number of pixels N of the plurality of sensor subarray units of the image sensor unit is... R The number N of Nyquist sampling points of the optical modulator unit is equal to or greater than the number of Nyquist sampling points of the optical modulator unit. N The method further includes calculating the color field characteristics corresponding to each unit lattice, generating multiple color field characteristics, and reconstructing at least one of the following groups based on a combination of color field characteristics: a full-color RGB image, a hyperspectral image without using spectral or color filters.

[0046] When the light field comprises multiple wavelengths, either different or consecutive, the method may further include estimating the spectral distribution of each identified point source. The method may also include reconstructing the spectral distribution map of the image.

[0047] According to another aspect of the present invention, a method for processing raw digital data is provided, the raw digital data indicating elements of an optical coherence function of the light field near the image plane of an optical system, the method comprising: calculating field characteristics based on the optical coherence function; identifying point sources in an image output near the image plane based on coherence and superposition information in the field characteristics; estimating the blur of each identified point source to generate a deblurred point source; and reconstructing the deblurred image as a synthesis of the deblurred point sources.

[0048] The method may also include any of the following operations or combinations thereof: estimating the aberration intensity of each point source; estimating the spatial spectral distribution of each point source to generate a set of point-by-point spectral distribution estimates; estimating the spatial map distribution of the reconstructed optical field spectral distribution based on the point-by-point spectra; converting the optical mutual coherence function to a Wigner distribution using a Wigner-Weyl transform, and performing the identification, estimation, and calculation based on the Wigner distribution. The operation of estimating the aberration intensity of each point source may also include classifying aberrations as static or dynamic aberrations.

[0049] According to one aspect of the present invention, a method for image processing is provided, comprising: generating raw digital data indicating elements of an optical coherence function of a light field near an image plane of an optical system; generating a raw digital image comprising: modulating the light field with a light modulator unit having a plurality of unit lattices; acquiring a raw digital image output by an image sensor unit, the image sensor unit being located downstream of the light modulator unit relative to the general direction of propagation of the input light field through the system; the image sensor unit comprising a sensor unit array defining a plurality of sensor subarray units, each sensor subarray corresponding to one unit lattice of the plurality of unit lattices of the light modulator unit, such that each unit lattice of the light modulator unit directs a portion of the input light collected thereon onto the corresponding sensor subarray unit and one or more adjacent sensor subarray units in a predetermined proximity region; and processing the raw data according to the method of an embodiment of the present invention.

[0050] According to another aspect of the present invention, a method for processing raw digital data is provided, the raw digital data indicating elements of an optical coherence function of an optical field near the image plane of an optical system, the method comprising: calculating characteristics of the field based on the optical coherence function; identifying point sources in an output image near the image plane based on coherence and superposition information in the field characteristics; estimating the spatial spectral distribution of each identified point source to generate a set of point-by-point spectral distribution estimates.

[0051] The method may further include any one or a combination of the following operations: reconstructing a spatial map of the optical field spectral distribution based on a point-by-point spectral distribution estimation set; for each identified point source, estimating its blurring, generating deblurred point sources, and reconstructing the deblurred image as a combination of deblurred point sources; estimating the aberration intensity of each point source; converting the optical mutual coherence function to a Wigner distribution using a Wigner-Weyl transform; and performing the identification, estimation, and computation based on the Wigner distribution. The operation of estimating the aberration intensity of each point source may further include classifying the aberrations as static or dynamic aberrations.

[0052] According to one aspect of the present invention, a wavefront imaging sensor system is provided, comprising an optical modulator unit and an image sensor unit. The image sensor unit is located downstream of the optical modulator unit relative to the general propagation direction of the input light field through the system, and is used to acquire a raw digital image output. The optical modulator unit comprises a plurality of unit lattices, and the image sensor unit comprises a sensor unit array, the sensor unit array defining a plurality of sensor subarray units, each sensor subarray corresponding to one unit lattice of the plurality of unit lattices of the optical modulator unit; the image sensor unit has a raw pixel count N of the plurality of sensor subarray units. R The number N of Nyquist sampling points of the optical modulator unit is equal to or greater than the number of Nyquist sampling points of the optical modulator unit. N The optical modulator unit is used to premodulate the input light collected by the image sensor unit, and each unit lattice of the optical modulator unit guides a portion of the input light collected thereon to its corresponding sensor subarray unit and one or more adjacent sensor subarray units in a predetermined neighborhood region; the design of the unit lattice follows at least one of the following conditions: the original number of pixels N of the plurality of sensor subarray units of the image sensor unit. R The number N of Nyquist sampling points of the optical modulator unit is equal to or greater than the number of Nyquist sampling points of the optical modulator unit. N A binary design with one disk unit per unit lattice; a multilayer design consisting of concentric rings with equidistant radii; a multilayer design consisting of at least two concentric rings with unequal radii; a binary design with two disks per unit lattice arranged at the edge of the unit lattice; and a multi-ring design with two sets of rings per unit lattice arranged at the edge of the unit lattice and having a quasi-periodic design.

[0053] According to another aspect of the present invention, a method for designing a wavefront imaging sensor system is provided. The wavefront imaging sensor system has an optical modulator unit and an image sensor unit located downstream of the optical modulator unit relative to the general propagation direction of the input light field through the system, for acquiring raw digital image output. The method includes: providing a plurality of unit lattices at the optical modulator unit and providing a sensor unit array at the image sensor unit, the sensor unit array defining a plurality of sensor subarray units, each sensor subarray corresponding to one unit lattice of the plurality of unit lattices of the optical modulator unit, such that the raw pixel number N of the plurality of sensor subarray units of the image sensor unit is... R The number N of Nyquist sampling points of the optical modulator unit is equal to or greater than the number of Nyquist sampling points of the optical modulator unit. NThe optical modulator unit modulates the input light collected by the image sensor unit in a predetermined manner, and each unit lattice of the optical modulator unit guides a portion of the input light collected thereon to its corresponding sensor subarray unit and one or more adjacent sensor subarray units within a predetermined proximity region; and the unit lattice is designed to meet at least one of the following conditions: the original number of pixels N of the plurality of sensor subarray units of the image sensor unit. R The number N of Nyquist sampling points of the optical modulator unit is equal to or greater than the number of Nyquist sampling points of the optical modulator unit. N The design includes: a binary design with one disk per unit lattice; a multilayer design consisting of concentric rings with equidistant radii; a multilayer design consisting of at least two concentric rings with unequal radii; a binary design with two disks per unit lattice arranged at the edge of the unit lattice; a multi-ring design with two sets of rings per unit lattice arranged at the edge of the unit lattice; and a quasi-periodic design.

[0054] According to one aspect of the present invention, an optical system is provided, comprising: an optical imaging unit for forming an optical image near an image plane of the optical system; an adaptive optics system unit located between the optical imaging unit and the image plane; a wavefront imaging sensor unit located near the image plane for providing raw digital data on a light field and an image output near the image plane; and a control unit for processing the raw digital data and the image output to provide a deblurred image output; wherein the control unit includes a storage unit for storing instructions and a processing unit for executing instructions to receive an image input and raw digital data of a light field illuminating the wavefront imaging sensor and to generate a deblurred image based on an analysis of the optical coherence function of the image plane, wherein the adaptive optics system unit includes an adaptive optics wavefront sensor and an adaptive optics wavefront modulator located near the pupil plane of the imaging system, and an adaptive optics control unit; wherein the adaptive optics wavefront sensor provides feedback under the control of the adaptive optics control unit to drive the wavefront modulator. The adaptive optics system unit may further include a pupil relay and a tube lens. The control unit is used to receive information from the adaptive optics wavefront sensor and provide a coarse measurement of wavefront aberrations. The optical imaging unit, the wavefront sensor, and the control unit (as well as the adaptive optics wavefront sensor, the adaptive optics wavefront modulator, and the adaptive optics control unit) can be implemented according to any aspect and embodiment of the invention. Attached Figure Description

[0055] To better understand the embodiments of the present invention, please refer to the accompanying drawings, wherein the same numbers throughout the text denote corresponding entities, and wherein:

[0056] Figure 1a This is a block diagram illustrating a digital optical aberration correction system according to an embodiment of the present invention;

[0057] Figure 1b This is a block diagram illustrating a digital optical aberration correction system according to another embodiment of the present invention;

[0058] Figures 2a-2c This is a schematic diagram of the optical configuration, illustrating aspects of embodiments of the present invention by way of example;

[0059] Figures 3a-3b This is a flowchart illustrating a method according to an embodiment of the present invention;

[0060] Figure 4 The source reference object is the Siemens star target.

[0061] Figures 5a-5b Example features of a system according to an embodiment of the present invention are shown;

[0062] Figure 6a and 6b The optical characteristics of the system according to an embodiment of the present invention are shown;

[0063] Figure 7 Another optical characteristic of the system according to an embodiment of the present invention is shown;

[0064] Figure 8a and 8b An embodiment of the present invention is shown. Figure 4 Blurred and deblurred images of the source reference object;

[0065] Figures 9a-9b The original pixel arrangement according to an embodiment of the present invention is illustrated schematically;

[0066] Figures 10a-10d An aspect of an optical modulator unit according to an embodiment of the present invention is illustrated schematically;

[0067] Figures 11a-11b Other aspects of the optical modulator unit according to an embodiment of the present invention are illustrated schematically;

[0068] Figure 12a An aspect of a wavefront imaging sensor according to an embodiment of the present invention is illustrated schematically;

[0069] Figure 12b This is a block diagram of a system according to an embodiment of the present invention;

[0070] Figure 13a A spectral band filter according to an embodiment of the present invention is illustrated schematically;

[0071] Figure 13b The spectral distribution according to an embodiment of the present invention is illustrated schematically; and

[0072] Figure 14 This is a block diagram illustrating a digital optical aberration correction system according to an embodiment of the present invention. Detailed Implementation

[0073] According to embodiments of the present invention, systems and methods for digital optical aberration correction are described. The proposed system can correct optical aberrations to the diffraction limit without introducing imaging artifacts. In some embodiments, the proposed system can correct optical aberrations without reducing the light transmittance through the optical system. The ability to correct optical aberrations in an imaging system can simplify optics, enhance operating parameters, and broaden applications.

[0074] according to Figure 1a The embodiment of the present invention shown includes a digital optical aberration correction system 10 comprising an optical imaging unit 102, also referred to as an "imaging optics device". The optical imaging unit 102 is used to form an optical image on or near an image plane 104; a wavefront imaging sensor unit 106; and a control unit 108. The control unit 108 is used to process the raw digital image output of the object 100 to provide a deblurred image output. The control unit 108 can also measure the intensity of optical aberrations on the image plane 104.

[0075] Embodiments of the present invention can be interpreted to satisfy several applications: microscopes, cameras, telescopes, remote imaging systems, etc. For example, in a microscope application, the optical imaging unit 102 may consist of an objective lens, optional tube lenses, and other relay or telescope optics (not shown in FIG. 1) for forming an image on the image plane 104 of the system. For camera applications, the optical imaging unit 102 can be used as a camera lens, which may be a wide-angle lens, a standard lens, a telephoto lens, or a zoom lens. Figure 1a (Not shown in the image). For telescopes and remote imaging systems, the optical imaging unit 102 may include a refracting, reflecting, or catadioptric telescope system. Figure 1a (Not shown in the image).

[0076] Figure 1bAn optical system 12 according to an embodiment of the present invention is illustrated: the wavefront imaging sensor unit 106 may include a light modulator unit 110 adjacent to the image plane 104. The light modulator unit 110 uses phase modulation, amplitude modulation, or both to modulate light. The wavefront imaging sensor unit 106 may also include an image sensor detector array 112, also referred to as a downstream image sensor unit 112. The image sensor detector array 112 is used to acquire raw digital image output.

[0077] Depending on the application, the wavefront imaging sensor unit 106 can be configured as any conventional type of imaging sensor, such as a 2D region sensor, a line scan sensor, a multi-line scan sensor, or a TDI sensor.

[0078] The control unit 108 (in) Figure 1a and Figure 1b (As shown in the figure) may include a processing unit and a storage unit (not shown). The processing unit may be used to perform digital algorithm processing, as described below. The storage unit may store pre-calculated, pre-measured, or otherwise predetermined data indicating the static aberrations of the optical imaging unit, as described below.

[0079] The storage unit can also be used to store pre-calculated, pre-measured, or otherwise predetermined data indicating the chromaticity characteristics of the optical imaging unit. The storage unit can store pre-calculated, pre-measured, or otherwise predetermined data indicating the chromaticity characteristics of the wavefront imaging sensor unit 106 and optionally the optical modulator unit 110.

[0080] The optical imaging unit 102 can be of any type: refractive, reflective, reflective-refractive, diffractive, or a combination thereof. The optical imaging unit 102 can be designed and constructed using relaxed optical aberration specifications to simplify its design and manufacturing process, while relying on the digital aberration correction capabilities of the system described herein to restore image quality and sharpness. Aberration types caused by the optical imaging unit 102 that can be corrected by the system 10 include, but are not limited to: Seidelaberration (spherical), coma, astigmatism, field curvature and distortion, lateral and longitudinal chromatic aberration, and optionally their higher-order counterparts.

[0081] The wavefront imaging sensor unit 106 is responsible for providing raw data to the control unit 108 for diffraction-limited deblurring reconstruction. The wavefront imaging sensor unit 106 can also provide measurements of optical aberration intensities on the image plane 104. The term "raw data" refers to the digitization of light intensity measured by the image sensor 106. The raw data is further processed by the control unit 108, for example, as described below, to provide measurements of elements of the optical mutual coherence function or equivalent entity (e.g., the Wigner distribution). The wavefront imaging sensor 106 is designed to provide data on elements of the optical mutual coherence function of the light field near the image plane 104 of the imaging system at a diffraction-limited sampling resolution. As described in the background section, standard Shack-Hartmann type sensors, all-optical camera sensors, or angle-sensitive pixel sensors cannot provide this data at a diffraction-limited sampling resolution.

[0082] As used in this article, the term "wavefront imaging sensor" refers to the ability to provide diffraction-limited sampling resolution. As used herein, the term "wavefront sensor" refers to conventional techniques that cannot achieve diffraction-limited sampling resolution, such as Shack-Hartmann sensors, all-optical cameras, or angle-sensitive pixels.

[0083] According to an embodiment of the present invention, the wavefront imaging sensor unit 106 and the control unit 108 can operate according to the principles described in PCT Patent Publication No. WO / 2018 / 185740, which is incorporated herein by reference. According to an embodiment of the present invention, the wavefront imaging sensor 106 comprises an optical modulator 110 (also referred to as an encoder) located near the image plane 104 of the imaging optics and a downstream image sensor detector array 112. The optical modulator 110 (encoder) can be a static passive optical element that modulates the amplitude, phase, or both of the light arriving at the image plane 104. If only phase modulation is performed, the optical modulator 110 does not cause any additional losses in the optical system 10. From there, the modulated light propagates to the downstream detector array 112, is sampled, and then digitized. The raw digital data (the output of the sub-pixel sensor unit array 112) is then processed by the control unit 106. The relationship between the encoder unit 110 and the Nyquist sampling of the light field at the image plane 104 will be described below. The control unit 106 can be used to perform the functions of the control unit described in WO / 2018 / 185740, as well as additional control functions, such as those described below. The control functions can be performed by a separate control unit.

[0084] It should be noted that in some embodiments of the present invention, the optical modulator 110 may be a separate unit from the detector array 112. In other embodiments, the optical modulator 110 may be monolithically integrated with the detector array 112. According to yet another embodiment, the optical modulator 110 may be part of a process stack for manufacturing the sensor array 112, for example, using metallization process steps, or process steps similar to those used in manufacturing microlens arrays, to improve the pixel fill-factors in the sensor array.

[0085] The wavefront imaging sensor unit 106 provides the measurement of the optical mutual coherence function on the image plane 104 of the imaging system. The optical mutual coherence function ρ of the light field E(x) defines the spatial correlation of the average time field, as follows:

[0086] (Equation 1)ρ(x,x')={E(x)E*(x')}

[0087] In Equation 1, x = (x, y) is related to the transverse spatial coordinates of the field. Typically, using detector arrays and digitized data requires discretizing the coherence to provide a coherence matrix of the following form:

[0088] (Equation 2)ρ ij ={E(x i E*(x) j )}

[0089] In Equation 2, E(x) i ) and point The conjugate complex number of the field is related, and E*(x) j ) and point The field's conjugate complex correlation. It should be noted that, physically, it is possible to realize a coherence matrix that is Hermitian and nonnegative.

[0090] As is known in the art, conventional imaging and detection techniques or typical detection devices or systems provide data indicating diagonal elements, which correspond to conventional intensity measurements, i.e., l i =ρ ii ={E(x i E*(x) i The measured intensity l i =ρ ii This represents the number of photons / rays hitting each detector pixel regardless of the incident angle, providing a discrete measurement of the lateral ray position x. Measurement intensity l i =ρii No information is provided about the off-diagonal element. The off-diagonal element of a mutually coherent function contains the phase and coherence relation of the optical field.

[0091] The phase and coherence relations in the off-diagonal elements represent the angular information of the light rays illuminating the wavefront imaging sensor 106 at the image plane 104 of the optical system. In physical optics, the correlation between wave position and angle is completely described by the Wigner distribution W(x,θ), where x = (x,y) is related to the transverse spatial coordinates of the field, and θ = (θ... x ,θ y The Wigner distribution describes the angle of a ray's trajectory. In the limit of ray optics, the Wigner distribution simplifies to the traditional light field description of geometric optics, where the position x and propagation angle θ of each ray are recorded. However, the Wigner distribution is actually a "quasi-probability" distribution that can reach negative values, representing wave-like phenomena in physical optics.

[0092] Mathematically, the Wigner distribution and the coherent function are completely equivalent; they are related through the Fourier transform (Wigner-Weyl transform):

[0093] (Equation 3)

[0094] As is known in the art, conventional all-optical cameras, angle-sensitive pixel sensors, or Shack-Hartmann type sensors cannot provide diffraction-limited imaging resolution or aberration deblurring. According to an embodiment of the invention, the wavefront imaging sensor 106 can provide diffraction-limited sampling of the mutually coherent function. From the Wigner-Weyl transform, we obtain a Wigner distribution with diffraction-limited spatial sampling. The angular sampling resolution of the obtained Wigner distribution is related to the maximum off-diagonal element distance measured in the mutually coherent function.

[0095] According to embodiments of the invention, the combined position angle information contained in the Wigner distribution is then used to perform digital aberration correction. For illustration, the following simple example demonstrates how digital aberration correction is performed. The simple example shown can be extended to cover all aberration types across any type of imaging scene.

[0096] Figure 2a The optical configuration 20 is depicted. In this simple example, we consider imaging two different but close point sources a and b using ideal optics 202. The overlapping blurred image of point a and the blurred image of point b are shown in... Figure 2bThis is illustrated in more detail below. Two point sources, a and b, are located outside the focal plane of the imaging system, meaning they are out of focus, or even possibly defocused to different degrees. In this example, defocus and subsequent blur are considered as examples of imaging aberrations in general and can be extended to any type of aberration. The degree of defocus and spatial separation is chosen such that the blurred images of the two point sources a and b overlap.

[0097] If the point sources are spaced further apart so that their blurred images do not overlap, then known techniques would be suitable for digital deconvolve on each of their images, and even for using phase retrieval or other techniques to estimate their aberration levels. Typically, blurred images contain many overlapping, and optionally different, blur kernels, each corresponding to a point source in the imaging object space.

[0098] Therefore, we find that the general problem at present is how to perform digital estimation of the blur kernel for two or more overlapping blur kernels that may have different degrees of aberration, as in the simple example discussed, such as Figure 2a As shown. We now refer to the ray tracing geometric optics description of the current problem, such as... Figure 2c As shown. We found that in the overlapping blurred region 214 of the two defocused blurred points a and b, there exists a point containing light rays 216 that are directed towards both the image of the first point source and the image of the second point source. Figure 2c In this specific example, the convergence point is placed behind the image plane 204 of the imaging system, and is therefore a defocused point source image. Without loss of generality, the same consideration is valid if one or both point sources are imaged before the image plane 204 (not shown). Within the overlap region 214, each spatial point contains two distinct rays 216 at different angles, one ray 216a originating from the first point source (point source a) and the other ray 216b originating from the second point source (point source b).

[0099] Assuming the use of a conventional all-optical light field sensor or an angle-sensitive pixel imaging sensor placed on the image plane 204: these types of sensors are configured to provide a one-to-one correspondence between output data and the input light rays illuminating the sensor. Each light field data point corresponds to a specific location in space x, with a unique propagating ray θ. The angle-sensitive pixel detector also provides such one-to-one information. When a particular pixel lights up, it indicates the ray position x and a single incident angle θ. On the other hand, as we just discussed... Figures 2a-2cAs illustrated in the simplified example, within the overlapping region 214, there exist positions containing two distinct light rays passing through them at different angles. Because the one-to-one correspondence is disrupted, conventional angle-sensitive or all-optical light field sensors cannot accurately record these types of dual-ray or multi-ray points. We also note that this one-to-one correspondence constraint is partly why conventional angle-sensitive pixel sensors and all-optical light field sensors cannot perform imaging up to the diffraction limit. When operating at a resolution coarser than the diffraction-limited spatial resolution, the ray-traced light field description of the light field does not include this blurred region where the one-to-one correspondence between ray positions and angles is disrupted.

[0100] On the other hand, the Wigner distribution W(x,θ) can safely describe and quantify the ambiguity caused by the disruption of the one-to-one angle-position correspondence. Recall that the Wigner distribution is extracted from the mutually coherent function ρ(x,x')={E(x)E*(x')} using the Wigner-Weyl transform, which is in turn estimated by the wavefront sensor unit. The Wigner distribution is not only a wave optical simulation of the traditional optical field geometry ray optics description; it also contains coherence and interference information. The Wigner distribution accurately records and describes the overlapping combinations of rays, such as in... Figures 2a-2c Ray 216 is described in the overlapping region 214 of the example shown above.

[0101] Using the Wigner distribution, overlapping but distinct light rays within the overlapping blurred region 214 can be measured and identified. By doing so, we can simplify the problem to a simpler, non-overlapping blurred point source. We can then perform deblurring and estimate the aberrations and defocusing that cause the blur.

[0102] The Wigner distribution contains data indicating the spectral content of the light field. This complements the data indicating the light field intensity, coherence, and phase. The spectral content can be obtained through the wavelength dependence described in Equation 3.

[0103] Wavefront imaging sensors can also be used to extract spectral information about the imaging input scene. The spectral information contained in the Wigner distribution can be used to estimate the spectral distribution (“spectrum”) of each point source. The spectral information can be optionally used to reconstruct a spectral map of the imaging scene, or as a color (RGB), multispectral, or hyperspectral image.

[0104] Figure 3a This is a flowchart illustrating method 30 according to an embodiment of the present invention, used for, for example... Figures 1a-1b The configuration in [the document] performs digital deblurring, aberration estimation, and possibly spectral distribution estimation. For ease of explanation, we refer to [the document / reference]. Figures 2a-2c The imaging scene shown illustrates these steps.

[0105] Method 30 begins with the operation 300 of capturing an image and generating raw data, for example, using... Figures 1a-1b The imaging wavefront sensor unit 106 is shown positioned near the image plane 104 of the imaging system. The imaging wavefront sensor unit 108 (e.g., composed of...) Figure 1b The raw data generated by the detector pixel array 112 shown is, for example, passed to... Figures 1a-1b The control unit 108 shown is shown.

[0106] Following operation 300 is operation 302, which, for example, via control unit 108, calculates the field characteristics of, for example, the Wigner distribution of the light field striking wavefront imaging sensor 106. According to one embodiment of the invention, operation 302 is performed via operations 302a and 302b, whereby operation 302a calculates the optical mutual coherence function based on the raw data; and operation 302b transforms the optical mutual coherence function into an equivalent optical Wigner distribution using the Wigner-Weyl transform (Equation 3).

[0107] Following operation 302 is operation 304, which separates the individual point sources. The coherence and angle combination information in the Wigner distribution is used to distinguish the different point sources that make up the entire imaging scene.

[0108] Following operation 304 is operation 306, which calculates the fuzziness of each point source.

[0109] Optionally, operation 308 is performed after operation 306 to estimate the aberration intensity of each point source.

[0110] Following operation 306 (or optionally, operation 308) is operation 309, which estimates the spatial spectral distribution of each point source.

[0111] Following operation 306 (or optionally, operation 308 or operation 309) is operation 310, which reconstructs the deblurred image. For example, the deblurred image is reconstructed as a composite of the deblurred point sources.

[0112] Following operation 310 is operation 312, which reconstructs spatial maps of various aberration intensities from point-by-point aberration estimation.

[0113] After operation 310 (or optionally, operation 312), operation 314 is optionally performed to calculate the spectral distribution of each point source.

[0114] Optionally, after operation 314, operation 316 is performed to reconstruct a spatial map of the spectral distribution of the input light from point-by-point spectral distribution estimation.

[0115] Figure 3b This is a flowchart illustrating method 34 according to an embodiment of the present invention, for use, for example... Figures 1a-1b The configuration in [the document] is used for spectral distribution estimation. For ease of explanation, we refer to [the document / reference]. Figure 2a Figure 2c illustrates these steps using the imaging scenario.

[0116] Method 34 begins with operation 340, for example, by using an imaging wavefront sensor unit 106, positioned near the image plane 104 of the imaging system as shown in Figures 1a-1b, to capture an image and generate raw data. The imaging wavefront sensor unit 108 (e.g., by...) Figure 1b The raw data generated by the detector pixel array 112 shown is, for example, passed to... Figures 1a-1b The control unit 108 shown is shown.

[0117] Following operation 340 is operation 342, in which, for example, control unit 108 calculates field characteristics, such as the Wigner distribution of the light field striking wavefront imaging sensor 106. According to one embodiment of the invention, operation 342 is performed by operations 342a and 342b, whereby operation 342a calculates the optical mutual coherence function based on the raw data; and operation 342b transforms the optical mutual coherence function into an equivalent optical Wigner distribution using the Wigner-Weyl transform (Equation 3).

[0118] Following operation 342 is operation 344, which separates the individual point sources. The coherence and angle combination information in the Wigner distribution is used to distinguish the different point sources that make up the entire imaging scene.

[0119] Following operation 344 is operation 346, which estimates the spatial spectral distribution for each point source.

[0120] Following operation 346 is operation 348, which reconstructs the spatial map of the spectral distribution of the input light from the point-by-point spectral distribution estimation.

[0121] Optionally, perform some or all of the following additional operations for aberration correction: operation 350 to calculate the blur of each point source; operation 352 to estimate the aberration intensity of each point source; operation 354 to reconstruct the deblurred image; and operation 356 to reconstruct a spatial map of various aberration intensities from point-by-point aberration estimates.

[0122] Without loss of generality, since the Wigner distribution and the mutual coherence matrix are mathematically equivalent and correlated through variations in the representative basis, methods 30 or 34 can be performed to compute the properties of the field in any basis representation, each containing the desired coherence, angle, phase, and spectral information. Therefore, operations 302, 304, and 306 of method 30 above, or operations 342, 344, and 346 of method 34 above, can be performed on a Wigner basis, a mutual coherence basis, or any other basis obtained as a combination of elements of the mutual coherence or Wigner distribution matrix.

[0123] To illustrate, we provide an example of numerical simulation to demonstrate the method and system performance according to embodiments of the present invention. In this example, a Siemens star-type target is used as the source reference object, such as... Figure 4 As shown. Target (e.g., Figure 4 It possesses extreme periodicity, contrast, and geometry. The following illustrates (for example, in...) Figure 1a or Figure 1b The performance of the optical imaging system for Siemens star-shaped targets.

[0124] Optical imaging systems (e.g.) Figures 1a-1b The system 10 or 12 shown in the diagram may have a composition of Figure 5a The point-spread function (PSF)50 shown describes the aberrations. Clearly, this is far from the diffraction-limited point. This corresponds to... Figure 5b The wavefront error depicted is 52, which is almost four peaks to peak. This aberration imaging system was used to image the Siemens star (…). Figure 4 Imaging produces corresponding optical mutual coherence functions, such as Figure 6a (Real part 60) and Figure 6b (Imaginary part 62) is shown. The equivalent Wigner distribution 70 comes from the Wigner-Weyl transform, as shown... Figure 7 As shown. Figure 8a A blurred image 80 obtained from an aberration optical imaging system is shown. Figure 8b The deblurred image 82 reconstructed from information in the Wigner distribution 70 using the method described above is shown.

[0125] The method according to embodiments of the present invention can be used to correct optical aberrations in a wide range of applications. Embodiments of the present invention can handle several types of aberrations:

[0126] Static aberrations are caused by the design and manufacture of optical imaging systems. These static aberrations can be determined in advance through proper calculation, measurement, and characterization of the optical imaging system.

[0127] Dynamic aberrations are caused by changes in the optical imaging system due to environmental factors or changes in the medium that performs the imaging, such as atmospheric turbulence and eye aberrations. These dynamic aberrations are unknown in advance and may vary from image to image due to changes in external factors affecting an individual imaging system.

[0128] Color aberrations can be lateral or longitudinal spectral aberrations, or combinations thereof, or higher-order spectral correlation variants. Based on the above definition, these types of color aberrations can be further classified as static or dynamic.

[0129] Back to Figure 3a Method 30: According to an embodiment of the invention, the fuzzy estimation operation 306 may consider classifying aberrations into static aberrations or dynamic aberrations. In the static case, the desired fuzziness is known in advance, while in the dynamic case, it is dynamically calculated based on each image instance. Static aberration parameters may be calculated or measured or otherwise predetermined and stored in the storage unit of the control unit.

[0130] The difference between static and dynamic aberrations also affects the number of pixels used in a wavefront imaging sensor unit. Figures 1a-1b (Component 106 in the text). To explain this, we need to distinguish the original number of pixels N of the image sensor based on the diffraction limit Nyquist sampling limit given by Δ = λ / 4NA. R and the required number of sampling points N N In conventional imaging, a well-sampled image can be obtained if the number of sensor pixels is equal to or less than the Nyquist limit Δ = λ / 4NA, meaning the number of original pixels is at least equal to the number of Nyquist sampling points. This is derived from the number of independent degrees of freedom of the light field intensity on the image plane. The number of independent degrees of freedom is expressed as the number of Nyquist sampling points N. N To describe them. Therefore, in order to fully account for these degrees of freedom, they need to be described using at least N. R ≥N N Sampling was performed at each sampling point.

[0131] This principle extends to wavefront imaging sensors (e.g.) Figures 1a-1b The design of component 106). If all aberrations are static and known in advance, then the number of unknown degrees of freedom is N. N Therefore, the required number of raw pixels should be at least Nyquist sampling points N. R ≥N N However, if dynamic aberrations are considered, additional unknown degrees of freedom exist. For each type of dynamic aberration A... i For example, magnification, coma, and astigmatic aberration are all associated with a new set of degrees of freedom numbering. Related to the hypothetical spatial variation of each aberration. The value depends on: for spatially invariant aberrations For dynamic aberrations where arbitrary intensity values ​​can be assumed at any point on the image plane For aberrations with slow spatial dependence

[0132] Therefore, in order to fully consider image degrees of freedom and dynamic aberrations, wavefront imaging sensors need at least in This depends on the spatial variability of each dynamic aberration. Naturally, using more raw pixels increases redundancy and can improve the robustness and signal-to-noise ratio of wavefront imaging sensors.

[0133] To obtain a higher pixel count according to the above relationship, the actual original pixel count needs to be less than the Nyquist limit Δ = λ / 4NA. This is in Figures 9a-9b It is shown schematically in the middle. Figure 9a The Nyquist sampling grid 90 is shown. Figure 9b The original pixel sampling grid 92 is displayed. For ordinary imaging sensors, this simply means oversampling the optical image without generating new information. However, for wavefront imaging sensors according to embodiments of the present invention (e.g., Figure 1b As shown), the optical modulator generates new optical information, which can be generated by an image sensor detector array (e.g., Figure 1b The element 112) is sampled to generate raw data, and this new information is used to perform deblurring and dynamic estimation of aberration intensity as described above.

[0134] Wavefront imaging sensors (e.g.) Figures 1a-1b Component 106 in the middle can also be used in N R <N N In this case, in addition to performing static aberration correction, it also acts as an anti-aliasing filter, producing a deblurred image with a resolution below the Nyquist limit, and an output resolution of N. O (Number of pixels in the output image). When in When used in conjunction with dynamic aberrations, the anti-aliasing effect still exists, resulting in resolutions below the Nyquist limit N. O <N N Deblurred image.

[0135] It can also be set to output resolution N O Below the number of Nyquist sampling points N N The calculation of color field characteristics (e.g., spectral bands or color channels in the output image) is performed. In these cases, in order to calculate N... O <N N Extracting N at the output resolution of each pointC For color field characteristics (e.g., spectral bands), we need N. C x N O ≤N R That is, the raw number of pixels of the image sensor.

[0136] In embodiments of the present invention, such as the wavefront imaging sensor described in PCT patent publication WO / 2018 / 185740, the encoder unit lattice is associated with a Nyquist sampling grid, optionally having a unit lattice size and a non-integer ratio between the Nyquist grids. The ratio between the unit lattice size and the Nyquist grid depends on the desired sampling level; a 1:1 ratio can provide good sampling, while oversampling occurs when the unit lattice is smaller than the Nyquist grid, and undersampling occurs when the unit lattice is larger than the sampling grid. The number of sub-pixels within each unit lattice can be a fixed integer, and the total number of original data pixels can be equal to the number of unit lattices multiplied by the number of sub-pixels per unit lattice.

[0137] If the unit lattice is larger than the Nyquist grid, well-sampled mutually coherent and Wigner distribution data can also be recovered, but the total number of original data pixels conforms to the above sampling relationship.

[0138] It should also be noted that encoder unit lattices, such as those described in PCT patent publication WO / 2018 / 185740, are described as periodic. However, in some embodiments, quasi-periodic or aperiodic structures can be used to illustrate spatially dependent optical characteristics within the desired field of view (FOV). For example, the encoder spacing can be adjusted to account for variations in the chief-ray angle (CRA) across the FOV. Furthermore, the degree and type of aberrations may differ across the FOV, thus requiring a different optimal optical encoder / modulator design for each location within the FOV. The modulator design can be varied according to these requirements, resulting in quasi-periodic or aperiodic designs.

[0139] Therefore, according to embodiments of the present invention, the optical modulator unit (e.g.) Figure 1b Element 110 in the middle may include multiple unit lattices ( Figure 1b (not shown in the image), and the image sensor detector array unit 112 may include a sensor unit array ( Figure 1b (Not shown in the image); the sensor unit array of element 112 can define multiple sensor subarray units ( Figure 1b(Not shown in the image), each sensor subarray corresponds to one of the plurality of unit lattices of the optical modulator unit 110; and the optical modulator unit 110 is used to apply premodulation to the input light collected by the image sensor unit 112. Each unit lattice of the optical modulator unit 110 can guide a portion of the collected input light incident thereon to its corresponding sensor subarray unit and one or more adjacent sensor subarray units in a predetermined neighborhood.

[0140] Figures 10a-10d Various embodiments of an optical modulator unit 110 with a periodic unit arrangement are schematically illustrated. The present invention is not limited to a specific unit arrangement of the optical modulator unit 110.

[0141] Figure 10a It is a binary design, with one disk cell per unit lattice. The disk can use phase modulation, amplitude modulation, or a combination of both. It should be noted that in some cases, phase modulation is suitable because it does not result in light loss, meaning 100% of photons reach the underlying sensor pixels. Typically, the disk can have a fill factor of 25% to 75%, and phase modulation can range from 90 degrees to 270 degrees at the design wavelength.

[0142] Figure 10b A multi-layered design consisting of concentric rings is described. The rings can have equally spaced radii or other increments. Similarly, phase or amplitude modulation, or a combination of both, is possible.

[0143] Figure 10c and Figure 10d This depicts a binary disk or multi-ring design, with two disk / ring groups arranged at the edge of each unit lattice. In practice, these are equivalent to... Figure 10a and Figure 10b The design was rotated 45 degrees and shrunk. times.

[0144] Figure 11a and Figure 11b The relationship between the unit lattice of the optical modulator unit 110 and the corresponding sensor subarray of the detector array 112 according to an embodiment of the present invention is depicted. Figure 11a In this system, each unit lattice has a 2x2 sensor subarray of pixels, while... Figure 11b In this design, each unit lattice has 3x3 sensor subarray pixels. Of course, a higher number of MxN pixels can be used in the sensor subarray, and the present invention is not limited by the number of pixels in the optical modulator unit 110 and the detector array 112.

[0145] We note that even if the unit lattice of the optical modulator is greater than the Nyquist limit Δ = λ / 4NA, as long as the original number of pixels is N...R (Determined by the number of subarray pixels multiplied by the number of unit lattice cells) is sufficient to ensure N R ≥N N Used to restore images, or in the case of dynamic existence. This allows for the attainment of diffraction-limited imaging resolution aberrations. Chromatic aberration can be addressed using techniques known in the art, employing wavefront imaging sensors designed with the desired spectral sensitivity bands. One technique utilizes conventional color filter arrays, such as Bayer filters. Figure 12a The example provided is non-limiting, where each number represents a different spectral band filter. Figure 12b An optical system 12 according to an embodiment of the present invention is schematically illustrated. System 12 includes a reference... Figures 1a-1b The components discussed are 100, 102, 104, and 108. System 12 also includes a multi-sensor configuration 206—including wavefront imaging sensors 206λ1-206λ3—with appropriate dichroic mirrors or other beam splitters 114 to provide spectral filtering. Spectral sensitivity banding can be achieved using the multi-sensor configuration 206. Other possible configurations may include combinations of dichroic or other beam splitters and spectral filters. Each spectral element can be processed using a separate Wigner distribution, and then combined spectral sensitivity deblurring is applied.

[0146] It should be noted that the total number of original pixels should include the required degrees of freedom as described above, and spectral bands should also be considered. According to one embodiment, the number of original pixels required to replicate a single spectral band, for all spectral bands, can be reflected in multiple original pixels used in conjunction with a color filter array, for example... Figure 12a As depicted therein. According to another embodiment, if according to... Figure 12b If the example uses a multi-sensor configuration, then each sensor 206 should include an appropriate number of raw pixels, sufficient to cover the degrees of freedom required for the relevant spectral bands.

[0147] According to embodiments of the present invention, using, for example Figure 1a , 1b The system shown in 12b is used to perform hyperspectral imaging (HSI). Hyperspectral imaging is a technique for analyzing a broad spectrum of light, rather than simply assigning primary colors (red, green, blue) to each pixel. The light illuminating each pixel is broken down into many different spectral bands to provide more information about what is being imaged.

[0148] According to embodiments of the present invention, hyperspectral imaging is achieved by providing a degree of redundancy in the total number of original sensor pixels, taking into account the number and type of aberrations being measured. The total number of original sensor pixels is selected to be higher than the number of measured spectral bands, taking into account the number and type of aberrations to be estimated.

[0149] For example, using 3x3(9) raw pixels per Nyquist pixel provides sufficient redundancy to extract intensity from 8 spectral bands and one additional depth measurement or aberration estimation channel. Under these conditions, the Wigner distribution of additional spectral bands can be extracted, which are interpolated or extrapolated from the spectral filters actually used. This can be achieved with relatively simple color filter arrays or dichroic prism configurations for hyperspectral imaging. For example, with an RGB Bayer filter, hyperspectral imaging data can be extracted for additional spectral bands in the red-green and green-blue spectral regions, as well as fine spectral sampling extrapolated to the violet and red regions of the spectrum.

[0150] Figure 13a Four spectral band filters 1-2-3-4 are schematically shown, corresponding to, for example, Figure 12a The color filter array arrangement shown, or as Figure 12b The dichroic prism configuration shown is illustrated. As described above, the spectral information can be refined within each spectral band, thereby generating spectral information with more than four wavelengths.

[0151] Furthermore, with sufficient redundancy in the total raw pixels, spectral information can be extracted into, for example, ultraviolet or infrared spectral regions across a wide spectral band (e.g., the entire visible light range, 400-700 nm, or even beyond). This enables full-color RGB imaging, multispectral imaging, or hyperspectral imaging without the use of any type of spectral filter or color filter array, significantly improving collection efficiency due to the absence of photon loss. Such a sensor is particularly useful in low-light conditions.

[0152] According to embodiments of the invention, the sensor is configured to capture standard RGB imaging in the visible range (e.g., 400-700 nm) and near-infrared (IR) light. Near-infrared (IR) light can be, for example, in the range of 700-1000 nm or even beyond, or in the SWIR range, depending on the spectral sensitivity of the underlying sensor pixels. Combined RGB+IR sensing is highly useful for various applications involving inconspicuous floodlight IR illumination. For example, combined RGB+IR sensing allows for the capture of full-color images while illuminating the scene with invisible floodlight IR to enhance low-light performance. According to another example, the scene is illuminated with an invisible IR dot pattern to enhance depth imaging.

[0153] Using wide spectral bands can introduce ambiguity when extracting spectral data. This ambiguity can be overcome by using a "comb" spectral filter, which divides the spectrum arriving at the sensor into a series of narrower, distinct spectral bands, such as... Figure 13b The spectral distribution shown is 1. (Reference) Figure 12b The system shown here applies this spectral filter to the entire image and can be positioned anywhere in the optical system 12 or as part of the image sensor 206. In addition to the comb pattern described herein, spectral deblurring can also be affected by other spectral filters with different spectral patterns, such as spectral distribution 2 in 13b.

[0154] Without loss of generality, we arrive at the state described above and in Figure 12b , Figure 13a and Figure 13b The control of the spectral content of the depicted sensor 206 can be achieved by combining it with the use of a suitable illumination source or by incorporating a different spectral filter. For example, a narrowband illumination source such as an LED can be used to illuminate RGB or other spectral distributions. The spectral content of the illumination source can be changed sequentially, essentially resulting in a control similar to the spectral content of the sensor 206. Figure 12b and Figure 13a The settings shown are equivalent to, or can be changed simultaneously, such as... Figure 13b As shown and described above.

[0155] The ability to digitally correct optical aberrations can relax design and manufacturing constraints on imaging optics, potentially leading to lower costs, smaller size, enhanced operating parameters, or increased robustness to environmental factors. According to embodiments of the invention, simplification of the optics can manifest itself as various combinations of one or more of the following operating parameters:

[0156] Reduced number of refractive and reflective optical elements: Due to greater aberration latitude, the need for complex combinations of mutually compensating optical elements is reduced.

[0157] Simpler optical surfaces, such as spheres, can be used: optical designs can be simplified to utilize spheres instead of aspherical or freeform surfaces.

[0158] Low-cost materials for optical components: reducing the need for specialized materials that may be more expensive or require more expensive manufacturing processes.

[0159] Shorter optical path: Reducing the number of components can result in a shorter optical path, which can optionally be combined with faster optics, see below.

[0160] Faster optics (lower f / #, higher NA): Relaxed aberration tolerances allow for the use of larger apertures or shorter focal lengths. In microscopes, this results in higher numerical apertures (NA).

[0161] Relaxed manufacturing tolerances: These can manifest as wider tolerances for individual optical elements, surfaces, or materials, resulting in higher throughput or lower costs.

[0162] Relaxed alignment tolerances: A wider range of optical element misalignment simplifies the installation and alignment process.

[0163] Relaxed system stability and repeatability requirements: The system can adapt to a wider range of environmental factors, such as heat, pressure, and shock, which can affect the alignment and performance parameters of optical components.

[0164] Extended focal length range: Compensates for out-of-focus blur to achieve an extended depth of focus range.

[0165] Larger field of view: In many cases, the aberrations of an optical system increase towards the edges of the field of view. Relaxing the aberration tolerance can allow for a larger field of view.

[0166] Adaptable to stronger field curvature: The extended focus range allows for stronger field curvature and optionally a larger field of view.

[0167] Larger working distances: In microscope objectives, large working distances typically require large and complex designs to overcome aberrations. The ability to digitally correct aberrations can simplify the design for a given working distance, or an increase in working distance, or both.

[0168] Microscope slides, coverslips, sample trays, and volume chambers: These allow for relaxed tolerances in microscope objective design using coverslips on microscope slides or thicker slides. They also allow inverted microscopes to operate at higher NA (nanometers) and with better penetration into the sample tray. They also allow for the use of thicker slides or sample chambers. Slides or sample chambers with volumetric or microfluidic channels and maneuverability can be used.

[0169] Immersion objective: Increases the tolerance for refractive index matching and eliminates the requirement for a refractive index adjustment ring for immersion objectives.

[0170] RGB color, multispectral, or hyperspectral imaging: The ability to perform real-time, single-snapshot RGB color, multispectral, or hyperspectral imaging without using a color filter array. This improves light sensitivity for several reasons, as no photons are filtered out; they all reach the pixel array of the detector unit. This feature can be used in conjunction with the deblurring function described above.

[0171] Many optical imaging systems have facilities for adjusting various imaging parameters, such as controlling the aperture stop, changing the focus, and changing the focal length using a zoom mechanism. According to embodiments of the invention, if these mechanisms are repeatable to the desired extent, then the variations in their effects within the imaging system unit can be predetermined and stored as various sets of static aberrations in the storage unit of the control unit, each such set corresponding to a specific imaging parameter configuration.

[0172] According to other embodiments of the invention, in order to simplify and reduce costs, certain optical imaging parameter mechanisms can have more relaxed repeatability tolerances, which can be compensated for using dynamic aberration deblurring.

[0173] The ability to calculate the dynamic aberration intensity at each point in the image plane can be used to estimate the depth at each point in the image plane, and to perform digital deblurring to compensate for defocusing and thus obtain extended depth focusing range. In this case, the relevant aberration is magnification, which depends on the distance from the imaging system to each point in object space. The spatial map of magnification aberration intensity is directly related to the depth map, and the algorithm's deblurring capability can restore diffraction-limited imaging resolution, and is also applicable to initially out-of-focus image portions. Furthermore, for a specific optical imaging system and its characteristic aberrations, the distance to each point in object space may cause additional aberrations beyond magnification, such as spherical aberration or other aberrations. Information about these additional aberrations can also be used to enhance the depth estimation for each point source in object space.

[0174] Without loss of generality, this scheme can be extended according to embodiments of the invention to operate over a wide spectral range, wherein chromatic aberration is used to estimate depth, and a wavefront imaging sensor can be used. Figures 10a-10b The system can be configured as depicted in one of the above or other configurations. According to embodiments of the invention, the system can also accommodate static aberrations present in optical imaging systems.

[0175] According to embodiments of the present invention, depth estimation can be used in conjunction with simplified imaging optics.

[0176] According to embodiments of the present invention, in order to perform digital adaptive optics, dynamic aberrations at each point in the image plane are calculated to accommodate various degrees of aberration beyond the magnification. Digital adaptive optics is used to perform digital deblurring and restore diffraction-limited imaging resolution when aberrations are not known in advance.

[0177] Aberrations cannot be known in advance when they are caused by various environmental factors that affect imaging system units in unpredictable ways, such as changes in temperature or pressure, shocks, airflow within the optical system, and the sagging of optical elements under their own weight. External environmental factors may also play a role, such as imaging through atmospheric turbulence. In many biological applications, aberrations caused by light passing through biological tissue limit the microscopic imaging of large tissues, either in vivo or in the laboratory. In microophthalmic imaging through the subject's eye, aberrations in the cornea and lens can lead to a significant decrease in image sharpness and quality.

[0178] In existing technologies, these types of aberrations are partially corrected using complex, bulky, and expensive adaptive optics systems. Such systems are difficult to integrate into existing imaging systems, requiring modifications to the optical path to combine wavefront sensors and wavefront modulation elements, typically in the form of additional optical relays and beam splitters.

[0179] Many adaptive optics systems are designed so that the wavefront sensor and wavefront modulation element are located at or near the pupil plane of the imaging system, thereby sampling and correcting optical wavefront aberrations uniformly across the entire imaging field. Spatially dependent adaptive optics correction is typically performed in scanning optical imaging systems, which complicates the overall system design.

[0180] In many cases, an additional "guide star" illumination system is required to provide sufficient feedback signals to the wavefront sensor, which further complicates the system and limits its usefulness and operational range.

[0181] The dynamic aberration correction according to embodiments of the present invention allows the original imaging system to be used in conjunction with a wavefront imaging sensor, thereby simplifying the overall optical system.

[0182] According to embodiments of the present invention, spatially correlated deblurring can be provided, a feat that is extremely complex using conventional adaptive optics systems.

[0183] According to a further embodiment of the present invention, the implementation of the embodiments of the present invention avoids the use of "star-drawing", thereby further simplifying the process.

[0184] According to embodiments of the invention, for example, in situations with low light levels, the invention can be extended using a "guide star" to improve the SNR of the digital deblurring algorithm.

[0185] According to embodiments of the present invention, a digital adaptive optics system can be implemented to compensate for changes in the optical imaging system caused by environmental factors, as well as to compensate for blurring caused by external factors affecting the imaging medium.

[0186] According to embodiments of the invention, a digital optical aberration correction system can be incorporated into an existing adaptive optics imaging system, wherein a wavefront imaging sensor replaces the conventional imaging sensor in the adaptive optics imaging system, or a conventional imaging sensor is provided in addition to it. This provides the “last mile” for aberration correction, compensating for residual aberration errors left by the adaptive optics system, and also allows for simpler compensation of spatially correlated aberrations. Such a configuration can be used with or without a “guide star.”

[0187] Figure 14 This is a schematic block diagram of an aberration correction system 14 with an adaptive optics system unit 140 according to an embodiment of the present invention. The adaptive optics system unit 140, enclosed by a dashed box, is shown in a general manner. According to an embodiment of the present invention, the adaptive optics system unit 140 includes a wavefront sensor 1410 and a wavefront modulator 1406 located near the pupil plane 1412 of the imaging system. This may require an extended optical path including a pupil relay 1402 and a tube lens 1408. The adaptive optics wavefront sensor 1410 provides feedback to drive the wavefront modulator 1406 under the control of the adaptive optics control unit 1414. Without loss of generality, Figure 14 An adaptive optics system is illustrated, wherein an adaptive optics wavefront sensor 1410 measures the original wavefront prior to wavefront correction of the wavefront modulator 1406. Other adaptive optics system configurations known in the art exist, wherein the adaptive optics wavefront sensor 1410 measures the residual wavefront error after correction by the wavefront modulator 1406. This design variation of the adaptive optics system, as well as other similar design variations, are within the scope of this invention, and Figure 14 The adaptive optics system described herein is for illustrative purposes only.

[0188] According to an embodiment of the present invention, an adaptive optics wavefront imaging sensor 1410 is located on the image plane 1414 of the system, replacing the imaging sensor 206. According to an embodiment of the present invention, for example, refer to... Figure 1a , Figure 1b Figure 3 Figure 12b The wavefront imaging sensor 306 and control unit 108 discussed herein can correct residual aberrations left by the adaptive optics system unit 140 and also compensate for spatially dependent blur.

[0189] According to an embodiment of the present invention, the control unit 108 can operate independently of the adaptive optics system unit 140.

[0190] According to other embodiments, the control unit 108 may also use information from the adaptive optics wavefront sensor 1410 to provide a coarse measurement of wavefront aberrations, and then perform fine, possibly spatially dependent, digital adaptive optics corrections, as described above with reference to FIG3.

[0191] The digital adaptive optics system 14 can be used to provide depth estimation and in conjunction with the previously described simplified optics scheme.

[0192] The embodiments of the present invention can be used in a variety of applications.

[0193] According to embodiments of the present invention, the digital aberration correction system can be used in cameras with shorter lens tracks and reduced number of optical elements, such as mobile phone cameras that do not protrude from the phone case, and small cameras for wearable devices, drones, and robots.

[0194] According to embodiments of the present invention, a digital aberration correction system can be used in cameras with RGB, multispectral, or hyperspectral capabilities but without a color filter array to improve sensitivity, and may also be used for digital deblurring and extended depth-of-field capabilities.

[0195] According to embodiments of the present invention, a digital aberration correction system can be used in cameras hidden behind display panels (e.g., displays in mobile phones or vehicles). Digital aberration correction can compensate for aberrations caused by imaging through the display device layer. Furthermore, color RGB, multispectral, or hyperspectral imaging can be obtained without a color filter array to improve light sensitivity, compensate for light loss, and address spectral imbalances that may result from imaging through the display device.

[0196] According to embodiments of the present invention, a digital aberration correction system can be used in a simplified ophthalmoscope with higher imaging resolution. The simplified ophthalmoscope allows for simpler area imaging rather than a scanning configuration. The simplified ophthalmoscope can provide improved retinal scan resolution for biometric purposes. The simplified ophthalmoscope can be used in place of a conventional adaptive optics system, or added to an existing system as a “last mile” fine correction.

[0197] According to embodiments of the present invention, the digital aberration correction system can be used in various combinations applicable to microscopes. Microscopes employing the digital aberration correction system according to embodiments of the present invention can provide one or more of the following advantages: simplified objective design, wider field of view; greater working distance; enhanced sample handling parameters, such as slides, coverslips, volume chambers, and microfluidic channels; increased working tolerance of immersion objectives; expanded focal depth; and digital aberration correction in turbid media.

[0198] The digital aberration correction system according to embodiments of the present invention can be implemented as part of a conventional imaging microscope; as part of a confocal microscope; or for use in a light sheet microscope.

[0199] The digital aberration correction system according to embodiments of the present invention can be used in microscopes for applications requiring RGB color, multispectral or hyperspectral sensitivity, especially for nonlinear imaging.

[0200] The digital aberration correction system according to embodiments of the present invention can be used in microscopes used for bright-field, dark-field, differential interferometry (DIC), phase aberration, quantitative phase imaging, and tomography, as well as other imaging techniques.

[0201] The digital aberration correction system according to embodiments of the present invention can be used in microscopes for conventional, fluorescence, two-photon, multiphoton, and nonlinear techniques.

[0202] The digital aberration correction system according to embodiments of the present invention can be used in place of a conventional adaptive optics system, or added to an existing adaptive optics system as a “last mile” fine correction.

[0203] The digital aberration correction system according to embodiments of the present invention can be used in remote monitoring and imaging systems, such as airborne, spaceborne, marine, terrestrial, and space telescopes and imaging systems.

[0204] By employing various embodiments of the present invention, such imaging systems can use simplified, lightweight imaging optics.

[0205] By employing various embodiments of the present invention, such imaging systems can provide a wider tolerance to environmental factors (such as temperature, pressure, shock, and vibration) that affect the imaging system.

[0206] Using various embodiments of the present invention, such imaging systems can use digital adaptive optics to correct for atmospheric turbulence and imaging system variability under environmental factors.

[0207] Using various embodiments of the present invention, such imaging systems can use digital aberration correction instead of conventional adaptive optics systems, or as “last-mile” fine correction when added to existing systems.

[0208] The digital aberration correction system according to embodiments of the present invention can be used in imaging systems for applications requiring RGB color, multispectral, high spectral sensitivity, and without a color filter array to improve sensitivity.

Claims

1. An optical system, characterized in that... The optical system includes: An optical imaging unit for forming an optical image near an image plane of an optical system; A wavefront imaging sensor unit is located near the image plane. The wavefront imaging sensor unit includes a light modulator unit and an image sensor unit. The image sensor unit is located downstream of the light modulator unit relative to the propagation direction of the input light field through the system and includes the original number of pixels. N R It is used to provide raw digital data of a light field and image output near the image plane; and A control unit is configured to process the raw digital data to provide an output image, the output image being at least one of the following: a full-color RGB image and a hyperspectral image; The control unit includes a storage unit and a processing unit. The storage unit stores instructions, and the processing unit executes the instructions to receive the raw digital data of the light field illuminating the wavefront imaging sensor, and processes the raw digital data to determine the color field characteristics based on the analysis of the optical mutual coherence function on the image plane, thereby reconstructing a wavefront imaging sensor with… N O Each output image pixel and N C The output image of each color field characteristic, where N O x N C ≤ N R .

2. The optical system as described in claim 1, characterized in that... The control unit is also used for: Calculate the field characteristics of the light field; Based on the coherence and superposition information in the field characteristics, point sources in the image output are identified. For each identified point source, estimate its degree of ambiguity; and The deblurred image is reconstructed from a deblurred point source.

3. The optical system as described in claim 2, characterized in that... The field property is a Wigner distribution or an equivalent entity related to the Wigner distribution through mathematical transformation.

4. The optical system as claimed in claim 1, characterized in that... : The optical modulator unit includes multiple unit lattices, and the image sensor unit includes a sensor unit array; The sensor unit array defines a plurality of sensor subarray units, each sensor subarray corresponding to one of the plurality of unit lattices of the optical modulator unit; and The optical modulator unit is used to premodulate the input light collected by the image sensor unit. Each unit lattice of the optical modulator unit guides a portion of the input light collected on it to its corresponding sensor subarray unit and one or more adjacent sensor subarray units in a predetermined neighborhood.

5. The optical system as described in claim 4, characterized in that... The original number of pixels of the plurality of sensor subarray units of the image sensor unit. N R The number of Nyquist sampling points equal to or greater than that of the optical modulator unit N N .

6. The optical system as claimed in claim 4, characterized in that... The original number of pixels of the plurality of sensor subarray units of the image sensor unit. N R and the number of Nyquist sampling points of the optical modulator unit N N Follow the following relationship: N R ≥ N N + , where 1 ≤ ≤ N N And among them This indicates the spatial variability of dynamic aberrations.

7. The optical system as claimed in claim 1, characterized in that... The optical system is configured to provide at least one image from the group consisting of: a full-color RGB image, a hyperspectral image without using a spectral filter or a color filter.

8. The optical system as claimed in claim 2, characterized in that... The light field includes multiple wavelengths, either different or consecutive, and the control unit performs one or both of the following operations: (1) estimating the spectral distribution of each identified point source; (2) reconstructing the spectral distribution map of the image.

9. The optical system as claimed in claim 1, characterized in that... The control unit also performs one or more of the following operations: (1) estimating the aberration intensity of each identified point source; (2) estimating the depth based on the dynamic aberration intensity estimated for each identified point source, generating a spatial map of photometric aberration intensity, and reconstructing a depth map based on the spatial map of photometric aberration intensity; (3) restoring the diffraction-limited imaging resolution of the defocused portion of the image based on the depth map.

10. The optical system as claimed in claim 1, characterized in that... The control unit further estimates the depth at each point in the image plane.

11. A method for digital optical aberration correction of an image formed by an imaging unit near an image plane of an optical system, characterized in that... The method includes: It provides raw digital data of a light field illuminating a wavefront imaging sensor unit located near the image plane and an image output formed near the image plane, the raw digital data including the raw pixel count. N R ; The raw digital data is processed by a control unit to determine color field characteristics based on the analysis of optical mutual coherence functions on the image plane; and Reconstruction with N O Each output image pixel and N C The output image of each color field characteristic, where N O x N C ≤ N R ; The output image is at least one of the following: a full-color RGB image and a hyperspectral image.

12. The method as described in claim 11, characterized in that... The wavefront imaging sensor unit includes an optical modulator unit located near the image plane and an image sensor unit located downstream of the optical modulator unit relative to the propagation direction of the input light field through the system, for acquiring raw digital image output, and the method further includes using at least one of phase modulation and amplitude modulation to modulate the light field.

13. The method as described in claim 12, characterized in that... The optical modulator unit includes multiple unit lattices, and the image sensor unit includes a sensor unit array; the sensor unit array defines multiple sensor subarray units, each sensor subarray corresponding to one unit lattice of the multiple unit lattices of the optical modulator unit; and the method further includes: The input light collected by the image sensor unit is pre-modulated by the optical modulator unit; A portion of the collected input light incident thereon is guided through each unit lattice of the optical modulator unit to its corresponding sensor subarray unit and one or more adjacent sensor subarray units in a predetermined neighborhood.

14. The method as described in claim 13, characterized in that... The original number of pixels of the plurality of sensor subarray units of the image sensor unit. N R and the number of Nyquist sampling points of the optical modulator unit N N Follow the following relationship: N R ≥ N N + , where 1 ≤ ≤ N N And among them This indicates the spatial variability of dynamic aberrations.

15. The method as described in claim 12, characterized in that... The method further includes: Calculate the field characteristics of the light field; Based on the coherence and superposition information in the field characteristics, point sources in the image output are identified. For each identified point source, estimate its degree of ambiguity; The deblurred image is reconstructed from a deblurred point source.

Images