Array of light sources, Fourier-tycographic imaging system, and method of performing Fourier-tycography

Abstract

The present disclosure relates to an array of light sources (2) for a Fourier ptychography imaging system (100), preferably with different characteristics of the emitted light rays, and comprising at least two adjacent light sources (2) configured to illuminate a sample (4) at different angles of incidence in order to reconstruct a single image of the sample (4) using Fourier ptychography, wherein the characteristics of the light rays of each light source (2) are configured to match the sample (4) in order to increase the contrast and / or the color or the amount of substance information in the reconstructed single image of the sample (4). In an embodiment, one or more light sources are UV light sources. In an embodiment, the sample is a biological sample. In an embodiment, the FPM comprises a UV-transparent component and / or substance such as quartz.

Description

Technical Field

[0001] The present disclosure relates to an array of light sources (such as ultraviolet light, i.e., "UV") for a Fourier ptychography imaging system, preferably a Fourier ptychography imaging system including one or more UV light sources, and a method of performing Fourier ptychography using an ultraviolet (UV)-based system, preferably for imaging a biological sample without staining.

Background Art

[0002] Fourier ptychography microscopy (FPM) is a computational microscopy technique that generates high-resolution images with a large field of view (FoV). This is achieved by capturing a sequence of images of the region of interest of the sample under various illumination angles greater than the numerical aperture (NA) of the objective lens used in bright-field mode and / or dark-field mode, and repeatedly combining these images in the Fourier space to obtain a single high-resolution image.

[0003] These images are obtained by an FPM microscope that optically corrects and assumes that the image of the sample can be obtained with relatively good quality without much distortion or chromatic aberration.

[0004] Typically, a Fourier ptychographic image is captured by a monochrome camera and color illumination, preferably a narrow-band light-emitting diode (LED), with a band-pass filter of a selected wavelength added to a light source such as white light or a light source with a broader spectrum. As the illuminator or light source, a flat array or hemispherical setup of LEDs is used. Also, some systems have a narrow-band band-pass filter to further narrow the bandwidth of light in a specific image. The illumination wavelength is selected by an LED used in combination with or combined with a band-pass filter having a full width at half maximum (FWHM) of 10, 50, or 80 nm, for example, in front of the monochrome camera. A color image is composed from separate sets of images acquired at different wavelengths, which are then separately reconstructed as (quasi) monochrome high-resolution images. Subsequently, if necessary, a color image is formed by combining these high-resolution images.

[0005] Microscopic samples such as biological and medical samples usually contain different substances or tissues, but the problem is that for different substances in each sample area, the substance contrast is still relatively or quite low at a given or specific wavelength. As a result, in the final reconstructed image using Fourier ptychography, changes in the substance or substance composition of the sample are not detected or clearly identified.

Summary of the Invention

Problems to be Solved by the Invention

[0006] Therefore, there is still a continuing need to provide a solution for Fourier ptychography that improves the contrast of the image and / or optimizes the identification of substances in the reconstructed single image of the sample.

Means for Solving the Problems

[0007] This object is uniquely achieved by the subject matter of the independent claims. Other advantageous developments can be obtained from the dependent claims.

[0008] In an embodiment, according to a first aspect, the present disclosure provides an array of light sources for a Fourier ptychography imaging system, including at least two adjacent light sources whose emitted light rays have different characteristics from each other and are configured to illuminate a sample at different incident angles in order to reconstruct a single image of the sample using Fourier ptychography. Here, the characteristics of the light rays of each light source are configured to match the sample in order to increase the contrast and / or color or material information content in the reconstructed single image of the sample, thereby providing an array of light sources. In an embodiment, one or more light sources are predetermined to provide UV light.

[0009] In an embodiment, the present disclosure utilizes the difference in the characteristics of light rays from different light sources with respect to spectral components when acquiring images of an image set in order to improve and optimize the material contrast between the entire image set, i.e., between different sub-images, and better detect and distinguish changes in the material or material composition of the sample.

[0010] According to the array of light sources according to the present invention, there is no need to increase the number of sub-images in a single image set. Rather, by increasing the absorption contrast or optimizing the sensitivity to diffraction efficiency or refractive index changes, material discrimination is improved and optimized, and the signal is optimized for a single image in a given illumination direction or a plurality of overlapping directions.

[0011] In an embodiment of the array of light sources, the characteristics of the light rays include the wavelength, bandwidth, spectral profile, and / or polarization of the light rays.

[0012] In an embodiment, the wavelength of the light rays emitted from the light source can range from UV (including DUV < 200 nm) through VIS to IR (including SWIR and NIR up to 2500 nm). The wavelength of the light rays is limited only by the camera technology and optical system for generating a proper image.

[0013] In an embodiment, it is advantageous to mix light sources in which the wavelength, bandwidth / spectrum, and / or polarization of the light rays generated for a single reconstruction are different from each other.

[0014] For example, according to a method using a multi-wavelength light source, it is possible to generate an image with optimized or adjusted contrast even for a sample that is difficult to image at a single given wavelength.

[0015] Also, in some embodiments, a filter is used that can shape the intensity distribution of an LED into a completely arbitrary desired shape, such as a multi-band for a single LED.

[0016] In another embodiment of the light source array, the contrast in a single reconstructed image of the sample is determined by the interaction characteristics of the light rays of each light source with a given position of the sample.

[0017] Embodiments of the present disclosure can utilize the fact that, for example, although single images acquired at different wavelengths have the same physical position and configuration structure on the image sensor, they can have different contrasts depending on the respective interaction characteristics of the light of each wavelength with a specific position of the sample.

[0018] In another preferred embodiment of the light source array, the interaction characteristics include absorption, refraction, scattering, and / or diffraction of the light rays of each light source at a given position of the sample.

[0019] In another preferred embodiment of the light source array, the interaction characteristics are determined by the characteristics of the light rays of each light source and the substance and / or shape at a given position of the sample.

[0020] In another preferred embodiment of the array of light sources, the array of light sources includes a plurality of light sources such as at least three, namely a first light source, a second light source, and a third light source, where the first light source is adjacent to the second light source, the second light source is further adjacent to the third light source, where the first and second light sources have different characteristics of the emitted light rays, where the second and third light sources have different characteristics of the emitted light rays, and where the first and third light sources include the same characteristics of the emitted light rays.

[0021] In the FPM system, it is assumed that adjacent light sources in the light source array have an overlap of at least 35% in the aperture or Fourier space. This overlap is necessary to properly stitch together image contents at different illumination angles.

[0022] In an embodiment, adjacent light sources can have different wavelengths. However, the third light source, which is a more distant light source (e.g., the next light source in a sequence), has the same wavelength as the first light source. In an embodiment, one or more light sources can include one or more wavelengths characterized as ultraviolet.

[0023] The second light source, which is the "central" light source, helps to ensure an image reconstruction algorithm that properly stitches together two adjacent light sources of the same type for proper alignment of contributions in the Fourier space.

[0024] This assignment of different wavelengths to the light sources is due to the fact that there is a direct physical reconstruction by Fourier transform of the color-corrected image, while omitting the separate conversion to a Cartesian coordinate system as commonly used in Fourier tycography.

[0025] In another preferred embodiment of the array of light sources, in the reconstruction process using Fourier tycography, the image obtained by the second light source includes a smaller weighting factor than the images obtained by the first and third light sources.

[0026] The information of the "central" light source is adapted or used only as a smaller weighting factor or as intermediate when splicing together information of light sources with the same characteristics such as wavelength, bandwidth, polarization, etc. in order to homogenize the information in the iterative reconstruction cycle.

[0027] This method is considered to be similar to the monochromatic reconstruction image of the same type as the normal reconstruction. However, the number of secondary images used for different wavelengths is substantially reduced.

[0028] This method of alternating light sources of different wavelengths can preferably be used for a small number of wavelengths, such as two or three different wavelengths arranged in a plane or 3D-shaped rectangle, hexagon, or circle, with the distance between the light sources from the center to the high aperture being equal or varying.

[0029] In another preferred embodiment of the light source array, in the reconstruction process using Fouriertychography, the image obtained by the second light source includes the same weighting factor as the images obtained by the first and third light sources.

[0030] Therefore, the "central" light source is weighted in the same way as the other light sources, regardless of the individual colors. In this case, the information amount of each of all the images contributes to the final composite image with the same weight, regardless of the wavelength used.

[0031] According to this method, a general resolution in which different wavelengths contribute to the final image is achieved. This method is applicable regardless of whether the image is obtained by a monochrome sensor or a color image generated from a plurality of monochrome images with different acquisition or illumination settings by a color sensor.

[0032] In another preferred embodiment of the array of light sources, the array of light sources includes a plurality of light sources, particularly four or more light sources, preferably six or more light sources, wherein the plurality of light sources form at least two clusters, wherein the light sources within each cluster are adjacent to each other, and wherein the characteristics of the light rays of the light sources in each cluster are configured to match a given position of the sample so as to increase the contrast in the reconstructed single image of the given position of the sample.

[0033] Thereby, light sources of different spectral components (such as wavelength, bandwidth, and polarization) are arranged as pseudo (local) clusters for each type of light source. For example, the light source at the center of the bright-field imaging unit is preferably selected according to the sample characteristics because it is adopted in accordance with the absorption contrast. On the other hand, the dark-field light source is adopted in accordance with the angular scattering characteristics of the sample regarding refraction, dispersion, and / or diffraction, which may prevent absorption due to the lengthening of the optical path in the sample, so the signal attenuates along its propagation path.

[0034] In another preferred embodiment of the array of light sources, the array of light sources is configured to be selected based on the type of sample.

[0035] For example, in the case of non-stained biological samples such as body fluids, blood cells, blood smear specimens, tissues, and tissue sections, the absorption contrast is quite weak in the visible spectrum. Therefore, the bright-field imaging light source is selected from the non-visible spectral region such as UV or IR. In the UV region, there are different absorption mechanisms based on the carbon bonds of organic compounds, and in the region of 200 - 380 nm, particularly in the region of 250 - 320 nm, the absorption contrast can be made very strong. This can be easily realized by using commercially available LEDs. The light source is preferably configured to emit light in the region of 200 - 380 nm, most preferably in the region of 250 - 320 nm.

[0036] In an embodiment, according to a second aspect, the present disclosure provides a Fourier tycography imaging system including the array of light sources according to the first aspect.

[0037] In an embodiment of a Fourier ptychography imaging system, the Fourier ptychography imaging system includes a bandpass filter, a notch filter, a filter defining a complex spectral profile, and / or an edge filter disposed between an array of light sources and a camera sensor of the Fourier ptychography imaging system. This filter is preferably inserted into the imaging beam path between the objective lens and the camera.

[0038] Thereby, the spectral characteristics and / or substance discrimination in (absorption) contrast are improved. Dark field illumination is considered to consist of different (e.g., longer) wavelengths in, for example, the VIS or IR region. This is because the light rays in these regions have the potential to be scattered more efficiently based on the constituent size of the test sample.

[0039] In another preferred embodiment of a Fourier ptychography imaging system, the Fourier ptychography imaging system includes an objective lens, where the objective aperture of the objective lens is divisible into a plurality of segments, and each segment of the objective aperture is associated with a light source or a cluster of light sources in the array of light sources.

[0040] To further utilize local clustering of light sources, this illumination system is structured such that the objective aperture range or the entire objective aperture in the bright field and / or dark field illumination section is divided into a plurality of segments / regions, such as two, three, four, six, or eight segments, and different light sources or different combinations of light sources (with different wavelengths, bandwidths, and / or polarizations) used in this illumination system are provided for each segment.

[0041] As described above in a preferred embodiment of the array of light sources, each light source can have the closest different types of adjacent light sources. However, light sources of the same type are preferably arranged such that a larger segment / portion of the illumination and / or imaging aperture space is filled for a given combination of this particular light source type or light source types.

[0042] From the high degree of symmetry of the Fourier space, for sectors at point-symmetric positions, completely different types of light sources can be used to optimize the sampling efficiency of the sample in the imaging and reconstruction processes. For example, when the light source has 4, 6, or 8 segments, for each segment, there are possibilities of selecting 2, 2 or 3, or 2 or 4 different light sources. These combinations are selected in a given sequence for adjacent segments. For example, in the case of 6 segments and 2 types of light sources, there are 3 segments for each type, and adjacent segments have different types of light sources. As a result, a setup is obtained in which no segment faces another segment of the same type, showing a wider range in the aperture space for one type / set of wavelengths without losing information about even minor details of the objective lens. Regarding the homogeneity of sampling for a given light source type, further optimization is achieved when the radially adjacent light sources are also configured with different types to sample the Fourier space with an optimized sampling method.

[0043] According to a third aspect, the present invention provides a method of performing Fouriertychography, comprising: illuminating a sample at different incident angles with at least two adjacent light sources in an array of light sources, wherein the at least two adjacent light sources have different characteristics of the emitted light rays from each other, and the characteristics of the light rays of each of the at least two light sources are adapted to the sample to enhance the contrast in a single image of the sample to be reconstructed; capturing an image of the sample by a camera with the light rays from the at least two adjacent light sources; and reconstructing a single image of the sample by using the acquired image.

[0044] In a preferred embodiment of this method, the characteristics of the light rays include the wavelength, bandwidth, and / or polarization of the light rays.

[0045] Hereinafter, the present invention will be described in more detail with reference to the exemplary embodiments described in the accompanying drawings. The accompanying drawings are for enabling a deep understanding of the present invention, and are incorporated herein and constitute a part of this specification. These drawings show embodiments of the present invention and, together with this specification, help to explain the principles of the present invention.

[0046] Many other embodiments of the present invention and the intended advantages of the present invention will be readily recognized as a deep understanding can be obtained by reference to the following detailed description. The same reference numbers refer to corresponding similar members. The drawings are not drawn to scale.

Brief Description of the Drawings

[0047]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

DETAILED DESCRIPTION OF THE INVENTION

[0048] Figure 1a is a perspective view showing the setup of a Fourier ptychography imaging system 100 including an array of light sources 2 placed on a carrier board 3 according to an embodiment of the present invention. As shown in the figure, the Fourier ptychography imaging system 100 further includes, from bottom to top, a sample 4 placed on a sample surface 5, an objective lens 6, a tube lens 10, and a camera 12 for capturing an image of the sample 4. As shown in Figure 1a, the region of interest 7 of the sample 4 is illuminated by an array of light sources 2 that achieve illumination at a specific illumination angle by continuous blinking. The hollow rectangle 8 represents the rear focal plane of the objective lens 6 and / or corresponds to the aperture plane of the objective lens 6.

[0049] In the embodiment shown in Figure 1a, the array of light sources 2 includes a spatial distribution of 15×15 = 225 separate low-aperture LEDs. The exact number and arrangement of the LEDs are determined by the desired resolution of the reconstructed image. In other embodiments not shown, laser-based or fiber-based light sources are possible.

[0050] As shown in Figure 1a, the LEDs 2 are arranged in a grid pattern with equal distances in the x and y directions on a plane. In other embodiments not shown, the LEDs are arranged in a grid pattern with equal distances only in the x direction or the y direction on a plane. In another embodiment, the LEDs 2 are arranged in a grid pattern on a plane such that the effective angular spacing as seen from the objective lens is regular, i.e., has an equal-distance shape, with the spacing between adjacent light sources changing.

[0051] In another embodiment, the light sources 2 are arranged on a three-dimensional structure such as a hemisphere, and each light source has the same effective geometric distance from the center of the field of view of the optical system. In a preferred embodiment, the optical axis of each LED points to the center of the FoV of the optical system in the optimal focal plane setting. Generally, any other spatial distribution of the effective light sources is possible. A specific design is calibrated to establish proper image reconstruction performance of the FPM system.

[0052] In some embodiments, at specific positions of the grating, it is preferable to install monochromatic or polychromatic light sources, and their intensities are individually controlled over time. For example, they can be switched on or off, and appropriate output levels that are fixed, pre-calibrated, preferably linearized, for the adaptation settings are set.

[0053] Conventionally, the same type of light source is provided at all grating positions. This design is essentially affected by the requirements of the reconstruction algorithm for the (pseudo) monochromatic image set. However, at all possible positions of the illumination grating, not all wavelengths are equally useful. For example, for the bright field region of the illumination system, wavelengths adapted to the strong absorption bands of the sample are preferred, while for the dark field light source, it is considered that the wavelength region where the scattering cross-section of the sample is large is adapted.

[0054] In an embodiment, different light sources, for example, those with different wavelengths and / or bandwidths and which are already known, are arranged at different positions of the grating, and it is not necessary to have nominally the same light source at all grating positions.

[0055] For example, a single image acquired at different wavelengths, although having the same physical position on the sample and thus the same constituent structure on the image sensor of camera 12, can have different contrasts according to the respective interaction characteristics of the light of each wavelength with a specific position of sample 4. The interaction characteristics include the absorption, refraction, scattering, diffraction, and / or fluorescence characteristics of the light rays of each light source 2 at a given position 7 of sample 4. The interaction characteristics are determined by the characteristics of the light rays of each light source, such as the wavelength, bandwidth, and / or polarization of the light rays, and are also determined by the material and / or shape at a given position 7 of sample 4.

[0056] According to the array of light sources 2, it is not necessary to increase the number of sub-images of the single image set. Rather, by, for example, increasing the absorption contrast or optimizing the diffraction efficiency or sensitivity to refractive changes, substance identification is improved and optimized, and the signal is optimized for a single image in a given illumination direction or multiple overlapping directions.

[0057] FIG. 1b shows the Fourier spectrum of sample 4 at the rear focal plane of the objective lens, related to the setup of the Fourier tycography imaging system shown in FIG. 1a. The circular sub-region 20 corresponds to the aperture size of the objective lens 6 and shifts with the shift of the illumination angle.

[0058] The light irradiation field emitted from a certain LED2 is approximated as a plane wave within a small area on the sample surface. This is because when the distance between the LED2 and the sample 4 is large, the spatial coherence of the LED2 increases. The plane wave has vector components (kx, ky) associated with the oblique illumination of the LED2 on the sample 4. When a certain LED2 is lit, the Fourier spectrum of the sample shifts by (kx, ky) at the rear focal plane of the objective lens. Therefore, the finite numerical aperture (NA) of the objective lens acts as a low-pass filter that transmits only a small sub-region at the center of the shifted Fourier spectrum. The Fourier components that have passed through the low-pass filter further propagate to the image plane and are captured by the camera 12. The LED2s are sequentially lit so that the captured images include Fourier sub-regions that partially overlap, and together they cover the entire Fourier spectrum of the sample 4.

[0059] In FIG. 1c, a partial enlarged view of the LED array on the carrier board 3 is shown. The parallel arrows and parallel broken lines shown in FIG. 1c represent the sequential switching of the LED2s at different positions on the matrix to achieve a shift in the illumination angle. In the state shown in FIG. 1c, the LED2i is switched on to provide illumination.

[0060] FIG. 2 shows an illumination device 14 including a base unit 16 and a hemispherical carrier 18 fixed on the base unit 16. A hemispherical set-up of LEDs (not shown) is provided on the hemispherical carrier 18, and each LED has the same effective geometric distance from the center of the field of view of the optical system. Due to the hemispherical set-up, the optical axis of each LED is oriented towards the axis of symmetry of the hemisphere, and all the optical axes intersect at the nominal focal position of the optical system, thereby improving the illumination uniformity with respect to the FoV.

[0061] FIG. 3 shows an array of light sources on the sample surface 5 according to another embodiment of the present invention. In this embodiment, 9×9 = 81 LEDs 2 arranged in a planar shape are employed to form a rectangular illuminator.

[0062] FIG. 4 is a diagram showing an array of light sources on the sample surface 5 according to another embodiment of the present invention. In this embodiment, a total of 76 LEDs 2 arranged in a three-dimensional shape are employed for forming a hemispherical illuminator. The LEDs 2 are at a certain radial distance from the intersection of the optical axis (+) and the sample surface 5. In FIG. 4, the angular increment of the rings formed by the LEDs is fixed, and for each ring, the LEDs have a "constant" angular interval. To avoid the harmonic effect, an individual azimuth offset is set for each ring.

[0063] An embodiment of a method of performing the Fouriertychography shown in FIG. 5 includes three steps. In step S10, the sample is illuminated at different incident angles by at least two adjacent light sources in the array of light sources, and the at least two adjacent light sources preferably have different characteristics of the emitted light rays, and the characteristics of the light rays of each of the at least two light sources are adapted to the sample in order to increase the contrast in the single image of the sample to be reconstructed. In step S20, an image of the sample is captured by a camera with light rays from at least two adjacent light sources. In step S30, a single image of the sample is reconstructed by using the acquired image. In the embodiment, additional steps are included such as preparing one or more biological samples such as body fluids, blood cells, blood smear specimens, tissues, and tissue sections. In the embodiment, one or more UV light waves for contacting the biological sample or specimen are preselected to improve the contrast of the derived image.

[0064] The present invention is not limited to microscopy and generally applies to Fouriertychography.

[0065] In some embodiments, the (local) clustering of light sources by wavelength is also used to further expand the NA according to the reconstruction method. When the wavelength becomes shorter for high NA illumination, or when there is an overlap with the normal wavelength of illumination for each set of images, the effective maximum NA becomes even larger for the reconstructed image, even when based on different wavelengths.

[0066] In an embodiment, in order to optimize the image quality of the images acquired for a single image set, it is considered preferable to perform color correction such as achromatization and apochromatization by selecting the wavelength of illumination corresponding to the design wavelength of the imaging optical system. If this is not possible, it is preferable to group illuminations with the same focus setting as a subgroup and optimize the setup so that one of the subgroups is used continuously. To further optimize the setup, it is preferable to order the subgroups such that the change from one subgroup to the next causes the focus position to change in the same direction, for example, by a positive or negative increment. This can minimize the movement, thus reducing any effect due to mechanical hysteresis. Alternatively, since the mechanical movement is minimized, the execution time of the image capture process can be minimized.

[0067] In an embodiment, using the overlap of image data in the Fourier space at different wavelengths based on the color-corrected image on the image plane enables the generation of a separate composite image for one or more of the wavelengths used for data capture by using one complete image set / a given number of raw images. A form of image synthesis with reduced weighting or proper stitching and image synthesis only for alignment in stitching is provided for image information farther away or with significantly reduced overlap for the same wavelength compared to standard reconstruction methods.

[0068] Image capture is based on different wavelengths that are employed when re - determining the focus of the optical system for each wavelength or bandwidth for optimal image contrast to compensate for chromatic (focus) aberration of the optical system. Re - determination of the focus can also mean re - scaling of the image. This is because when the focus position changes, the effective magnification also changes slightly. This dependence of magnification on the focus position and / or wavelength is determined, for example, by the use of a scale bar or length artifacts, or in a standard calibration routine that determines the system magnification as a relative magnification associated with an appropriate test structure (e.g., a binary pattern in a chromium - on - glass structure). These calibration data are used to re - scale the size of an image captured at a given wavelength prior to its application to each Fourier - transform holographic reconstruction.

[0069] In some embodiments, the present disclosure is an array of light sources (2) for a Fourier ptychography imaging system (100), wherein the characteristics of the emitted light rays are different from each other, and are configured to illuminate a sample (4) at different incident angles in order to reconstruct a single image of the sample (4) using Fourier ptychography, and includes at least two adjacent light sources (2), wherein the characteristics of the light rays of each light source (2) are configured to match the sample (4) in order to increase the contrast and / or color or material information content in the reconstructed single image of the sample (4). Regarding the array of light sources. In some embodiments, the characteristics of the light rays include the wavelength, bandwidth, spectral profile, and / or polarization of the light rays. In an embodiment, the characteristics of the light rays are preselected to include light rays having a wavelength characterized as ultraviolet. In an embodiment, the contrast in the reconstructed single image of the sample (4) is determined by the interaction characteristics of the light rays of each light source (2) with a given position of the sample (4). In an embodiment, the interaction characteristics include absorption, refraction, scattering, and / or diffraction of the light rays of each light source (2) at a given position of the sample (4). In an embodiment, the interaction characteristics are determined by the characteristics of the light rays of each light source (2) and the material and / or shape at a given position of the sample (4). In an embodiment, the array of light sources (2) includes at least three light sources (2) that are a first light source, a second light source, and a third light source, wherein the first light source is adjacent to the second light source, and the second light source is further adjacent to the third light source, wherein the first and second light sources have different characteristics of the emitted light rays from each other, wherein the second and third light sources have different characteristics of the emitted light rays from each other, and wherein the first and third light sources include the same characteristics of the emitted light rays. In an embodiment, in the reconstruction process using Fourier ptychography, the image obtained by the second light source includes a smaller weighting factor than the images obtained by the first and third light sources. In an embodiment, in the reconstruction process using Fourier ptychography, the image obtained by the second light source includes the same weighting factor as the images obtained by the first and third light sources.In an embodiment, the array of light sources (2) includes a plurality of light sources (2), in particular four or more light sources (2), preferably six or more light sources (2), wherein the plurality of light sources form at least two clusters, wherein the light sources within each cluster are adjacent to each other, and wherein the characteristics of the light rays of the light sources in each cluster are configured to match a given position of the sample (4) in order to increase the contrast in the reconstructed single image of the given position of the sample (4). In an embodiment, the array of light sources is configured to be selected based on the type of the sample (4).

[0070] In some embodiments, the present disclosure includes a Fourier ptychography imaging system (100) including an array of light sources as illustrated or described herein. In an embodiment, the Fourier ptychography imaging system of the present disclosure includes a band-pass filter, a notch filter, a filter defining a complex spectral profile, and / or an edge filter disposed between the array of light sources (2) and the camera sensor (12) of the Fourier ptychography imaging system (100). In an embodiment, the Fourier ptychography imaging system of the present disclosure includes an objective lens (6), wherein the objective aperture of the objective lens (6) is divisible into a plurality of segments, and wherein each segment of the objective aperture is associated with a light source (2) or a cluster of light sources (2) in the array of light sources (2).

[0071] In an embodiment, the Fourier ptychography imaging system (100) includes one or more UV-transparent components, such as a component including quartz or a component made of quartz. Non-limiting examples of the UV-transparent component include one or more of the light source (2), the carrier board (3), the sample surface (5), the objective lens (6), the region of interest (7), the hollow rectangle (8), the tube lens (10), the camera (12) or its UV-transparent component, the illumination device (14) or its UV-transparent component.

[0072] In some embodiments, the present disclosure relates to a biological sample embedded in a refractive index matching medium. The refractive index matching medium is preferably transparent in UV, such as silicone oil or the like.

[0073] In an embodiment, the present disclosure is a method of performing Fouriertychography, comprising: illuminating a sample at different incident angles with at least two adjacent light sources in an array of light sources (S10), wherein the at least two adjacent light sources have different characteristics of the emitted light rays, and the characteristics of the light rays of each of the at least two light sources are adapted to the sample to increase the contrast in a single image of the sample to be reconstructed; capturing an image of the sample with a camera by the light rays from the at least two adjacent light sources (S20); and reconstructing a single image of the sample by using the acquired image (S30). In an embodiment, the characteristics of the light rays include the wavelength, bandwidth, and / or polarization of the light rays. In an embodiment, illuminating the sample at different incident angles (S10) further includes illuminating with ultraviolet light. In an embodiment, illuminating the sample at different incident angles (S10) further includes providing one or more biological samples of the present disclosure.

[0074] In some embodiments, the present disclosure is an array of light sources (2) for a Fouriertychography imaging system (100), preferably including at least two adjacent light sources (2) configured to illuminate a biological sample (4) at different incident angles, wherein the characteristics of the emitted light rays are preferably different from each other, and the characteristics of the light rays of each light source (2) are configured to be adapted to the sample (4) to increase the contrast and / or color or material information content in the reconstructed single image of the sample (4), and wherein one or more light sources include ultraviolet light.

[0075] In some embodiments, the present disclosure relates to a Fouriertychography imaging system (100) including an array of light sources (2) according to the present disclosure, such as an array of UV light sources.

[0076] In some embodiments, the present disclosure is a method of performing Fourier ptychography, comprising: illuminating a biological sample at different angles of incidence with at least two adjacent light sources in an array of light sources, wherein the at least two adjacent light sources preferably have different characteristics of the emitted light rays, and the characteristics of the light rays of each of the at least two light sources are adapted to the sample in order to increase the contrast in a single image of the sample to be reconstructed, wherein at least one of the adjacent light sources is an ultraviolet light source (S10); capturing an image of the biological sample with a camera by light rays from the at least two adjacent light sources, wherein the biological sample is characterized as non-stained (S20); and reconstructing a single image of the sample by using the acquired image (S30).

[0077] It is to be understood that all the advantageous options and modified variations described and above in connection with the embodiments of the array of light sources according to the first aspect are equally applicable to the embodiments of the Fourier ptychography imaging system according to the second aspect and the embodiments of the method according to the third aspect, and vice versa.

Example

[0078] Currently, biological samples analyzed by microscope, such as blood smear specimens or tissue sections, need to be stained to visualize important information such as cell nuclei. This is a problem because it is a major obstacle in the path towards further automation, increases costs, and may vary greatly from laboratory to laboratory with respect to diagnostic results.

[0079] In an embodiment, the deficiencies of Fourier ptychography microscopy (FMP) are overcome by a microscopy technique based on UV-FPM that can directly image biological samples such as blood smear specimens and digital pathology samples without the need for staining. The embodiment includes a method of digitally staining these images to provide clinicians with images similar to those obtained for stained samples.

[0080] FPM is a novel computational microscopy technique that combines the ability to record high-resolution images with a wide field of view. This is achieved by illuminating the sample from multiple directions at both large and small angles, rather than by the numerical aperture of the objective lens used. Then, the obtained low-resolution images are seamlessly stitched together in the Fourier space by using an iterative algorithm based on the Gerchberg-Saxton algorithm. Conventionally, FPM has achieved great success in the visible region of the spectrum. However, in this spectral region, biological samples have almost no absorption. By shifting the wavelength from visible to UV (the natural absorption bands of nucleotides and proteins are approximately 260 nm and 280 nm, respectively), the absorption contrast of biological samples is significantly improved.

Example

[0081] UV-FPM for Imaging Biological Samples without Natural Absorption Contrast Fourier ptychographic microscopy (FPM) is a computational microscopy technique that generates high-resolution images with a large field of view (FoV). This is realized by capturing images of the sample at various illumination angles larger than the numerical aperture (NA) of the objective lens used and iteratively combining these images in the Fourier space to obtain a high-resolution image. The illumination angles are selected such that the images overlap by approximately 40% (such as 35% - 45%) in the Fourier space, but are determined by multiple parameters such as the objective lens or detector used. The resolution limit of the FPM microscope is determined by the sum of the microscope objective lens used, the maximum illumination angle (α max ), and the refractive index (n) of the medium:

Equation

[0082] Currently, illumination in Fourier ptychography microscopy is achieved by using a visible light LED array. For such arrays, many designs have been proposed, such as flat arrays, hemispherical designs, etc. However, generally, all of these follow the same design concept: namely, the LEDs are attached to some carrier board, which is placed at a specific distance from the sample. And by the continuous blinking of the LEDs, illumination under a specific illumination angle is realized. In an embodiment, such LEDs are arranged together with one or more light sources. The potential applications of this technology are extensive. From a diagnostic perspective, FPM can replace a standard optical microscope. For example, in hematology (especially, blood smear specimen analysis) or digital pathology. Such potential applications have attracted great interest not only in the industry but also in the academic community. For most diagnostic applications, the workflow can be summarized as follows. That is, a sample is acquired, fixed on a carrier (such as a microscopy slide), stained, and then imaged on an FPM microscope. Such a workflow is ideally automated for throughput improvement, cost reduction, and consistent result realization. Staining is performed to create the absorption contrast necessary to extract information from the microscope image. At the same time, staining leads to an increase in cost, is very time-consuming, and has significant differences in procedures from laboratory to laboratory, thus becoming a major obstacle. The present disclosure provides an FPM-based method by non-staining imaging. Due to the nature of image reconstruction based on the Gerchberg-Saxton iterative phase retrieval algorithm, FPM reconstruction leads not only to a high-resolution amplitude image but also to the calculation of qualitative phase differences. Some have proposed using this phase information to visualize objects without natural absorption contrast. However, the phase differences shown in today's FPM images are not sufficient to distinguish white blood cells (WBCs).

[0083] In an embodiment, the present disclosure includes a UV-based FPM system that can image a biological sample without the need for staining. Currently, samples are stained to generate the absorption contrast necessary to extract information from biological samples (e.g., identify white blood cell nuclei). FPM-based systems are based on visible light, which is a frequency at which most biological samples exhibit minimal absorption. As described above, this is a problem because it requires staining the sample to create the absorption contrast necessary for FPM imaging. Accordingly, the present disclosure proposes shifting the wavelength region used for FPM. Nucleic acids and proteins are well known to exhibit absorption at 260 nm and 280 nm, respectively (see Figure 6). For example, radiation at 260 nm is often used for DNA quantification. C=C double bonds also exhibit absorption in the UV, but the position of the absorption band can be expected to shift significantly because it is determined by the amount of electron delocalization. Referring now to Figure 6, Figure 6 shows the UV absorption of proteins and nucleic acids (from the World Wide Web: handling-solutions.eppendorf.com / sample-handling / photometry / applications / detailview-applications / news / uv-vis-spectrophotometry-easy-and-quick-quantification-of-nucleic-acids / ).

[0084] In addition to the natural absorption contrast, shortening the wavelength also has the additional benefit of increasing the resolution at a given numerical aperture (see Equation 1). This also means that a smaller NA sys is required to obtain the same resolution as visible light FPM. Conversely, this means that to achieve the target resolution, it is necessary to reduce the number of LEDs (with a smaller illumination angle), or it is also possible to use an objective lens with a smaller NA.

[0085] The following examples focus on the identification and classification of nuclei within blood smear specimens and white blood cells, but this concept can be easily extended to other use cases where the sample has no absorption contrast in the visible wavelength region. As another example, tissue sections that are stained to visualize cell nuclei and other components within the sample can be considered.

[0086] By shifting the wavelengths used for FPM imaging to this wavelength region, it becomes possible to image biological samples without the need for staining. Figures 7A and 7B show images of the same white blood cell measured once with visible light (approximately 450 nm) and UV light (280 nm), respectively. These images clearly show that the nucleus is only visualized in the case of imaging with UV light. Note that since these images are single images without FPM reconstruction, the resolution is quite low. Referring to Figures 7a and 7b here, these are magnified images of an unstained blood smear specimen on a quartz slide imaged once at 7(a) 600 nm and 7(b) 280 nm. These images were acquired with a single UV LED under the sample and a 10x / 0.2NA objective lens. There is a WBC in the center of the image. The increase in the absorption contrast of the nucleus is evident when imaging at 280 nm.

[0087] Referring to Figure 8 here, Figure 8 shows the FPM reconstruction of a white blood cell when imaging at 280 nm. A total of 121 images were acquired as various illumination images up to a maximum illumination angle of approximately 35°, and they were stitched together integrally in the Fourier space to generate a high-resolution image. Figure 8 shows the FPM reconstruction of an image of a blood smear specimen imaged at 280 nm.

[0088] The biological samples used in FPM are known to benefit from being embedded in a refractive index matching medium such as DPX mountant or microscope immersion oil and then sandwiched between two slides. In the case of UV FPM, these slides and mounting media preferably need to be UV transparent in the region of 250 nm to 320 nm. An example of such a medium is silicone oil. As an option to reduce the cost associated with using quartz as a sample carrier, an inverted FPM microscope can be employed. In this case, only a thin quartz coverslip for minimizing absorption is required. In such a scenario, the sample can be directly mounted on the coverslip without the need for a second slide.

[0089] The analysis of stained samples is performed by a pathologist. Therefore, it is preferable to present a color image to increase the throughput. Digital staining of the FPM image is used for generating a color FPM image. Such staining can be achieved by directly superimposing FPM images measured at multiple wavelengths with the white balance of the image adjusted, or by training an AI algorithm to stain the image. For any of the staining methods, it is desirable to acquire UV-FPM images at multiple (at least two) wavelengths. Also, it is desirable that at least one of the wavelengths used is in the non-absorbing region of the sample that acts as a contrast stain.

[0090] Also, the ability to image biological samples without the need for staining with a large FoV and high resolution opens the door for promoting the automation of the diagnostic workflow at low cost. However, it should be noted that a large FoV provided by FPM scanning of the sample is still required. Also, by seamless stitching of the images, it is possible to create an image of the entire sample with the ability to zoom in and zoom out mimicking an optical microscope. Furthermore, FPM enables digital re-focusing of the sample. Combined with the ability to perform a Z-stack, imaging such as tomography of the sample becomes possible.

[0091] As described above, in FPM, the most common form of LED array is used. After different wavelengths are used at different positions, it is also possible to pattern the light source so that it is reconstructed as a single image. Also, in UV-FPM, a polarization filter or a polarization-sensitive camera can be used. This minimizes the influence of background light scattering and leads to noise reduction. The concept of polarization FPM will be covered in more detail in the disclosure of a separate invention.

[0092] It should be noted that in order to change the wavelength to UV, some modification to a standard FPM microscope is required. Replace the LED (or any other light source used) with a corresponding UV light source. Since UV LEDs are readily available today, this is not a major problem. Also, all optical components and the camera need to be UV-compatible. This means, for example, changing all glass-based members to quartz. Similarly, the sample needs to be mounted on a quartz microscope slide.

[0093] Generally, the above concept is extended to all cases where the sample is observed in the UV rather than the visible light region (for example, digital pathology). Also, tissue sections are stained in the sample and in FPM to create absorption contrast.

[0094] A major difference from known solutions is the use of UV light-based FPM. Instead of using one or more stains to create absorption contrast that is not present in the sample, the naturally occurring absorption bands are utilized by changing the wavelength of the light. As a result: automation is facilitated, hands-on time is reduced, variability due to differences in staining procedures is eliminated, and comparison of results generated in different laboratories becomes easier.

[0095] In the above detailed description, for the purpose of simplifying the present disclosure, various configurations are integrally grouped as one or more examples. It is to be understood that the above description is intended to be illustrative and in no way limiting. It is also intended to cover modifications, improvements, and equivalents. Many other examples will be apparent to those skilled in the art upon reviewing this specification.

Explanation of Signs

[0096] 2 Light source 3 Carrier board 4 Sample 5 Sample surface 6 Objective lens 7 Region of interest 8 Hollow rectangle 10 Tube lens 12 Camera 14 Lighting device 16 Base unit 18 Hemispherical carrier 20 Circular sub-region 100 Fouriertychography imaging system S10 - S30 Steps of the method

Claims

1. An array of light sources (2) for a Fouriertychography imaging system (100), preferably with different characteristics of the emitted light rays, and configured to illuminate a sample (4) at different angles of incidence in order to reconstruct a single image of the sample (4) using Fouriertychography, wherein the characteristics of the light rays of each light source (2) are configured to match the sample (4) in order to increase the contrast and / or color or amount of substance information in the reconstructed single image of the sample (4). The array of said light sources.

2. The array of light sources according to claim 1, wherein the characteristics of the light rays include the wavelength, bandwidth, spectral profile, and / or polarization of the light rays.

3. The array of light sources according to claim 1 or 2, wherein the contrast in the reconstructed single image of the sample (4) is determined by the interaction characteristics of the light rays of each light source (2) with a given position of the sample (4).

4. The array of light sources according to claim 3, wherein the interaction characteristics include absorption, refraction, scattering, diffraction, and / or fluorescence of the light rays of each light source (2) at a given position of the sample (4).

5. The array of light sources according to claim 3 or 4, wherein the interaction characteristics are determined by the characteristics of the light rays of each light source (2) and the substance and / or shape at a given position of the sample (4).

6. The array of light sources (2) includes at least three light sources (2), namely a first light source, a second light source, and a third light source, wherein the first light source is adjacent to the second light source, and the second light source is further adjacent to the third light source, wherein the first and second light sources have different characteristics of the emitted light rays, and wherein the second and third light sources have different characteristics of the emitted light rays, and wherein the first and third light sources include the same characteristics of the emitted light rays. The array of light sources according to any one of claims 1 to 5.

7. In the reconstruction process using Fouriertychography, the image obtained by the second light source includes a weighting factor smaller than the images obtained by the first and third light sources. The array of light sources according to claim 6.

8. In the reconstruction process using Fouriertychography, the image obtained by the second light source includes the same weighting factor as the images obtained by the first and third light sources. The array of light sources according to claim 6.

9. The array of light sources (2) includes a plurality of light sources (2), in particular four or more light sources (2), preferably six or more light sources (2), wherein the plurality of light sources form at least two clusters, wherein the light sources within each cluster are adjacent to each other, and wherein the characteristics of the light rays of the light sources in each cluster are configured to match a given position of the sample (4) in order to increase the contrast in the reconstructed single image of the sample (4). The array of light sources according to any one of claims 1 to 8.

10. The array of light sources is configured to be selected based on the type of the sample (4). The array of light sources according to any one of claims 1 to 9.

11. A Fourier-tycography imaging system (100) including the array of light sources (2) according to any one of claims 1 to 10.

12. A Fourier-tycography imaging system (100) according to claim 11, including a band-pass filter, a notch filter, a filter defining a complex spectral profile, and / or an edge filter disposed between the array of light sources (2) and the camera sensor (12) of the Fourier-tycography imaging system (100).

13. The Fourier-tycography imaging system (100) includes an objective lens (6), wherein the objective aperture of the objective lens (6) can be divided into a plurality of segments, and wherein each segment of the objective aperture is associated with a light source (2) or a cluster of light sources (2) in the array of light sources (2). The Fourier-tycography imaging system (100) according to claim 11 or 12.

14. A method of performing Fourier-tycography, comprising: Illuminating the sample at different incident angles by at least two adjacent light sources in the array of light sources, wherein the at least two adjacent light sources preferably have different characteristics of the emitted light rays, and the characteristics of the light rays of each of the at least two light sources are adapted to the sample in order to increase the contrast in the single image of the sample to be reconstructed (S10); Capturing an image of the sample by a camera with light rays from at least two adjacent light sources (S20); Reconstructing a single image of the sample by using the acquired image (S30), The method as described above.

15. The method according to claim 14, wherein the characteristics of the light beam include the wavelength, bandwidth, and / or polarization of the light beam.

Images