Apparatus and methods employing interferometric scattering (iSCAT) microscopy with illumination light having tailored spatial coherence

Abstract

An interferometric scattering microscope (iSCAT) apparatus (100) for imaging a sample to obtain an image based on iSCAT includes an optical imaging system having an illumination device (10), an optical relay device (20) including a beam splitter (21), a sample container (30), and a detector device (40). The illumination device (10) includes a laser source device (11) arranged to generate illumination light (2). The optical relay device (20) is arranged between the laser device (11) and the sample container (30) for relaying the illumination light (2) to the sample container (30). A beam splitting device (21) is arranged to deflect a first portion (2A) of the illumination light (2) toward the sample container (30), to deflect a second portion (2B) of the illumination light (2) toward the detector device (40), and to superimpose the scattered light (2C) scattered at the sample (1) disposed at the sample container (30) with the second portion (2B) of the illumination light (2). The detector device (40) is arranged to receive the superimposed scattered light (2C) and the second portion (2B) of the illumination light (2) in the image plane of the optical imaging system. A coherence setting device (50) is arranged to selectively set the point spread function of the optical imaging system in the image plane by applying and controlling the degree of spatial coherence of the illumination light (2) output by the laser source device (11). Furthermore, an iSCAT microscopy method is described, which includes imaging a sample (1) based on iSCAT, particularly a sample comprising biological particles such as protein particles and / or protein molecules, wherein an iSCAT device (100) is employed.

Description

Technical Field

[0001] This invention relates to an interferometric scattering microscope (iSCAT) apparatus configured for imaging a sample under study based on iSCAT. The interferometric scattering microscope includes an illumination device, a beam splitting device, a sample container, and a detector device. Furthermore, this invention relates to a method for imaging a sample under study based on iSCAT. Applications of this invention include, for example, the fields of microscopy imaging and the sensing of sample particles and / or sample structures (such as particles in biological samples, such as biological cells or cellular components), as well as the fields of materials science and / or semiconductor technology. Background Technology

[0002] In this specification, the following prior art is referenced to illustrate the technical background of the present invention: [1]K. Lindfors, T. Kalkbrenner, P. Stoller, and V. Sandoghdar, "Detection and Spectroscopy of Gold Nanoparticles Using Supercontinuum WhiteLight Confocal Microscopy" Phys. Rev. Lett. 93, 037401 (2004); [2]RW Taylor and V. Sandoghdar, “Interferometric ScatteringMicroscopy: Seeing Single Nanoparticles and Molecules via RayleighScattering” Nano Lett. 19, 4827–4835 (2019); [3]Y.-H. Lin, W.-L. Chang, and C.-L. Hsieh, “Shot-noise limited localization of single 20 nm gold particles with nanometer spatial precisionwithin microseconds” Opt. Express 22, 9159–9170 (2014); [4]H. Ewers, V. Jacobsen, E. Klotzsch, et al., “Label-free opticaldetection and tracking of single virions bound to their receptors insupported membrane bilayers” Nano Lett. 7, 2263–2266 (2007); [5]P. Kukura, H. Ewers, C. Müller, et al., “High-speed nanoscopictracking of the position and orientation of a single virus” Nat. Methods 6,923–927 (2009); [6]R. F. Garmann, A. M. Goldfain, C. R. Tanimoto, et al., “Single-particle studies of the ef-fects of RNA–protein interactions on the self-assembly of RNA virus particles” Proc. Natl. Acad. Sci. U.S.A. 119,e2206292119 (2022); [7]Y.-F. Huang, G.-Y. Zhuo, C.-Y. Chou, et al., “Coherent brightfieldmicroscopy provides the spatiotemporal resolution to study early stage viralinfection in live cells” ACS Nano 11, 2575–2585 (2017); [8]A. D. Kashkanova, M. Blessing, A. Gemeinhardt, et al., “Precisionsize and refractive in-dex analysis of weakly scattering nanoparticles inpolydispersions” Nat. Methods 19, 586–593 (2022); [9]A. D. Kashkanova, M. Blessing, M. Reischke, et al., “Label-freediscrimination of extracel-lular vesicles from large lipoproteins” J.Extracell. Vesicles 12, 12348 (2023);

[10] J. Ortega Arroyo, J. Andrecka, K. Spillane, et al., “Label-free,all-optical detection, imaging, and tracking of a single protein” Nano Lett.14, 2065–2070 (2014);

[11] M. Piliarik and V. Sandoghdar, “Direct optical sensing of singleunlabelled proteins and super-resolution imaging of their binding sites” Nat.Commun. 5, 4495 (2014);

[12] G. Young, N. Hundt, D. Cole, et al., “Quantitative mass imagingof single biological mac-romolecules” Science 360, 423–427 (2018);

[13] M. Dahmardeh, H. Mirzaalian Dastjerdi, H. Mazal, et al., “Self-supervised machine learn-ing pushes the sensitivity limit in label-freedetection of single proteins below 10 kDa” Nat. Methods 20, 442–447 (2023);

[14] M. Celebrano, P. Kukura, A. Renn, and V. Sandoghdar, “Single-molecule imaging by opti-cal absorption” Nat. Photonics 5, 95–98 (2011);

[15] Y. Park, W. Choi, Z. Yaqoob, et al., “Speckle-field digitalholographic microscopy” Opt. Express 17, 12285–12292 (2009);

[16] M. N. Akram, Z. Tong, G. Ouyang, et al., “Laser speckle reductiondue to spatial and an-gular diversity introduced by fast scanningmicromirror” Appl. Opt. 49, 3297–3304 (2010);

[17] P. F. Almoro, G. Pedrini, P. N. Gundu, et al., “Phase microscopyof technical and biological samples through random phase modulation with adiffuser” Opt. Lett. 35, 1028–1030 (2010);

[18] Y. Choi, T. D. Yang, K. J. Lee, and W. Choi, “Full-field andsingle-shot quantitative phase microscopy using dynamic speckle illumination”Opt. Lett. 36, 2465–2467 (2011);

[19] B. Redding, M. A. Choma, and H. Cao, “Speckle-free laser imagingusing random laser illumination” Nat. Photonics 6, 355–359 (2012);

[20] M. Nixon, B. Redding, A. Friesem, et al., “Efficient method forcontrolling the spatial co-herence of a laser” Opt. Lett. 38, 3858–3861(2013);

[21] T. H. Nguyen, C. Edwards, L. L. Goddard, and G. Popescu,“Quantitative phase imaging with partially coherent illumination” Opt. Lett.39, 5511–5514 (2014);

[22] H. Farrokhi, T. M. Rohith, J. Boonruangkan, et al., “High-brightness laser imaging with tunable speckle reduction enabled byelectroactive micro-optic diffusers” Sci. Rep. 7, 15318 (2017);

[23] V. Bianco, P. Memmolo, M. Leo, et al., “Strategies for reducingspeckle noise in digital holography” Light Sci. Appl.7, 48 (2018).

[24] Cao,H., Chriki, R., Bittner, S., Friesem, A. A.&Davidson, N. Complex lasers withcontrollable coherence. Nat. Rev. Phys. 1, 156-168 (2019);

[24] H. Cao, R. Chriki, S. Bittner, et al., “Complex lasers withcontrollable coherence” Nat. Rev. Phys. 1, 156–168 (2019);

[25] I.-B. Lee, H.-M. Moon, J.-S. Park, et al., “Three-dimensionalinterferometric scattering microscopy via remote focusing technique” Opt.Lett. 45, 2628–2631 (2020);

[26] P. Kukura, H. Ewers, C. Müller, et al., “High-speed nanoscopictracking of the position and orientation of a single virus” Nat. Methods 6,923–927 (2009);

[27] H. M. Dastjerdi, M. Dahmardeh, A. Gemeinhardt, et al., “Optimizedanalysis for sensitive detection and analysis of single proteins viainterferometric scattering microscopy” J. Phys. D: Appl. Phys. 55, 054002(2021);

[28] R. W. Taylor, R. G. Mahmoodabadi, V. Rauschenberger, et al.,“Interferometric scattering microscopy reveals microsecond nanoscopic proteinmotion on a live cell membrane” Nat. Photonics 13, 480–487 (2019);

[29] N. Jiao, S. Lin, D. Feng, et al., “Defocus-integrationinterferometric scattering microscopy for speckle suppression and enhancingnanoparticle detection on substrate” arXiv:2402.16527 (2024);

[30] Y.-T. Hsiao, T.-Y. Wu, B.-K. Wu, et al., “Spinning diskinterferometric scattering confocal microscopy captures millisecond timescaledynamics of living cells” Opt. Express 30, 45233–45245 (2022);

[31] M. Küppers, D. Albrecht, A. D. Kashkanova, et al., “Confocalinterferometric scattering microscopy reveals 3D nanoscopic structure anddynamics in live cells” Nat. Commun. 14, 1962 (2023);

[32] R. G. Mahmoodabadi, R. W. Taylor, M. Kaller, et al., “Pointspread function in interfero-metric scattering microscopy (iSCAT). Part I:aberrations in defocusing and axial localiza-tion”Opt. Express 28, 25969–25988 (2020);

[33] K. Kasaian, M. Mazaheri, and V. Sandoghdar, “Long-range three-dimensional tracking of nanoparticles using interferometric scattering(iSCAT) microscopy” arXiv:2401.12939 (2024);

[34] P. J. Tadrous, “Use of a refractive index–coupled diffuser toboth generate and measure high-numerical aperture illumination for lightmicroscopy” J. Microsc. 292, 135–147 (2023);

[35] S. Wäldchen, J. Lehmann, T. Klein, et al., “Light-induced celldamage in live-cell super-resolution microscopy” Sci. Rep. 5, 15348 (2015);

[36] J. Nixon-Abell, C. J. Obara, A. V. Weigel, et al., “Increasedspatiotemporal resolution re-veals highly dynamic dense tubular matrices inthe peripheral ER” Science 354, aaf3928 (2016);

[37] H. T. Perkins and V. Allan, “Intertwined and finely balanced:endoplasmic reticulum mor-phology, dynamics, function, and diseases” Cells10, 2341 (2021);

[38] C. Ounkomol, S. Seshamani, M. M. Maleckar, et al., “Label-freeprediction of three-dimensional fluorescence images from transmitted-lightmicroscopy” Nat. Methods 15, 917–920 (2018);

[39] V. Badrinarayanan, A. Kendall, and R. Cipolla, “SegNet: A deepconvolutional encoder-decoder architecture for image segmentation” IEEETrans. Pattern Anal. Mach. Intell. 39, 2481–2495 (2017);

[40] A. Curtis, “The mechanism of adhesion of cells to glass: a studyby interference reflection microscopy” J. Cell Biol. 20, 199–215 (1964);

[41] L. Limozin and K. Sengupta, “Quantitative reflection interferencecontrast microscopy (RICM) in soft matter and cell adhesion” ChemPhysChem 10,2752–2768 (2009);

[42] V. A. Barr and S. C. Bunnell, “Interference reflectionmicroscopy” Curr. Protoc. Cell Biol. 45, 4–23 (2009);

[43] T. H. Nguyen, M. E. Kandel, M. Rubessa, et al., “Gradient lightinterference microscopy for 3D imaging of unlabeled specimens” Nat. Commun.8, 210 (2017);

[44] A. D. Kashkanova, A. B. Shkarin, R. G. Mahmoodabadi, et al.,“Precision single-particle localization using radial variance transform” Opt.Express 29, 11070–11083 (2021);

[45] J. W. Goodman, “Some fundamental properties of speckle” J. Opt.Soc. Am. 66, 1145–1150 (1976); and

[46] I. Hamarová, P. Šmíd, P. Horváth, and M. Hrabovsky`, “Methods fordetermination of mean speckle size in simulated speckle pattern” Meas. Sci.Rev. 14, 177–182 (2014)。

[0003] Label-free optical imaging has attracted much attention in many areas of science because it allows visualization of the structure and function of samples in their undisturbed, label-free material state. Several contrast mechanisms, such as Raman spectroscopy or second harmonic generation, have been explored, but the small cross-section of these processes has hindered their widespread use, especially in ultrasensitive applications. In particular, interferometric scattering (iSCAT) microscopy has become a powerful and sensitive tool for detecting nanoparticles and proteins with unprecedented precision. Recent advances in iSCAT have shown that the inherent cross-section of Rayleigh scattering is large enough to detect and image materials at the nanoscale [1, 2], including gold nanoparticles (GNPs) [1, 3], viruses [4-7], extracellular vesicles [8, 9], semiconductor quantum dots [5], proteins [10-13], and even single dye molecules

[14] .

[0004] The iSCAT signal is generated by the interference of light scattered by the beam and the nanoparticle. Because the scattered signal is unsaturated, a single nanoparticle can generate a large signal, resulting in high shot noise limiting sensitivity and high temporal resolution. iSCAT microscopy can be performed in several illumination and detection schemes [2]. Wide-field illumination mode is particularly desirable because it can provide high-speed imaging, limited only by the technology of the imaging device. The combination of wide-field illumination and reflectance detection has proven particularly advantageous because it provides co-path mechanical stability, interferometric resolution in the axial direction, and high-speed imaging. The high sensitivity of iSCAT, especially when imaging complex samples such as biological cells, also results in a speckle background that is ubiquitous because even the slightest ripples or refractive index modulations that are usually randomly arranged in the sample are enough to scatter to produce complex interference patterns [2].

[0005] Speckles are known to pose a significant challenge to numerous imaging applications such as holography and microscopy through turbid media. Various strategies for averaging speckle patterns have been explored, most of which utilize the spatial and temporal coherence of the illumination or detection path [15-25]. In the most sensitive iSCAT applications, such as the detection of single molecules, quantum dots, and proteins [5, 11, 14], the substrate and sample are selected to reduce speckle to a small residual level. The acquired images are then corrected using schemes such as differential rolling averaging [11, 12, 26, 27], cross-correlation analysis

[28] , machine learning

[13] , and defocus integration

[29] . Transmission measurements

[30] and confocal mode measurements

[31] have also been used to bypass speckle, although with some trade-offs. The former sacrifices the enhanced axial resolution of iSCAT by losing interference information about the propagating phase difference between the reference field and the scattered field. Confocal imaging sacrifices speed when imaging large fields of view (FOV).

[0006] One of the most timely applications of label-free microscopy is in cell biology, where the various structures and dynamic processes in living cells are of interest. Ideally, it is desirable to image cells in 3D, at a large field of view, at high speed, at high resolution, and with nanoparticle sensitivity in a label-free manner. However, no single method can provide all of these desired features. Recently, it has been shown that confocal iSCAT (C-iSCAT) allows speckle-free 3D imaging of living cells, albeit at the cost of a much slower imaging speed than WF-iSCAT

[31] .

[0007] Purpose of the invention The object of this invention is to provide an improved apparatus and method for interference scattering microscopy that avoids the limitations of conventional techniques. In particular, the object of this invention is to provide an iSCAT microscope that improves background suppression, particularly the suppression of speckle background from the sample and / or sample substrate; enables a larger FOV in wide-field applications; provides imaging with improved imaging resolution, for example, for imaging biological nanoparticles and nanostructures; provides imaging with increased imaging speed; and / or improves the quantitative assessment of physical properties of nanoparticles and nanostructures, such as size, mass, morphology, refractive index, and / or particle rate, for example, biological materials. Furthermore, the iSCAT microscope is provided for novel imaging applications, particularly in imaging biological samples such as biological cells or cellular components, materials science, and / or semiconductor technologies. Summary of the Invention

[0008] These objectives are respectively achieved by an apparatus and method for interference scattering microscopy that includes the features of the independent claims. Preferred embodiments and applications of the invention arise from the dependent claims.

[0009] According to a first general aspect of the invention, the above objective is achieved by an interferometric scattering microscope apparatus configured to image a sample under study based on iSCAT to obtain a microscopic sample image, the interferometric scattering microscope apparatus comprising an optical imaging system having an illumination device, an optical relay device including a beam splitting device, a sample container, and a detector device.

[0010] The illumination device includes a laser source device arranged to generate illumination light. An optical relay device is arranged between the laser device and the sample container to relay the illumination light to the sample container. A beam splitting device, such as a beam splitting cube and / or a beam splitting mirror, is arranged to deflect a first portion of the illumination light toward the sample container (e.g., a transparent sample substrate carrying the sample under investigation), to deflect a second portion of the illumination light toward a detector device, and to superimpose the scattered light scattered at the sample disposed at the sample container with the second portion of the illumination light.

[0011] The detector device is arranged to receive a second portion of superimposed scattered and illuminated light in the image plane of the optical imaging system. Preferably, the detector device can receive the superimposed light field with a defined tolerance for defocusing based on the system's field depth. The detector device can be coupled to an image processing device arranged to receive and process the detector output signal to create a microscopic image of the sample. Signal and image processing can be implemented as is known in conventional iSCAT microscopes.

[0012] According to the present invention, the iSCAT microscope apparatus further includes a coherence setting device arranged to selectively set the point spread function (iPSF) of the optical imaging system in the image plane by applying and controlling the spatial coherence of the illumination light output from the laser source device.

[0013] According to a second general aspect of the invention, the above-mentioned objective is achieved by an interference microscope method comprising imaging a sample under study based on iSCAT to obtain a sample image, wherein an interference scattering microscope apparatus according to a first general aspect of the invention or an embodiment thereof is employed, wherein the sample comprises biological particles, for example, protein particles and / or protein molecules, wherein the point spread function of the optical imaging system at the sample container is set by applying and controlling the spatial coherence of the illumination light output from the laser source device using a coherence setting device.

[0014] Typically, a coherence setting device (or: coherence setting element) is capable of altering the relative phase, wavefront, and / or amplitude of the optical field wave contributions included in the illumination light in a preferably randomized manner. The coherence setting device can be incorporated as a component of an optical imaging system, such as in and / or attached to an optical repeater. Additionally or alternatively, the coherence setting device can be included in and / or provided by the illumination device.

[0015] The illumination light can initially be generated as coherent light by the light source device. A coherence setting device can be arranged to set the partial coherence of the illumination light, the degree of which is reduced compared to the coherence of the illumination light output from the laser device. Therefore, the illumination light (the measurement light provided by the illumination device and relayed to the beam splitter) has spatial coherence that is selectively reduced and / or shaped (or: customized) to a predetermined degree compared to the initial laser coherence by the action of the coherence setting device. Preferably, the coherence setting device can be configured to apply spatial decoherence to the laser output from the laser source device.

[0016] A coherence setting device is arranged to set the interference point spread function of the optical imaging system in the image plane of the optical imaging system. The interference point spread function is a quantitative measure of the image of a point in the sample imaged by the interference scattering device. Using the coherence setting device, a stochastic randomization of the relative phase, wavefront, and / or amplitude contribution of the light field wave is obtained, particularly at the sample container, and especially within the sample. The degree of stochastic randomization of the relative phase, wavefront, and / or amplitude of the light field wave provides a quantitative degree of coherence of the illumination light, which is set in a targeted manner by the coherence setting device. The degree of coherence can be set according to the imaging task of the iSCAT microscope. In particular, the degree of coherence can be selected by the user of the iSCAT microscope based on the desired spatial imaging resolution and / or according to the imaging method such as particle tracking, for example based on numerical simulation and / or experimental testing. Alternatively or additionally, the degree of coherence can be set based on the image analysis results of sample images collected using the iSCAT microscope.

[0017] Specifically, to collect sample images, the iPSF can be configured to optimize spatial imaging resolution and suppress speckle in the sample images. Specifically, for particle tracking, the iPSF can be configured to ensure that concentric iPSF rings are sufficiently distinguishable from each other and that speckle is suppressed, thereby allowing the encoding of the axial position of sample elements such as nanoparticles.

[0018] Advantageously, the inventors have discovered that by controlling the spatial coherence of the illumination beam, speckle background from the sample can be strongly suppressed in wide-field iSCAT (WF-iSCAT), thereby introducing an operating mode that can be called diffuse illumination iSCAT (D-iSCAT) or equivalently customized spatial coherence iSCAT (TSC-iSCAT). This invention overcomes the speckle problem of conventional techniques while maintaining the sensitivity of wide-field iSCAT microscopy. This is particularly achieved through the ability to precisely design, for example, the spatial coherence of the illumination, via a rotating diffuser mask strategically positioned in the illumination path combined with an adjustable aperture. This allows for the customization of the interferometric point spread function (iPSF) to suit different applications. The method of this invention can achieve, for example, a basic imaging frame rate of at least 25 kHz with a field of view of at least 80 µm × 80 µm while maintaining diffraction-limited resolution, thereby allowing the collection of moving image sequences (video sequences). Furthermore, particularly by optionally utilizing a series of deep neural networks, the inventors have achieved automated segmentation of images of biological samples (e.g., endoplasmic reticulum (ER) networks), highlighting the invention's ability to track intracellular dynamics, such as within biological cells. As illustrated in the examples described below, the inventors have experimentally demonstrated the high throughput of this method by performing 3D tracking of over a thousand vesicles throughout a COS-7 cell. This invention promises to bridge the gap between high-resolution label-free imaging, sensitive detection, and 3D tracking. Advantageously, the TSC-iSCAT of this invention is suitable for label-free studies of complex systems such as living cells at high spatiotemporal resolution.

[0019] The inventors have specifically demonstrated that by designing the spatial coherence of the illumination beam, speckle background from the sample and / or sample substrate can be strongly suppressed in wide-field iSCAT (WF-iSCAT). The inventors have demonstrated specific implementations of embodiments of the invention using, for example, a rotating diffuser mask in the path of the incident laser beam. The inventors have shown that, in addition to eliminating unwanted speckle, a considerably large field of view (FOV) can be accessed in diffuse illumination (customized spatial coherence) iSCAT microscopy.

[0020] Preferably, the coherence setting device can be configured to set the spatial coherence of the illumination light such that the coherence volume in the sample (where the spatial portion of the radiative coherence, e.g., the product of the coherence area and the coherence length) has a characteristic dimension, such as the cross-sectional dimension of the coherence volume, which generates a point spread function in the image plane with a characteristic coherence length less than the characteristic correlation length of the sample image imaged without the coherence setting device. In terms of method, according to another preferred embodiment of the invention, the spatial coherence of the illumination light can be set such that the coherence volume in the sample has a characteristic dimension that generates an iPSF in the image plane with a characteristic coherence length less than the characteristic correlation length of the sample image image without the coherence setting device.

[0021] Specifically, the coherence volume can represent a finite spatial extent in which radiation is coherent, while the overall radiation in at least two different spatial extents of the sample, each possessing the coherence volume, is incoherent. Advantageously, the coherence volume can be small enough that the spatial imaging resolution of features of adjacent samples is not affected by background speckle light (or the effect of background speckle light on imaging is negligible).

[0022] From an application perspective, the coherence volume at the sample can have the following cross-section, preferably selected from the range of 250 nm to 1 µm, particularly for high-resolution imaging tasks, or the cross-section can be selected from the range of 1 µm to 3 µm, particularly for particle tracking tasks.

[0023] Advantageously, various techniques exist for reducing the spatial coherence of a light source to address the speckle problem. In particular, rotating diffusers made of frosted glass, diffusing colloidal particles in a suspension, and / or moving microelectromechanical mirrors have been used. Therefore, according to a preferred embodiment of the invention, the coherence setting device can be configured for at least one of the following: incorporating a time-varying scattering system into an illumination device, setting coherence using a refractive index modulation medium, setting coherence using a microelectromechanical mirror array, and increasing the number of transverse lasing modes of the laser source device.

[0024] According to a particularly preferred embodiment of the invention, the coherence setting device may include a movable diffuser mask as a time-varying scattering system. The diffuser mask comprises a distribution of mask elements and is arranged to apply time-varying, spatially random wavefront characteristics to the illumination light. In particular, the diffuser mask is a rotatable diffuser mask. The diffuser mask may be arranged to transmit or reflect light paths within the coherence setting device. Mask elements (or: mask grains, mask particles, mask features) influence the phase, wavefront, and / or amplitude of the light field wave contribution included in the illumination light in a random manner. Advantageously, the residual coherence after reflection and / or transmission, and after projecting (imaging) the diffuser mask into the sample, can be proportional to the effective size (also called the effective grain size) of the mask elements projected into the sample. The preferred physical dimension of the mask element can be selected from the range of 10 nm to 100 µm, while the preferred effective size of the mask element (projected onto the sample) can be selected from the range of 100 nm to 3 µm.

[0025] Furthermore, according to another preferred embodiment of the invention, the coherence setting device may include at least one coherence setting component of an optical relay device. Advantageously, the optical relay device or a portion thereof may be used to further control or adjust the coherence generated by the coherence setting device.

[0026] Particularly preferably, the at least one coherence setting component may include at least one lens (including, for example, an illumination lens and / or a microscope objective), having a focal length, and arranged between the laser device and the sample container, for example, between the diffuser mask and the beam splitter, for projecting the diffuser mask onto the sample container, particularly onto the sample, and / or arranged between the laser device and the diffuser mask for projecting illumination light onto the diffuser mask. The optical relay device, particularly its at least one coherence setting component, may be arranged to set the effective grain size of the mask element projected onto the sample, which is specifically adjusted by the focal length of the optical relay device. The coherence of the illumination light can be set by the effective grain size of the mask element. The effective grain size is determined by the projection of the diffuser mask onto the sample, i.e., by the final illumination at the sample location.

[0027] Advantageously, the optical repeater allows the coherence of the illumination light to be set by setting an effective grain size. Preferably, the optical repeater can be configured to provide an effective grain size in the range of approximately half the wavelength of the illumination light (particularly the center wavelength of the laser device) divided by the numerical aperture of the optical imaging system to approximately five times the wavelength of the illumination light divided by the numerical aperture of the optical imaging system.

[0028] An optical relay device may include illumination optics, such as at least one optical lens that provides a focal length for projecting a diffuser mask onto a sample. The optical relay device may be adapted to replace lenses; for example, it may include a lens container arranged to house lenses, such as lenses whose focal length is selected for a particular application, or it may include a lens container arranged to house multiple lenses and for moving one of the lenses into the illumination path of the illumination device.

[0029] Preferably, at least one optical lens of the optical relay device can have a variable, for example, tunable focal length, and the optical relay device can be arranged to set the effective (projected) grain size by adjusting the focal length of the at least one optical lens. Advantageously, this feature allows control of the effective grain size without changing the diffuser mask of the coherence setting device and / or even without using a diffuser mask.

[0030] Particularly preferably, the optical relay device may include at least one tunable lens having a tunable focal length and positioned between the laser source device and the sample container. For example, a diffuser mask may be positioned between two lenses, wherein at least one of the lenses is a tunable lens with a tunable focal length. The tunable lens with a tunable focal length may be positioned downstream of the diffuser mask.

[0031] Using optical relay devices, particularly those with tunable lenses, to set the effective grain size is not limited to implementations using movable diffuser masks. Optical relay devices (e.g., those with variable focal lengths) can also be used, alone or in conjunction with other implementations, such as microelectromechanical mirror arrays, to set the coherence level.

[0032] The effective grain size can be set based on the collected TSC-iSCAT images to suppress speckle background to a certain extent. Loop control can be used to set the effective grain size according to image features. In particular, if the spatial imaging resolution obtained through image analysis is insufficient to image the predetermined details to be studied, the focal length of the optical relay can be set to increase spatial decorrelation.

[0033] According to another advantageous embodiment of the invention, the coherence setting device may include a driving device arranged to set the speed of the diffuser mask movement. Advantageously, the driving device provides additional degrees of freedom in adjusting the illumination light. Preferably, the speed of the diffuser mask movement, for example, the rotation frequency, can be faster than the imaging speed of image acquisition using the detector device, so that image acquisition is unaffected by the mask movement. Alternatively, the speed of the diffuser mask movement can be slower than the imaging speed, for example, if an averaging of the acquired images is provided after image acquisition.

[0034] Preferably, the coherence setting device may include a spatial mask, particularly an aperture (diaphragm, variable aperture), which is arranged to suppress reflected light generated at the sample plane by a high-angle k-vector produced by the diffuser mask. Advantageously, the spatial mask allows for further improvement in imaging contrast.

[0035] According to another preferred embodiment of the invention, the detector device may be coupled to an image processing device arranged to process sample images by applying a neural network to the detector output data. Additionally or alternatively, other signal processing techniques may be applied, such as edge recognition in the image.

[0036] The feature dimension of a coherence volume can be equal to or less than the average distance of the sample features included in the sample and resolved in the sample image. Typically, sample features can include, for example, structures of the sample and / or sample substrate that can optically interact with illumination light through absorption or scattering. Preferably, sample features can include details of the sample to be imaged. For example, if sample features (such as certain structures and / or particles, such as vesicles or protein molecules) and / or features of the sample substrate (such as roughness or refractive index variations of the substrate, such as a glass substrate) are included in the sample and / or sample substrate, the coherence volume can make the light in different non-overlapping coherence volumes incoherent.

[0037] Alternatively, another coherence volume can be set according to the application of the technology of the present invention, for example, greater than the average distance of the sample features.

[0038] Preferably, to image the sample with maximum spatial resolution and detection sensitivity, the coherence of the illumination light can be minimized. Alternatively, the coherence of the illumination light can be adapted to tracking moving elements within the sample.

[0039] According to a preferred application of the present invention, iSCAT-based sample imaging includes at least one of the following: imaging of biological particles, for example, said biological particles are protein particles, particularly, said protein particles are protein molecules, RNA particles, DNA particles, vesicles, etc.; quantitatively assessing the mass of biological particles, for example, said biological particles are protein particles, particularly, said protein particles are protein molecules; imaging of particles and / or cellular components included in biological cells by mass spectrometry, particularly said cellular components are vesicles; and sample imaging in materials science and / or semiconductor technology. Determining particle mass by mass spectrometry is based on measuring the iSCAT contrast of the particles, which is proportional to the particle mass. The contrast that distinguishes the signal of interest from the background can be defined as the signal detected at the particle location (…). ) and background ( Normalized intensity difference between intensities Background intensity can be obtained from areas within the FOV that lack sample structures such as cellular structures (e.g., regions containing only sample containers, such as those with only cover glass). Here, the reflected light intensity is used as a baseline for the background, thus providing a reference point for subsequent contrast calculations in TSC-iSCAT imaging.

[0040] The features disclosed within the context of the methods or embodiments of the present invention also represent preferred features of the iSCAT microscope apparatus or embodiments thereof. The foregoing aspects, as well as the preferred features of the present invention, particularly regarding the configuration of the apparatus and the dimensions and composition of the various components described in the apparatus, also apply to these methods. The preferred embodiments, variations, and features of the present invention described above can be combined as needed. Attached Figure Description

[0041] Further details and advantages of the invention are described below with reference to the accompanying drawings, which schematically illustrate: Figure 1 Features of embodiments of the TSC-iSCAT device of the present invention; Figure 2 Features of another embodiment of the TSC-iSCAT device of the present invention; Figures 3 to 5 Additional features of TSC-iSCAT settings and image formation; Figure 6 Images of non-invasive, label-free visualization of live COS-7 cell structures using TSC-iSCAT, particularly illustrating network dynamics and segmentation using SegNet; Figure 7 and Figure 8 High-speed, high-throughput tracking of vesicles in COS-7 cells; Figure 9 and Figure 10 : Measure the average speckle grain size of the diffuser mask; Figure 11 The architecture of the SegNet neural network for image segmentation; and Figure 12 : Used for training data preparation for applying neural networks to detector output data. Detailed Implementation

[0042] The following reference configuration describes features of a preferred embodiment of the invention by way of example, wherein the coherence setting means is provided in particular by a movable diffuser mask. The invention is not limited to the use of a diffuser mask. Alternatively or additionally, the coherence setting means may be provided by other means added to the optical relay device or its components, as described above. Furthermore, features of iSCAT devices and their operation known from the prior art are not described further.

[0043] Refer to the configuration used for the following experimental tests (see Figure 1 , Figure 2 The iSCAT device is described using the term (). Specifically, the iSCAT device is configured as a microscope. For routine applications, modified operating parameters and / or configurations may be used depending on the application conditions. For example, the sensor unit and / or BFP imaging unit may be omitted in routine applications (see below).

[0044] Customized Spatial Coherence iSCAT Figure 1 This is a schematic diagram of a TSC-iSCAT device 100 for imaging sample 1 based on iSCAT according to an embodiment of the present invention. The iSCAT device 100 includes an illumination device 10, an optical relay device 20 including a beam splitting device 21, a sample container 30, a detector device 40, and a coherence setting device 50. Optionally, the iSCAT microscope device 100 may be provided with additional components ( Figure 1 (Illustrated schematically) The additional components include, for example, an image processing device 60 coupled to the detector device 40; a control device 70 arranged for controlling the iSCAT microscope apparatus 100, specifically coupled to the detector device 40 and / or the image processing device 60; and / or a sensor device 80 provided for monitoring purposes. Details of the optical devices, as are known from conventional iSCAT microscopes themselves, will not be described further.

[0045] Illumination device 10 includes laser source device 11, such as a Lastack LAB-520-140 or Lastack FL-525-1200 laser with a wavelength λ = 525 nm. Optical relay device 20 includes illumination optics 22 and objective lens 23, particularly a microscope objective lens. Using illumination device 10 and optical relay device 20, illumination light 2 is generated and relayed to sample 1 contained in sample container 30. Beam splitting device 21 (shown schematically) is arranged to deflect a first portion 2A of illumination light 2 toward sample container 30 and a second portion 2B of illumination light 2 toward detector device 40. Sample container 30, such as a transparent substrate, for example made of glass, is arranged above objective lens 23, and sample 1 is arranged on sample container 30. Beam splitting device 21 is also arranged to superimpose backscattered (reflected) scattered light 2C from sample 1 with the second portion 2B of illumination light 2. The superimposed light field, including the second part 2B of the scattered light 2C and the illumination light 2, is imaged onto the detector device 40. The detector device 40 receives the superimposed light field and generates an output signal that provides an iSCAT signal. In particular, the detector device 40 includes an imaging device 41 and an imaging lens 42.

[0046] The coherence setting device 50 is arranged to selectively set the spatial coherence of the illumination light 2. Figure 1 In this configuration, the coherence setup 50 includes: a transmissive rotating diffuser mask 51 mounted on a drive unit 52 (e.g., a DC motor); and, for example, a focusing lens 53, which is a first lens L1 of the illumination optics 22 (see below). Alternatively, one or more other lenses of the illumination optics 22 may be used for coherence setup. Thus, depending on the configuration of a particular embodiment of the iSCAT device 100, at least a portion of the optical relay 20 may simultaneously provide a portion of the coherence setup 50. Preferably, the diffuser mask 51 may be mechanically decoupled from the support platform, such as an optical stage, carrying the iSCAT device 100 to avoid vibration.

[0047] In more detail, Figure 1 The illumination optics 22 includes five imaging lenses L1 to L5. The first lens L1 is arranged to image the illumination light 2 emitted by the laser device 11 onto a diffuser mask 51 positioned between the first and second lenses (L1 and L2). The first lens L1, particularly its focal length, can be selected to set the degree of decoherence by adjusting the degree of focus of the illumination light 2 on the diffuser mask 51. For this purpose, the first lens L1 can be replaceable, or preferably, a tunable lens, such as a manually tunable or electrically tunable lens, can be used. Tunable lenses can be advantageous in facilitating adaptation of the setup to the sample to be used and the iSCAT imaging conditions.

[0048] Using a second lens and a third lens (L2 and L3), the illumination light 2 is focused onto the projection back focal plane (BFP) 24, where an aperture I is arranged, the diameter of which is, for example, in the range of 0.1 mm to 25 mm. Focusing onto the projection BFP 24 is not necessary, as described in the following reference. Figure 2 As described in the implementation method, a wide-field illumination light field to be relayed to sample 1 is formed using a fourth lens L4 and a fifth lens L5 (including a wide-field lens (WFL)).

[0049] The diffuser mask 51 is made of glass, for example, with a surface microstructure having diffuse features (grains), typically in the range of 10 nm to 100 µm. The diffuser mask 51 is rotatable, such that by rotating the diffuser mask 51, decoherence is applied to the initially coherent illumination light 2 emitted by the laser device 11. The number of diffuse features passing through the diffuser mask 51 by the illumination light 2, i.e., the size of the light field on the diffuser mask 51, is determined by the degree of focusing using the first lens L1. The diffuser mask 51 can rotate about an axis coinciding with or offset from the optical axis of the illumination optics 22.

[0050] Figure 2 This is a schematic diagram of a TSC-iSCAT device 100 for imaging sample 1 based on iSCAT, according to an alternative embodiment of the present invention. Figure 1 Similarly, the iSCAT device 100 includes an illumination device 10 with a laser device 11, an optical relay device 20 with a beam splitting device 21 and an illumination optics device 22, a sample container 30, a detector device 40, and a coherence setting device 50. Figure 2 The implementation may also include components 60 to 80 (see...) Figure 1 , Figure 2 (Not shown in the image).

[0051] and Figure 1 Unlike other illumination optics, illumination optics 22 only includes four imaging lenses L1 to L4, and Figure 1 The aperture I is replaced by a larger aperture I, preferably an adjustable aperture I, which has a diameter, for example, in the range of 0.1 mm to 25 mm. By omitting... Figure 1The focusing lens L3, along with an enlarged aperture I, relays a portion of the light field transmitted through the diffuser mask 51 to the sample 1. This avoids potential high-angle reflections within components and surfaces inside the objective lens 23. The laser beam emitted by the laser device 11 is first focused onto the rotating diffuser mask 51 by, for example, a lens L1 with a tunable focal length. The diffuser mask 51 widens and expands the angle of the laser beam, filling the pupil of lens L2. Next, the collimated laser light is guided by lenses L3 and L4 (wide-field lenses, WFL) to the back focal plane (BFP) 15 of the microscope objective lens 23.

[0052] also, Figure 2 A BFP imaging apparatus 90 is shown that can be arranged for diagnostic purposes (see below) but can be omitted in practical applications of the present invention. The BFP imaging apparatus 90 includes a beam splitter 91, a reflector 92 (optionally tiltable), an imaging lens 93, and a camera 94.

[0053] As Figure 1 and Figure 2 In a practical example of the configuration, the first lens L1 can be selected as the focal plane (optionally tunable), for example, in the range of 30 mm to 100 mm. The second lens L2 is a lens with a fixed focal length of, for example, 100 mm. The distance between L1 and L2 is, for example, 200 mm. The diffuser mask 51 is preferably positioned at the focal plane of the second lens L2. The diffuser mask 51 is, for example, a Thorlabs DG10-220, and the tunable lens L1 is, for example, a lens ML-20-37 (manufacturer Optotune). Optical elements, such as polarizers and λ / 4 waveplates, can be arranged between the laser device 11 and the first lens L1 to provide circular polarization of the illumination light 2. The objective lens 23 is, for example, a high NA objective lens (Olympus UPlanSApo 100x / 1.40NA oil immersion objective), and the beam splitter device can be a 50:50 unpolarized beam splitter. The aperture I placed at the projection position of the BFP is preferably an adaptive aperture I to control the beam diameter and thus the INA. The imaging device 41 is, for example, an ultra-high-speed CMOS imaging device (Phantom v1610, Vision Research) or a Photon Focus imaging device (MV1-D1024E-160-CL) for imaging. The Phantom imaging device produces a maximum field of view (FOV) of 80 µm × 80 µm, while the Photon Focus imaging device allows a maximum FOV of 100 µm × 100 µm.

[0054] Below, for reference Figure 3 The illustrations depict the speckle pattern suppression effect of the present invention. Figure 3The TSC-iSCAT contrast enhancement features are shown, in which Figure 3 Image A shows a standard background-corrected WF-iSCAT image of 40 nm gold nanoparticles (GNPs) at the glass-water interface. Figure 3 B shows a background-corrected TSC-iSCAT image of an 80 nm GNP recorded at the glass-air interface; Figure 3 C shows the relationship with Figure 3 The B is the same, but it is imaged at a frame rate of 25 kHz on a large FOV of 80 µm × 80 µm; Figure 3 D shows an image of the BFP in the detection path, i.e., after the illumination has been reflected from the sample (in this case, the aperture in the illumination path is opened, for example, up to 25 mm). Figure 3 E shows the same as D in 3, but now the aperture is almost closed (e.g., to 0.1 mm) to block light that would otherwise be reflected from the sample surface near the critical angle; and Figure 3 F shows the ratio of TSC-iSCAT contrast to WF-iSCAT contrast as a function of illumination numerical aperture (INA), which is adjusted by changing the size of aperture I.

[0055] In WF-iSCAT, the detected signal can be represented as: (1) Where E r and E s These represent the reference field and the scattered field at camera device 41, respectively. Indicates the position on camera device 41 The degree of complex coherence between the scattered electric field and the reference electric field at a given location is defined as: (2) In (2), This represents the time average. For a monochromatic source with a uniform wavefront, Simplified to ,in It is E s With E r The phase difference between them. This uniform phase across the sample allows E r E caused by the finite aperture of the microscope objective s The interference between the circular diffraction patterns produces a ring structure across the interferometric point spread function (iPSF)

[32] . Figure 3 A presents an example of a WF-iSCAT image of a 40 nm GNP placed on a sample container 30, including, for example, a glass substrate. The resulting iPSF is superimposed on a weak background speckle caused by substrate inhomogeneities.

[0056] Concentric iPSF rings have proven to play a crucial role in 3D tracking applications because they encode the axial position of nanoparticles [28, 33]. However, if the sample medium contains many scattering sources, the superposition of the corresponding rings can complicate the image. The inventors have shown that by manipulating the spatial coherence of the imaging system... In particular, by utilizing the first lens L1, one can customize the range of the iPSF ring and thus find the optimal trade-off between information regarding speckle suppression and the axial position of the nanoparticles encoded in the iPSF ring (the position of sample 1 along the illumination axis) in each application. Specifically, this allows for adjustment of the coherence volume of sample 1 that produces the detected speckle pattern.

[0057] Figure 3 A and Figure 3 The comparison with B shows that when the rotating diffuser mask 51 is inserted, the outer ring of the iPSF disappears, while its central lobe is preserved. To understand this phenomenon, it is helpful to note that the rotating diffuser mask 51 randomizes the phase of the illumination field at different locations on sample 1. Thus, for example, a given nanoparticle in sample 1 is exposed to light at different phases during image acquisition. However, this process does not affect the central iPSF lobe because at the nanoparticle location on the imaging device 41, E... s phase and E r The phase remains perfectly synchronized. At other locations on the imaging device 41 where diffraction rings will form, the E-phase of the nanoparticles... s It has the same phase as the center position, but E r The phase is randomized because the reflected field originates from parts of the sample other than the nanoparticle locations. As a result, the iPSF rings are washed away.

[0058] In WF-iSCAT, the range of the illumination field is governed by the focal length of the wide-field lens. Figure 1 L5 or Figure 2 (L4 in the image), for practical reasons, this focal length is typically on the order of several hundred millimeters. Therefore, the total FOV of the illuminated area is approximately 10 µm × 10 µm. The insertion of the diffuser mask 51 not only randomizes the wavefront but also produces a large angular spread in the illumination beam, which in turn results in an even larger FOV of the illuminated area.

[0059] The lateral extent of the illumination k-vector at the BFP (which can be represented by the illumination numerical aperture (INA)

[34] ) can be controlled by placing the aperture stop I at the focal point of the lens L3 (see

[34] ). Figure 2 ).like Figure 3 As shown in C, the resulting FOV can be nearly two orders of magnitude larger than the range typically used in WF-iSCAT.

[0060] In order to study the function of aperture I, the inventor used Figure 2 The optional BFP imaging device 90 images the BFP 15 of the microscope objective 23 in the detection path, i.e., after the laser beam is reflected from the sample 1, as described below.

[0061] The numerical aperture (NA) of a microscope objective lens is defined as... ,in It is the refractive index of the medium (oil immersion), and It is the maximum angle of the k-vector that is allowed to enter the microscope objective 23. This definition indicates the range of angles of the k-vector that the microscope objective 23 can collect from sample 1.

[0062] In the TSC-iSCAT of the present invention, the amount of reflected light plays a role in the final normalized contrast on the imaging device 41 because the interference between scattered light and reflected light—especially reflected light from the substrate interface—is measured. In addition, the Fresnel reflection coefficient varies with the incident angle. Therefore, the angular distribution of illumination in the TSC-iSCAT determines the final normalized contrast. Within this framework, the concept of illumination numerical aperture (INA) is used to describe the distribution of the k-vector of illumination at different angles

[39] .

[0063] (3) in It is the maximum angle of the k-vector of illumination in the sample space.

[0064] To characterize the INA in the device of the present invention, the back focal plane (BFP) 15 of the microscope objective 23 is imaged, such as... Figure 2 As shown. To achieve this, a 4f optical system is used to project the BFP 15 of the microscope objective 23 onto the imaging device 94. Using X-ray optics, one can indicate the diameter of the image in the BFP. It is associated with INA through the following equation: (4) in That is the focal length of objective lens 23. Figure 5 (See below) This equation is used to evaluate INA.

[0065] Figure 3 The D-value reveals an outer loop on the projected BFP, which originates from the illumination k-vector undergoing strong reflection outside a critical angle of approximately 62 degrees. These components increase E. rThis reduces image contrast. Therefore, although the light does not contribute to the scattered signal, it enhances the background, leading to a decrease in the contrast of the iSCAT signal. Contrast can be controlled, particularly increased, by placing spatial masks, such as apertures in the illumination path, and / or by reducing the opening of aperture I. Figure 3 E shows an example of modified lighting in a projected BFP obtained using the present invention. Figure 3 The symbol in F indicates the ratio of TSC-iSCAT contrast to WF-iSCAT contrast for different aperture settings, i.e., INA settings. The solid curve shows the theoretical expectation. As mentioned above, in some applications, such as three-dimensional (3D) particle tracking [28, 32, 33], complete elimination of the iPSF ring can be avoided.

[0066] The effect of altering spatial coherence is described below with reference to Figure 4, where Figure 4A shows an image of the illuminated FOV segment in WF-iSCAT (left), a measured 40 nm GNP image (middle), and a theoretical iPSF model (right); Figure 4B shows the same as Figure 4A but in TSC-iSCAT mode. Each row in Figure 4B indicates different effective speckle grain sizes obtained through autocorrelation analysis, resulting in full width at half maximum (FWHM) of 1.09 µm (row 1), 490 nm (row 2), 192 nm (row 3), and 141 nm (row 4). These grain sizes correspond to focal lengths of lens L1 of 100 mm, 90 mm, 75 mm, and 30 mm, respectively. In the middle block, the measured exposure time was varied: from left to right, the exposure times were 0.0014, 0.014, and 1, in units of diffuser rotation periods. Figure 4C shows the average FWHM of speckle grains in the illumination field as a function of focal length L1; Figure 4D shows the standard deviation (SD) of background contrast in the TSC-iSCAT image as a function of exposure time, measured relative to the diffuser mask rotation time. The method used to quantify background variation as a function of exposure time, as depicted in Figures 4A through 4D, involves capturing the dynamic speckle pattern at a high frame rate of 10 kHz while the diffuser mask 51 rotates at 5 revolutions per second. The frames are then computationally averaged to create images corresponding to different exposure times.

[0067] Advantageously, through customized spatial coherence The extent to which iPSF rings can be suppressed can be determined. The inventors have discovered two variations that can be used alone or in combination. First, the effective grain size of the rotating diffuser mask can be adjusted, particularly by focusing the lens L1. The effective grain size determines the average feature size of the speckle illumination. Specifically, modifying the focal length of L1 changes the projected grain size of the diffuser on the sample plane (or alternatively, by adjusting INA in this configuration). This allows for optimization of the degree of spatial coherence for different applications. For example, the lateral extent of the iPSF can be minimized for imaging subcellular structures, while a certain degree of ring structure can be preserved for 3D particle tracking. Second, the rotational speed of the diffuser mask can be adjusted to obtain a preferred degree of spatial averaging.

[0068] In the left pane of Figure 4A, a raw example of the FOV imaged in conventional WF-iSCAT is shown, the middle pane shows a background-corrected image of the nanoparticles, and the right pane shows the corresponding theoretical model. The matrix data shown in rows B of Figure 4 presents equivalent data recorded in TSC-iSCAT for different effective diffuser grain sizes with varying focus settings via lens L1, and for different camera exposure times. As the effective grain size decreases, spatial coherence decreases, and therefore the number of iPSF rings decreases.

[0069] By increasing the exposure time in each row of the middle block in Figure 4B, suppression of the measured background fluctuations can also be observed. Figure 4C plots the average illumination grain size as a function of the lens focal length L1, indicating the ability to customize the illumination grain size and thus the degree of spatial coherence. In Figure 4D, the reduction in background noise is shown for longer integration times. Background noise is minimized when integration is performed at integer multiples of the diffuser rotation time. As shown in Figure 4B (last row), the minimum grain size produces an iPSF that retains only the center lobe.

[0070] In summary, Figures 4A through 4D illustrate the dependence of the iPSF ring structure on the projected illumination pattern for TSC-iSCAT with different diffuser grain sizes, such as those controlled by a tunable lens L1. As shown in Figure 4B, as the effective grain size decreases, the effective spatial coherence decreases, and therefore the number of iPSF rings decreases.

[0071] Alternatively or additionally, coherence can be adjusted by changing the INA. Figures 4E to 4G illustrate the effect of the INA, which can be configured by one or more parts of, for example, optical relay device 20, on spatial coherence.

[0072] Figures 4E through 4G depict the iPSF generated by various INA values ​​ranging from 0.5 to 1.3. Figure 4E depicts the projected BFP of TSC-iSCAT under different INA values. Closing the aperture changes the illumination area at the BFP of the objective lens, and thus changes the INA. Figure 4F provides the corresponding iPSF under different INA values. The iPSF center contrast shows the inverse relationship with INA, and for increasing INA values, the iPSF ring disappears. Figure 4G shows the iPSF as a function of INA. The fitted values ​​(see Equation 9) indicate that spatial coherence decreases by increasing INA. These measurements are taken at the lens. This was performed with the focal plane set to 30 mm.

[0073] To demonstrate that customized spatial coherence allows for imaging of nanoparticles in complex scattering media, the inventors fabricated a well-defined sample made from a glass substrate containing randomly distributed features, with diameters on the order of 200 nm and depths of 80 nm. Figure 5 Image A presents a scanning electron microscope (SEM) image of this sample (scale bar: 1 µm). Figure 5 In B, it is shown Figure 5 The WF-iSCAT image of region A with strong speckle. In contrast, Figure 5 The image in C shows that TSC-iSCAT provides speckle-free images with the same structure, where all features of the SEM image can be clearly identified.

[0074] High-speed label-free cell imaging This invention has been tested in high-speed, label-free imaging of biological cells, as described below. In particular, the inventors have demonstrated that TSC-iSCAT allows for very rapid imaging of live cells while maintaining a large field of view (FOV) and eliminating stray speckle. Measurements were taken at 0.076 kW cm⁻¹. - The measurements were performed at the minimum power density, which is significantly below the photodamage threshold

[35] . In fact, no adverse cell response was observed, even for extended periods.

[0075] Figure 6 Label-free live-cell microscopy examination using TSC-iSCAT is shown, in which Figure 6 A shows a composite view of a live COS-7 cell and some adjacent cells, with a field of view (FOV) of 100 µm × 100 µm. The focal length of lens L1 is set to 30 mm, and aperture I is open. Figure 6Figure B shows a magnified area from the box in 68A, including the cell nucleus (dashed box) and the surrounding area, with arrows indicating the actin bundles that provide structural support for the cell. The actin bundles provided by the vesicle structures are presented with varying contrast. The coarse contrast modulation is due to the shape of the cell membrane, which is not perfectly flat on the glass surface supporting the cell. Advantageously, image quality comparable to that of the C-iSCAT mode

[31] was obtained.

[0076] Figure 6 As shown in C, the corresponding WF-iSCAT image of the region within the box drawn in 6B lacks structural detail because the image is dominated by a strong speckle pattern that obscures finer cellular features. Figure 6 D shows a close-up view of the cell nucleus from the dashed box in 6B at a later time. The basement membrane focused on the cell nucleus reveals contrast changes attributable to nuclear membrane invagination (similar to the C-iSCAT image of the cell nucleus

[31] ).

[0077] Figure 6 E shows cytoplasm densely distributed with vesicles (see arrow), some of which diffuse while others are actively transported, and Figure 6 F shows elongated tubular mitochondria (white arrows) with negative contrast surrounded by a fine network of endoplasmic reticulum (ER, black arrows). Notably, these organelles are highly dynamic at different time scales and constantly interact with their environment, which is difficult to capture with microscopy due to labeling limitations or lack of sensitivity

[36] . Indeed, the dynamics and organization of the ER network are of fundamental interest because this organelle interacts with many cellular components, responds rapidly to external stimuli, and plays a crucial role in regulating intracellular function

[37] .

[0078] Endoplasmic reticulum structure and dynamics Advantageously, the high sensitivity of iSCAT makes it possible to receive signals of considerable size from a wide range of nanoscale constituents of biological cells. However, in some cases, it may not be straightforward to identify a structure based solely on its iSCAT contrast, as the iSCAT contrast can modulate between completely destructive and completely constructive interference values, depending on the axial position of the structure. Furthermore, variations in both size and refractive index can affect the contrast. These limitations can be overcome if image processing is performed using neural networks such as the SegNet architecture, and specificity similar to that known in fluorescence microscopy

[38] can be achieved. Neural networks such as the SegNet architecture have been shown to effectively capture and maintain spatial hierarchical structures in images

[39] . Examples of the SegNet architectures used are referenced below. Figure 11 and Figure 12 Describe it.

[0079] Figure 6 G and Figure 6 H represents the TSC-iSCAT image of the ER network and the corresponding output of the neural network consensus, respectively. Figure 6 G shows the ER network recorded at 1.5 kHz, where the focal length of lens L1 is set to 30 mm and INA=1.2. Figure 6 H represents the neural network generated as described below. Figure 6 G segmentation and denoising of the image. Figure 6 Figure I shows a close-up time series of a dotted-line square region from 6G and the corresponding segmented image, illustrating the dynamics of the ER branch. Specifically, Figure 6 The diagram illustrates the dynamics of the ER branch. Neural networks identify intracellular features through characteristics such as shape, size, and granularity.

[0080] To train the neural network, multiple manually segmented TSC-iSCAT images were used (see [link]). Figure 12 Once the individual networks are trained, their outputs are aggregated to form a consensus segmentation. For each pixel in the image, the number of networks that identify it as part of the ER is counted. This count is then mapped to values ​​from 0 to 1 to build a confidence score. Neural networks of varying depths, from 2 to 7 layers, were used (see [link to documentation]). Figure 11 Each SegNet is equipped with three distinct input channels: a grayscale image, the magnitude of the image gradient, and the direction of the image gradient. The grayscale image, highlighting intensity variations, serves as the base layer for the neural network to identify structural features. The other two channels, the magnitude and direction of the gradient, complement this by highlighting edges and their orientation within a single image.

[0081] Vesicle 3D Tracking on a Large Field of View Figure 7 and Figure 8 The application of TSC-iSCAT in simultaneous 3D tracking of numerous vesicles in living biological cells is illustrated. Here, video was recorded over several minutes at a frame rate of 1 kHz on a large FOV of 80 µm × 80 µm, with aperture I open, i.e., INA = 1.4. As previously presented, 3D localization in iSCAT imaging utilizes the rich annular structure of the iPSF [28, 33]. Therefore, by adjusting the focus of lens L1... Tuning was performed to allow a small number of loops around the central iPSF lobe. Figure 7 The images in the upper section show several snapshots of the lateral iPSF along the trajectory of a single vesicle. Figure 7The lower plate shows the radial profile of the iPSF as a function of time. Analysis of this type of data provides the 3D trajectory of each vesicle [28, 33].

[0082] 3D tracking methods can be used to achieve high-resolution localization of vesicles based on the analysis of the radial profile of the iPSF. To track vesicles in 3D, each particle can be localized individually in each frame. 2D localization can be performed by utilizing a method based on radial variance transform (RVT), which was previously introduced in

[44] . The spatial coherence of the system is tuned to obtain a minimum number of concentric rings. 3D tracking involves calibrating the imaging system and creating a computational model that simulates the experimental iPSF, as described in

[33] .

[0083] Figure 8 Figure A shows the trajectories of over 1000 vesicles monitored simultaneously. While most trajectories depict diffusion behavior, some indicate directional transport. Figure 8 B provided Figure 8 A close-up view of the exemplary area enclosed by the white box in A, and Figure 8 Figure C shows a 3D representation of this trajectory. After a 4-second time point, the vesicle transports in the opposite direction. Figure 8 In D, the trajectory of a vesicle moving along the ER tubule, from another measurement, is shown. Here, INA is reduced to 1.2 to enhance contrast (see D). Figure 3 The large number of data points in the trajectory allows us to apply mean square displacement (MSD) analysis to short segments of the trajectory, thereby identifying different patterns of vesicle motion.

[0084] When particles undergo active transport superimposed on diffusion motion, MSD can be expressed as ,in Indicates the number of dimensions. It is the diffusion coefficient. Let t represent the average velocity, and t be time.

[0085] Figure 8 E drew Figure 8 The MSD of the trajectory in D. The initial confinement phase (mode I) indicates possible interaction or temporary anchoring with the ER or microtubule. After 0.6 seconds of labeling, the vesicle transitions to active transport mode (mode II), with an average velocity of 0.579 µm / s. - ¹, during which time it also changes its direction and its velocity (mode III, average speed 0.289 µm / s) - ¹). In the fourth mode, the vesicles exhibit slow diffusion motion, characterized by a diffusion coefficient D = 0.05 µm²s. - ¹ indicates a limited mobility.

[0086] Average grain size of diffuse illumination Figure 9 The average speckle grain size at the measurement sample location is shown, where Figure 9 A shows that when the rotating diffuser mask 51 is closed, at f L1 Full field-of-view image of speckle illumination at 90 mm (bars represent grayscale values ​​recorded by the camera), and Figure 9 B shows Figure 9 The two-dimensional autocorrelation of A. The illustration shows a close-up view of the area indicated by the box.

[0087] The average speckle grain size in TSC-iSCAT is calculated by first capturing an image of the static speckle pattern projected onto the sample at the detection site by a diffuser. A two-dimensional autocorrelation function is then applied to the image. The result of the autocorrelation is a peak, where the width of the peak is related to the average size of the transverse intensity modulation. For fully formed speckles, the full width at half maximum (FWHM) of this peak can be used as a measure of the average grain size.

[0088] More specifically, when a coherent beam interacts with a diffuse medium, it produces a speckle pattern. For a fully developed static speckle pattern, where each speckle feature is produced at position (x, y) by the interference of a number of scattered waves with random phases, the spatial intensity distribution over the field of view can be approximated by a Rayleigh distribution, where the probability density function (PDF) of intensity I is given by

[45] : (5) Where I0 represents the average intensity.

[0089] To determine the average grain size of the speckle illumination, we first capture an image of the static speckle pattern of the diffuser at the detector (see [link]). Figure 9 (A). Then consider the two-dimensional autocorrelation function with coordinate lag. (6), Perform calculations on the image (see) Figure 9 (B). The function exhibits a distinct peak at zero displacement, which decreases with increasing displacement. The full width at half maximum (FWHM) of this peak serves as a direct indicator of the average speckle grain size.

[0090] The results show that the spatial autocorrelation function associated with the speckle pattern can be approximated by a Gaussian function

[46] : (7) Where A0 represents the peak amplitude at the zero-space hysteresis, δ x δ y This represents spatial lag in both the horizontal and vertical dimensions, and δsp This is the standard deviation of the speckle size distribution. Therefore, the average speckle size (S) can be correlated with the FWHM of the autocorrelation function through the following relationship: (8).

[0091] Simulation of iPSF in TSC-iSCAT The complex coherence function at the detector of TSC-iSCAT can be modeled by the following equation: ,in (9) and It is the phase difference between the scattered light and the reference field.

[0092] To model the iPSF in TSC-iSCAT, the iPSF under coherent illumination conditions is modeled based on a vector diffraction model, which rigorously considers the scattered electric field and the polarization and complex amplitude distribution of the reference beam, as detailed in [47, 48]. Next, the iPSF is modified by the Gaussian coherence function mentioned above in equation (9) to take into account the partial spatial coherence of the illumination.

[0093] To identify the model that best approximates the experimentally measured iPSF, the Gaussian width (σ) was varied and the correlation between the simulated and measured iPSFs was calculated. The highest correlation coefficient indicates the most accurate representation of the degree of spatial coherence in the TSC-iSCAT model.

[0094] Figure 10 The relationship between spatial coherence and average speckle size is shown, where Figure 10 A shows that for f L1 =30 mm and f L1 =100 mm, correlation values ​​between the modeled iPSF and the measured iPSF image constructed using coherence functions of different widths (σ). The asterisks on the curves mark the maximum correlation values ​​and the corresponding σ. Figure 10 B shows that for each f L1 The σ value, which produces the maximum correlation, is a function of the corresponding speckle grain size. The lines indicate the linear relationship between these parameters. Therefore, Figure 10 The figure in B shows a clear linear relationship between the speckle characteristic size derived from the autocorrelation function FWHM according to equation (8) and the standard deviation σ of the Gaussian coherence function in equation (9).

[0095] SegNet Neural Network Consensus The consensus of neural networks (NNs) can be used to segment ER videos. This consensus involves 6 NNs, each with a different depth ranging from 2 to 7, all utilizing the SegNet architecture [7], such as Figure 11 As shown. Figure 11 Figure A illustrates a 3-layer SegNet NN example, processing an input consisting of a TSC-iSCAT image, its absolute gradient, and gradient angle at a resolution of 256×256 pixels. NNs trained with different depths can be used to segment images, such that the outputs of the NNs are summed to form a probability map. Figure 11 B illustrates a consensus mechanism that integrates outputs from multiple SegNet models with varying depths from 2 to 7 to produce a comprehensive and segmented final output.

[0096] The neural network (NN) can be trained using 27,000 training datasets and 500 validation datasets. A NN with a depth of 7 achieves a maximum accuracy of 96% and a validation accuracy of 94%. The NN can be trained using MATLAB 2021b on a GPU (NVIDIA A100-PCIE-40GB). Training completes after 5 epochs, taking approximately 90 minutes. The parameters used for training are listed below: Optimizer: adam Initial learning rate: 0.0004 Random shuffling: every period Small batch size: 10 Gradient decay factor: 0.9.

[0097] The trained neural network consensus can segment videos with a resolution of 256 px × 256 px at approximately 10 Hz.

[0098] Training data for neural network consensus Figure 12 The training data preparation is shown, where Figure 12 Image A shows a TSC-iSCAT image of COS-7 cells with the background illumination removed (which shows the ER structure in the cells). Figure 12 B shows Figure 12 The manually segmented image corresponding to A, Figure 12 C shows Figure 12 An example of an enhanced image of A, which is enhanced by rotation, translation, and the addition of additional shot noise. Figure 12 D shows Figure 12 The magnitude of the two-dimensional gradient of C, Figure 12 E shows Figure 12 The angle of the two-dimensional gradient of C, and Figure 12 F shows the relationship with Figure 12The segmented ER image corresponding to C.

[0099] Three images—from the beginning, middle, and end of a video—were selected to generate training data for endoplasmic reticulum (ER) segmentation within cells. The background of these images was subtracted using temporal median background correction. These images were then manually segmented into binary masks, where 1s and 0s were assigned to ER and non-ER regions of the image, respectively. Figure 12 A, Figure 12 B shows the background-corrected image and its corresponding binary mask.

[0100] To enhance the data used for neural network training and validation, 30, 10, and 30 variations of translation, rotation, and shot noise were applied to each of the three manually segmented images. Their corresponding binary masks were also subjected to similar translation and rotation as the images. Finally, 27,000 sets of images and binary masks were generated for the video, with 500 sets used as the validation dataset for the NN training process. Figure 12 C shows an example of an enhanced image of ER.

[0101] To provide the neural network with additional information to guide its better recognition of ER structures, two additional images were added to the network's input. These include the absolute values ​​of the 2D gradients of the images and their corresponding directions (see...). Figure 12 D, Figure 12 The E). As the output of the NN, a binary mask with the same rotation and translation as the image is created (see E). Figure 12 (F).

[0102] Overview of specific features and advantages of the present invention Using this invention, TSC-iSCAT is introduced as a new mode of iSCAT microscopy, in which the spatial coherence of the illumination is customized. Specifically, a key concept for controlling the speckle effect is reducing the range of the interferometric point spread function (iPSF), i.e., the number of its rings. The inventors demonstrate that by designing the illumination coherence, the speckle background generated by the sample in wide-field iSCAT microscopy can be counteracted. As a specific but non-limiting implementation, a rotating diffuser mask in the illumination path combined with an adjustable aperture is employed, allowing manipulation of the iPSF.

[0103] Notably, this invention ensures high-throughput acquisition of speckle-free images at at least 25 kHz across a field of view of at least 1024 × 1024 pixels, while maintaining diffraction-limited resolution. The benefits of this advancement in imaging speed and sharpness are particularly evident in applications in cell biology. The detailed intracellular images obtained underscore the effectiveness of this method. The integration of deep neural networks further highlights this in the case of automatic segmentation of the endoplasmic reticulum (ER) network. The ability to track vesicles in the dynamic ER environment of live COS-7 cells exemplifies the system's capabilities in capturing structural detail and rapid cell dynamics.

[0104] TSC-iSCAT provides a valuable bridge between high-resolution imaging, high-sensitivity detection, and dynamic 3D tracking. With the continued growth in demand for non-invasive high-resolution imaging techniques in biological research, this invention makes significant contributions to label-free bioimaging and mass spectrometry.

[0105] The features of the invention disclosed in the foregoing description, drawings, and claims, whether individually, in combination, or in sub-combinations, can be important for implementing the invention in its various embodiments. The invention is not limited to the preferred embodiments described above. Rather, various modifications and derivatives are possible, which also utilize the inventive concept and therefore fall within the scope of protection. Furthermore, the invention claims protection independently of the features and claims referenced herein, asserting the subject matter and features of the dependent claims.

Claims

1. An interferometric scattering microscope (iSCAT) apparatus (100) configured to image a sample (1) under study based on iSCAT to obtain a sample image, the interferometric scattering microscope apparatus comprising an optical imaging system having an illumination device (10), an optical relay device (20) including a beam splitter (21), a sample container (30), and a detector device (40), wherein, - The lighting device (10) includes a laser source device (11) arranged to generate illumination light (2). - The optical relay device (20) is arranged between the laser device (11) and the sample container (30), and the optical relay device (20) is used to relay the illumination light (2) to the sample container (30). - The beam splitting device (21) is arranged to deflect a first portion (2A) of the illumination light (2) toward the sample container (30), to deflect a second portion (2B) of the illumination light (2) toward the detector device (40), and to superimpose the scattered light (2C) scattered at the sample (1) disposed at the sample container (30) with the second portion (2B) of the illumination light (2), and - The detector device (40) is arranged to receive superimposed scattered light (2C) and the second portion (2B) of the illumination light (2) in the image plane of the optical imaging system. Its features are, - The coherence setting device (50) is arranged to selectively set the point spread function of the optical imaging system in the image plane by applying and controlling the spatial coherence of the illumination light (2) output by the laser source device (11).

2. The interference scattering microscope apparatus according to claim 1, wherein, - The coherence setting device (50) is configured to adjust the spatial coherence of the illumination light (2) such that the coherence volume in the sample (1) has a feature dimension that produces a point spread function with a feature coherence length in the image plane that is less than the feature correlation length of the sample image image without the coherence setting device.

3. The interference scattering microscope apparatus according to any one of the preceding claims, wherein, - The coherence setting device (50) is configured to be used for at least one of the following: incorporating a time-varying scattering system into the illumination device (10), setting coherence with a refractive index modulation medium, setting coherence with a microelectromechanical mirror array, and increasing the number of transverse lasing modes of the laser source device (11).

4. The interference scattering microscope apparatus according to any one of the preceding claims, wherein, - The coherence setting device (50) includes a movable diffuser mask (51) which includes a distribution of mask elements and is arranged to apply time-varying, spatially random wavefront characteristics to the illumination light (2). In particular, the diffuser mask is a rotatable diffuser mask.

5. The interference scattering microscope apparatus according to claim 4, wherein, - The coherence setting device (50) includes a drive device (52) arranged to set the moving speed of the diffuser mask (51).

6. The interference scattering microscope apparatus according to any one of claims 3 to 5, wherein, - The coherence setting device (50) further includes at least one coherence setting component of the optical relay device (20).

7. The interference scattering microscope apparatus according to claim 6, wherein, - The at least one coherence setting component in the optical relay device (20) includes at least one lens (53) having a focal length and being arranged between the laser device (11) and the sample container (30). - The at least one lens (53) is arranged to set the effective grain size of the mask element of the diffuser mask (51) projected into the sample (1), the effective grain size being adjusted by the focal length of the at least one lens (53), and - The coherence of the illumination light (2) is adjusted by setting the effective grain size of the mask element.

8. The interference scattering microscope apparatus according to claim 7, wherein, - The at least one lens (53) has a variable focal length, and - The at least one lens (53) is arranged to set the effective grain size by adjusting the focal length of the at least one lens (53).

9. The interference scattering microscope apparatus according to any one of claims 7 to 8, wherein, - The at least one lens (53) is configured to provide an effective grain size in the range of approximately half the wavelength of the illumination light (2) divided by the numerical aperture of the optical imaging system to approximately five times the wavelength of the illumination light (2) divided by the numerical aperture of the optical imaging system.

10. The interference scattering microscope apparatus according to any one of claims 7 to 9, wherein, - The at least one lens (53) includes at least one tunable lens having a tunable focal length and is arranged between the laser source device (11) and the sample container (30).

11. The interference scattering microscope apparatus according to any one of claims 4 to 10, wherein, - The coherence setting device (50) further includes a spatial mask arranged to suppress reflected light generated by the diffuser mask (51) at the sample container (30), for example, the spatial mask is arranged to suppress total internal reflection light generated by the diffuser mask (51) at the sample container (30), and in particular, the spatial mask is a variable aperture (I).

12. The interference scattering microscope apparatus according to any one of the preceding claims, wherein, - The detector device (40) is coupled to an image processing device (60), which is arranged to process sample images by applying a neural network to the detector output data.

13. An interferometric scattering microscopy (iSCAT) method, the method comprising imaging a sample (1) under investigation based on iSCAT to obtain a sample image, wherein, The interference scattering microscope apparatus (100) according to any one of the preceding claims is used, particularly where the sample comprises biological particles, for example, protein particles and / or protein molecules, wherein, - The point spread function of the optical imaging system at the sample container (30) is set by applying and controlling the spatial coherence of the light output by the laser source device (11) using the coherence setting device (50).

14. The interference scattering microscope method according to claim 13, wherein, - Adjust the spatial coherence of the illumination light (2) so that the coherence volume in the sample (1) has the following characteristic dimension, which generates a point spread function with the following characteristic coherence length in the image plane, which is less than the characteristic correlation length of the sample image imaged without the coherence setting device.

15. The interference scattering microscopy method according to claim 13 or 14, wherein, - To adapt the coherence of the illumination light (2) to tracking the moving elements within the sample (1).

16. The interferometric scattering microscopy method of one of claims 13 to 15, wherein, The iSCAT-based imaging of the sample (1) includes at least one of the following: - Imaging of biological particles, specifically protein particles, for example, protein molecules. - The mass of the biological particles is quantitatively assessed using mass spectrometry, specifically, the biological particles are protein particles, for example, the mass of protein molecules. - Imaging of granules and / or cellular components included in biological cells, particularly vesicles, and - Imaging of samples in materials science and / or semiconductor technology.

Images