Focus adjustment method for holographic imaging systems

Abstract

This invention provides a method for adjusting the focus of a holographic imager to acquire an image of a surface of interest in a sample. [Solution] The method includes: - arranging a sample (1) having a known shape and including at least one reference object described by characteristic parameters including at least a positional parameter; - acquiring an image and determining the position of the reference object relative to the acquisition plane (2') by applying a light diffraction model including spatial parameters of the reference object estimated by approximating the appearance of the reference object in the acquired holographic image; and - determining the position of the surface of interest relative to the acquisition plane from the position of the reference object and the focus adjustment of the image acquisition.

Description

Technical Field

[0001] The present invention relates to the field of imaging, and more precisely, aims to adjust the focus for obtaining an image of a surface of interest, and endeavors to identify the position of the surface of interest (hereinafter referred to as the "surface of interest") of a sample by a holographic method.

Background Art

[0002] Several focus adjustment methods for imaging have been developed. These are typically methods for accurately measuring a surface of interest, which is a plane, so as to be able to set / select the object focal plane of an imaging system, and thus obtain a high-quality image of that surface of interest.

[0003] For example, EP3,339,835 provides a method of directing a wavefront of coherent radiation through a sample of an object in a suspension, capturing an interference diagram between the wavefront of the coherent radiation and the wavefront of the radiation diffracted by the object with an image sensor, digitally determining the focal plane of at least one object, and digitally reconstructing an out-of-focus image of at least one object from an interference pattern in a plane having a predetermined offset with respect to an image plane substantially parallel to the image sensor and the focal plane. This method further includes identifying at least one portion of the out-of-focus image corresponding to at least one object in the sample and calculating at least one characteristic of the corresponding object from each portion.

[0004] The method provided by EP3,339,835 is easy to implement, but the step of determining the focal plane of at least one object is performed by conventional autofocus algorithms, which have various drawbacks such as lack of accuracy, some of which are described below. The position of the object focal plane in a holographic imaging system corresponds to the plane (sharpness, contrast, etc.) on which the object to be imaged must be positioned to maximize the quality of the generated image, and can be roughly estimated by knowing the optical properties (e.g., focal length) of the holographic imaging system and its position. However, several difficulties arise. For example, in the context of holographic microscopy, since the object to be imaged at the surface of interest is usually in the micrometer size range, a position error of a few micrometers in the object focal plane has a significant impact on the quality of the resulting image. Therefore, even in the simplest case (microscope slide), where the surface of interest to be imaged corresponds to a physical holder of a flat sample, focusing is critical and can be complex and lengthy. Furthermore, if the sample is a complex or three-dimensional object, the surface of interest to be imaged may not be in a known position on the surface of the sample holder, but may be at a certain distance from that surface. Such a sample may have multiple surfaces to be imaged at different positions relative to the imaging system. Furthermore, like the surface of interest, the object's focal plane (commonly misused as the "focal plane") may not be planar if optical aberrations are present.

[0005] In practice, it is often necessary to describe the relative position of the sample to the image sensor and to acquire and analyze a series of images taken at different positions on the optical axis, i.e., at different z-coordinates on the optical axis. Descriptors are extracted from each of these images, from which the merit factor is derived, and images of a quality in which the z-coordinate is estimated to be the coordinate of the focal plane are identified. These methods are based on the hypothesis that image contrast is maximized at the focal plane, and that the merit factor refers to maximizing local contrast, gradient, variance, entropy, power spectral density, etc.

[0006] Such assumptions are relevant in some cases, particularly when an opaque object is present at the focal plane, but these methods do not apply to all situations. For example, in bright-field microscopy of objects that do not absorb light, the contrast is very low even at the focal plane, and in such situations, most quality factors may not be maximized.

[0007] Other methods can only be used under specific material conditions. For example, certain focus adjustment criteria can only be used for objects that act as pure phase shifters or amplitude absorbers. This is, for example, Liebling, Mr. & Unser, M. “Autofocus for digital Fresnel holograms by use of a Fresnelet-sparsity criterion” JOSA A 21, 2424-2430 (2004), Dubois, F., Schockaert, C., Callens, N. & Yourassowsky, C. “Focus plane detection criteria in digital holography microscopy by amplitude analysis” Opt. Express 14, 5895-5908 (2006), and This is the case of Trujillo, CA & Garcia-Sucerquia, J. “Automatic method for focusing biological specimens in digital lensless holographic microscopy” Opt. Lett. 39, 2569-2572 (2014).

[0008] Other criteria require the use of several wavelengths for illumination, which can be experimentally challenging and still rely on specific assumptions about the dependence of absorption and phase shift of the imaged object as a function of wavelength.

[0009] To use in focusing via the merit factor, it may be possible to add particles with known optical properties to the sample. For example, gold particles that exhibit very high amplitude contrast and minimal phase contrast at the focal plane can be used for this purpose. For example, Bon, P. et al. “Three-dimensional nanometre localization of nanoparticles to enhance super-resolution microscopy.” Nat. Comm. 6, 7764 (2015) provides such a method. However, this approach requires specialized optical instruments because it necessitates precise phase and amplitude measurements. [Overview of the project]

[0010] The present invention aims to enable focus adjustment for acquiring an image of a surface of interest on a sample using an image sensor, even under unfavorable conditions such as bright-field imaging of low-contrast objects, without relying on high-performance equipment.

[0011] For this purpose, the present invention is a focus adjustment method for acquiring an image of a surface of interest of a sample by an image sensor of an imaging system, - A step of arranging a sample, wherein the surface of interest is placed within the field of view of the image sensor of the imaging system, the sample includes at least one reference object having a known shape and described by characteristic parameters associated with the reference object, the characteristic parameters include at least a position parameter of the reference object, and the reference object is located at a position corresponding to the position of the surface of interest, - A step of illuminating a sample with illumination light, and a step of acquiring a holographic image of the interference pattern caused by a reference object on an acquisition plane. - A step of determining the position of the reference object relative to the acquisition plane, wherein a light diffraction model of at least one reference object is used, the light diffraction model is based on the known shape of the reference object and includes characteristic parameters related to the reference object, and at least the position parameters of the reference object are estimated by adjusting the light diffraction model to approximate the interference pattern caused by the reference object in the holographic image, - A step of determining the position of the surface of interest relative to the acquisition plane from the position of the reference object defined by the position parameter, - Includes the step of adjusting the focus of image acquisition based on the position of the surface of interest relative to the acquisition plane.

[0012] This method allows for focus adjustment by aligning the position of the surface of interest with the acquisition plane of the image sensor through the analysis of a simple two-dimensional image. Unlike conventional methods, the method according to the present invention does not require phase reconstruction on the acquisition plane, illumination at several wavelengths, or mechanical scanning analysis of several z positions on the optical axis of the sample.

[0013] The present invention is advantageously complemented by the following distinct features, either alone or in different possible combinations thereof: - The reference object is either an artifact added to the sample, or a microorganism present in the sample; - The reference object is spherical, cylindrical, or elliptical; - Optical diffraction models include the Mie, generalized Mie, Thompson, or Rayleigh models; - Characteristic parameters associated with a reference object also include geometric parameters that describe the known shape of the reference object; - The geometric parameters of the reference object are estimated by adjusting the optical diffraction model to approximate the interference pattern caused by the reference object in the holographic image; - Characteristic parameters related to the reference object include the refractive index of the reference object, estimated by tuning the optical diffraction model, and approximate the interference pattern caused by the reference object in the holographic image; - The sample is placed in an immersion medium, and the optical diffraction model, which includes the refractive index of the immersion medium estimated by adjusting the optical diffraction model, approximates the interference pattern caused by the reference object in the holographic image; - The sample includes at least three reference objects that are not aligned within the field of view of the image sensor, the position parameters of these objects relative to the acquisition plane are determined, and the step of determining the position of the reference surface includes determining the position of the reference surface on which the reference objects are placed, and the position of the surface of interest is determined from the position of the reference surface by the correspondence between the positions of the reference objects and the positions of the surface of interest; - The position parameters of the reference object include position coordinates relative to the acquisition plane; the position of the reference surface is determined by interpolation of the position coordinates, and then the parameter coordinates of the reference surface relative to the acquisition plane are determined; - Multiple holographic images are acquired by an image sensor in the acquisition plane at different positions and / or with illumination light of different frequencies, the position of at least one reference object relative to the acquisition plane of the holographic image is determined, and the determination of the position of the surface of interest includes multiple positions of at least one reference object; - Focus adjustment for image acquisition involves relatively displacing the sample and the image sensor over a displacement distance derived from the position of the surface of interest relative to the acquisition plane, at least along the optical axis of the imaging system; - Focus adjustment for image acquisition involves changing the optical components of the imaging system that displace the acquisition plane; - Image acquisition focus adjustment involves transforming the acquired holographic image using a propagation model that includes the position of the surface of interest relative to the acquisition plane.

[0014] The present invention also relates to holographic imaging systems: - An image sensor configured to acquire a holographic image in the acquisition plane within the field of view, - A light source configured to illuminate a sample placed within the field of view of an image sensor, wherein the sample includes a surface of interest and further includes at least one reference object having a known shape, described by characteristic parameters associated with the reference object, the characteristic parameters include at least a position parameter, and the reference object is located at a position corresponding to the position of the surface of interest, - An automated data processing system configured to receive a holographic image acquired by the image sensor and determine the position of a reference object relative to the acquisition plane, wherein the system uses a light diffraction model of at least one reference object, the light diffraction model is based on a known shape of the reference object and includes characteristic parameters related to the reference object, at least the position parameter of the reference object is estimated by adjusting the light diffraction model to approximate the interference pattern caused by the reference object in the holographic image, the position of the surface of interest relative to the acquisition plane is determined from the position of the reference object defined by the position parameter, and the holographic imaging system is configured to perform a focus adjustment method for acquiring an image of a surface of interest of a sample by the image sensor according to the present invention. [Brief explanation of the drawing]

[0015] Other features, purposes, and advantages of the present invention will become apparent from the following description, but this is purely illustrative and not limiting, and should be read in reference to the accompanying drawings shown below. [Figure 1] Figure 1 shows a simplified schematic diagram of a holographic imaging system according to one possible embodiment of the present invention. [Figure 2]FIG. 2 is a diagram showing a schematic cross-section of a sample having an object surface and a reference object according to one possible embodiment of the present invention. [Figure 3] FIG. 3 is a diagram showing an example of a holographic image of a sample containing a plurality of polystyrene balls as reference objects. [Figure 4] FIG. 4 is a diagram showing a three-dimensional model indicating the positions of the reference objects in the image of FIG. 3 determined by a method according to one possible embodiment of the present invention. [Figure 5] FIG. 5 is a diagram showing the result of digitally focusing the image of FIG. 3 from the positions of the reference objects determined by a method according to one possible embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0016] Figure 1 schematically represents the imaging system, which is an online holographic imaging system in which a sample 1 is imaged by a digital image sensor 2 positioned on the image plane of the holographic imaging system. The imaging system is called a holographic imaging system because it can acquire holographic images, but this same imaging system can also acquire non-holographic images. The optical axis 5 connects the sample 1 and the image sensor 2. Here, this optical axis 5 is shown as a straight line, but it can be more complex as a function of the configuration. The light source 4 is configured to illuminate the sample 1 within the field of view of the holographic imaging system with an illumination beam of illumination light that is sufficiently coherent to acquire a hologram, i.e., coherent or partially coherent. The illumination light has conventional properties for holographic imaging and there are no particular additional restrictions. Therefore, the illumination light can be monochromatic (e.g., with a wavelength of around 500 nm) or, in some cases, can be composed of several wavelengths, which can be used, for example, sequentially. The imaging system may include a set of optical elements 8 in the optical path between the sample 1 and the image sensor 2. In the example shown in the figure, the holographic imaging system includes a microscope objective lens 8a and a tube lens 8b positioned between the sample 1 and the digital image sensor 2. However, the optical elements such as the microscope objective lens 8a are arbitrary, and the present invention is not limited to a holographic microscope with lenses or a specific set of optical elements. Of course, the arrangement described here is a non-limiting example. Any holographic imaging system can be used online or offline, with or without microscope objective lenses, etc. In fact, this method relies on the use of the holographic image acquired by the imaging system. Therefore, as long as the imaging system can acquire a holographic image in which interference patterns caused by the reference object appear, this imaging system is suitable for implementing this method.The imaging system also includes an automatic data processing system, not shown, which includes at least one processor and a memory, receives at least one holographic image from the image sensor 2, and processes this holographic image to determine the position of the surface of interest 6 with respect to the holographic imaging system (particularly with respect to its acquisition plane 2’).

[0017] The sample 1 includes the surface of interest 6 to be imaged. The surface of interest 6 may be planar or curved in the simplest case. The surface of interest 6 may extend in a plane perpendicular to the optical axis 5 or may be inclined with respect to the plane perpendicular to the optical axis 5. The position of the surface of interest 6 is defined in the imaging system as the spatial arrangement of the surface of interest 6, including the position and possible inclination. In particular, if the sample 1 is a three-dimensional object having a specific volume that includes several surfaces or layers at several positions on the optical axis 5, the surface of interest 6 can be part of the sample 1. More generally, the surface of interest 6 corresponds to the position of the part of the sample 1 where imaging is attempted. Usually, the sample 1 is placed on a holder 12 such as a microscope slide, and advantageously the surface of interest 6 corresponds to the interface 12a between the holder 12 and the sample 1, or to a surface parallel to this interface 12a as in the example shown, or its position can be estimated from the position of the interface 12a between the holder 12 and the sample 1. It should be noted that the selection of the surface of interest 6 can benefit from forms of prior knowledge about the sample 1 and the object to be observed, for example, from the size of the microorganism 15 that is likely to be present in the sample 1 and placed on the holder 12.

[0018] Sample 1 includes at least one reference object 10 located at a position corresponding to the position of the surface of interest 6. The reference object 10 has a known shape. This shape is known because it can be described by geometric parameters. Alternatively, the shape may be known only with respect to the size of the reference object 10, for example, if the reference object 10 is on the Rayleigh scattering borderline, i.e., if its dimensions are 5 to 10 times smaller than this wavelength, then no geometric parameters exist. Sample 1 preferably includes several reference objects 10, at least three reference objects 10, and more preferably at least five reference objects 10. If Sample 1 may include a large number of reference objects 10, typically one or two dozen or more reference objects 10 do not need to appear in the acquired image.

[0019] The characteristic parameters associated with the reference object 10 include at least positional parameters, usually positional coordinates, that individually position each reference object 10. The position of the reference object 10 means the spatial arrangement of the reference object 10 in the imaging system, including its position and possible inclination. Preferably, the characteristic parameters associated with the reference object also include geometric parameters that describe the known shape of the reference object. The geometric parameters correspond to a priori knowledge of the geometric shape of the reference object 10. In this respect, the reference object 10 has a simple geometric shape, preferably a sphere, cylinder, or ellipse. In the case of a reference object 10 having a spherical shape, the geometric parameters may simply consist of the radius of the sphere modeling the reference object 10 and positional coordinates corresponding to the position of the center of that sphere. More generally, the geometric parameters are parameters considered by the optical diffraction model that will be used (e.g., Mie model, generalized Mie model, Thompson model, Rayleigh model). Therefore, for a reference object 10 that is basically a sphere, the Mie diffraction model (or Lorentz-Mie solution) requires only the radius of the sphere as a geometric parameter. For a reference object 10 that is basically a cylinder, only the radius and length constitute the geometric parameters necessary to describe the shape. Therefore, it is not necessary to know all the geometric parameters of the reference object 10, but only the parameters that correspond to the approximation of the shape of the reference object 10 and are used by the diffraction model.

[0020] The size of the reference object 10 is not critical and can have a diameter (larger dimensions) in the range of, for example, 10 nm to 100 μm. The size of the reference object 10 depends on the secondary aspect ratio. On the one hand, the size of the reference object 10 must be such that it can produce sufficient contrast, even considering the wavelength of the illumination light, the refractive index difference between the reference object 10 and the surrounding medium, or the contrast generated by other objects in the holographic image. Therefore, the diameter of the reference object 10 is preferably greater than 10 nm, and more preferably greater than 100 nm. On the other hand, the projected area of ​​the reference object 10 on the image sensor 2 should not be too large so as not to alter the quality of the holographic image. Typically, a projected area of ​​less than 10% (even less than 1% is preferable) allows for good holographic image quality if the rest of the sample 1 is not too dense otherwise. Furthermore, the reference object 10 should not mask the rest of the sample 1, especially the surface 6 of interest. Therefore, the size of the reference object 10 is chosen so as not to take up too much space in the field of view of the image sensor 2. In this regard, the reference objects 10 are preferably separated from each other rather than adjacent, and preferably distributed across the entire field of view of the image sensor 2.

[0021] The reference object 10, like any material, has a refractive index. Because the reference object 10 is distinct from the rest of the sample 1, its refractive index differs, albeit very slightly, from that of the surrounding medium. Therefore, a difference of 0.01 in the refractive index of the reference object 10 from that of the surrounding medium allows for the identification of the effect on optical diffraction in the acquired image, and a method for doing so can be implemented. However, preferably, the refractive index of the reference object 10 differs from that of the portion of the sample 1 immediately adjacent to the reference object 10 by at least 0.05, and preferably at least 0.1. The refractive index of the reference object 10 is preferably known and input into the optical diffraction model. In particular, if the reference object 10 is opaque, i.e., if the complex component of its refractive index can be considered to tend toward infinity at the wavelength of interest, the optical diffraction model can be appropriately modified to limit the number of parameters to adjust in order to reduce computation time and to avoid over-adjustment. In the case of a transparent or partially transparent reference object 10, the refractive index of the reference object 10 may also be unknown or only known inaccurately, and can be estimated using a diffraction model in the same way as the geometric or positional parameters related to the reference object 10, and thus can be part of the characteristic parameters related to the reference object 10.

[0022] Sample 1 may include an immersion medium 14, and the optical diffraction model includes the refractive index of the immersion medium 14, which is estimated by tuning the optical diffraction model to approximate the interference pattern caused by a reference object in a holographic image in the same way as the object and associated characteristic parameters, by using the diffraction model.

[0023] There are few restrictions on the selection of the reference object 10, except that the reference object 10 must have a known shape described by the relevant characteristic parameters or be of a very small size with respect to the wavelength of the illumination light. The reference object 10 can be opaque or transparent and can be composed of various materials such as silica, polystyrene, or metals such as gold. In light of the above considerations, the reference object 10 may be an artifact added to the sample 1. The advantage of adding an artificial reference object lies primarily in the refined knowledge of its geometric parameters and refractive index, as well as the regularity of its shape. In the case of artificial reference objects 10 added to the sample 1, they are selected to have a simple, regular shape, preferably cylindrical or elliptical, and more preferably spherical, in order to allow for optimal validity between the actual shape of the reference object 10 and the approximation described by the characteristic parameters considered by the diffraction model. For example, as shown in Figure 2, the reference object 10 is an opaque polystyrene ball with a diameter of 1 μm.

[0024] The reference object 10 may also be present on the holder 12 of the sample 1, and may form part of the holder 12 at the interface with the sample 1. For example, the surface 12a of the holder 12 can be etched by photolithography to reveal a preferred rounded shape that can satisfy the requirements of the reference object (e.g., a rounded nib).

[0025] Reference object 10 could be a microorganism present in sample 1. For example, the bacterium Staphylococcus epidermidis is quasi-spherical and has a known size within a specific range (and thus can be estimated by adjustment) and a known refractive index within a specific range (and thus can be estimated by adjustment). Furthermore, it is a human symbiotic bacterium typical of the skin's bacterial flora and is therefore likely to be commonly present in the sample to be imaged (naturally or due to contamination). Other types of microorganisms can be used as long as they have a shape that can be described by geometric parameters likely to be considered in the optical diffraction model, and their size and refractive index can be estimated. Therefore, natural components of sample 1 can be used advantageously. It is also possible to use both inert artificial materials added to sample 1 and microorganisms present in sample 1.

[0026] As mentioned above, the reference object 10 is located at a position corresponding to the position of the surface of interest 6, that is, there is a correspondence between the position of the surface of interest 6 and the position of the reference object 10. In certain configurations, there are no problems, for example, when the reference object 10 is naturally present on the surface of interest 6 of the sample 1 (e.g., in the case of microorganisms), or when the surface of interest 6 coincides with the surface of the holder 12 on which the reference object 1 is formed. In other configurations, however, a configuration may be required to ensure the correspondence between the position of the reference object 10 and the surface of interest 6. If the surface of interest 6 coincides with the surface 12a of the holder 12, or is coupled to this surface 12a of the holder 12, for example by a parallel relationship, the reference object 10 can be deposited directly onto the surface 12a of the holder 12 before the rest of the sample 1 is placed on the holder 12. Therefore, before placing the rest of the sample 1, a fluid immersion medium 14 (e.g., water) containing the floating reference object 10 can be deposited on the surface 12a of the holder 12 in advance.

[0027] When sample 1 is placed, sample 1 is illuminated by illumination light, and image sensor 2 acquires at least one two-dimensional holographic image. This may be a series of images acquired separately or as a sequence of images, in particular a series of holographic images acquired at different positions on the acquisition plane 2' relative to sample 1 along the optical axis 5 and / or at different wavelengths relative to the illumination light.

[0028] These different positions of the acquisition plane 2' relative to the sample 1 can be obtained for each different position of the sample 1 along the optical axis 5 relative to the image sensor 2, for example, by displacing the sample 1 and its holder 12 along the optical axis between each image acquisition using a motor-driven stage. Alternatively, the image sensor 2 can be displaced via a motor-driven rail or motor-driven stage. Furthermore, these different positions of the acquisition plane 2' can also be obtained by changing the optical components of the holographic imaging system by changing the focusing of the light beam incident on the image sensor 2 in order to displace the acquisition plane 2' from the image sensor 2.

[0029] As described above, various holographic image acquisition techniques can be used, as long as it is clear from the acquired image that the optical effects caused by the presence of the reference object 10 within the field of view of the digital image sensor 2, particularly the interference patterns resulting from the interference between illumination light and light scattered by the reference object 10, appear in the holographic image as interference patterns caused by the reference object, so that it can be used with any type of holographic image system. However, it goes without saying that the image is acquired with a holographic image system configured to image the reference object 10 (or rather, the interference patterns caused by them), and therefore appropriate settings common to those skilled in the art (such as illumination) are made.

[0030] Figure 3 shows an example of a two-dimensional intensity holographic image acquired by the image sensor 2 of the holographic imaging system. The reference object 10 here is an opaque polystyrene ball with a diameter of approximately 1 μm placed on the surface of the holder 12. This sample is imaged with illumination light at a wavelength of approximately 510 nm.

[0031] Once a holographic image is acquired, the positions of one or more reference objects 10 relative to the acquisition plane 2' are determined. For this purpose, a light diffraction model is used for the reference objects 10. The diffraction model is used to detect the reference objects 10 in the acquired holographic image and determine their positions relative to the acquisition plane 2'. The diffraction model allows for the prediction of interference caused by the presence of the reference objects 10 in the field of view, and therefore the appearance of the reference objects 10 in the acquired holographic image, as a function of their position relative to the acquisition plane 2', based on the properties of the reference objects 10. More precisely, the light diffraction model considers the geometric parameters and positional coordinates of the reference objects 10, and optionally the refractive index of the reference objects 10. Other parameters, particularly the index or refractive index of the immersion medium 14, or properties of the holographic imaging system such as the wavelength, magnification, and digital aperture of the illumination light, may be considered. However, these other parameters are user-defined design elements of the instrument, known, and fixed. Therefore, they are merely corresponding to the settings and do not change for different reference objects 10 in the same image in any case. Therefore, the important parameters are the geometric parameters and positional coordinates of the reference object 10, and to a lesser extent, its optical properties (especially its refractive index).

[0032] The characteristic parameters associated with the reference object are variables that are adjusted to best fit the interference pattern described by the model with the interference pattern that appears in the acquired holographic image. The parameters associated with the reference object 10, or at least the position coordinates of the reference object, are estimated by adjusting the optical diffraction model to approximate the interference pattern attributable to the reference object in the acquired holographic image. The validity between the interference pattern described by the diffraction model and the interference pattern that appears in the acquired holographic image allows us to determine that the characteristic parameters (position parameters, geometric parameters, optical parameters) used in the diffraction model are those of the reference object 10. This estimation can be done by minimizing one criterion, which may be an inverse problem approach that includes constraints based on and / or a priori (e.g., quadratic reminders of the expected values ​​of the geometric and optical parameters) and / or (e.g., on the range of acceptable values ​​for the geometric or optical parameters). For example, under the assumption of Gaussian noise relating to the intensity sensed by each pixel of the digital image sensor 2, a potentially weighted least squares method may be used between the optical diffraction data D (i.e., the acquired image) resulting from the interaction of the incident wave and the reference object and the model M, depending on a vector of spatial parameters V describing this reference object, to predict the electromagnetic field and detailed intensity in a given image plane for the image sensor 2. If the reference object 10 is used as a sphere of radius r with its center at coordinates (x,y,z), then the vector of parameters to be estimated is V=(x,y,z,r) t The parameter estimate E is determined by minimizing the standard L2 of the difference between the data D and the model M applied to the vector V of the parameters to be estimated:

number

[0033] Furthermore, if the refractive index is one of the characteristic parameters to be estimated by adjusting the diffraction model, it can be added to the vector of parameters to be estimated, as well as the geometric and positional parameters. In the above embodiment, the vector of parameters to be estimated is V=(x, y, z, r, n) with refractive index n of the reference object 10. t It takes the form of [this]. Other parameters involved in the diffraction or propagation of light, which introduce uncertainty, can also be estimated. However, preferably, the number of parameters to be estimated is kept as small as possible so as not to change the accuracy of the method.

[0034] Preferably, several reference objects 10 are present within the field of view of the image sensor 2 and appear in the acquired holographic image. The position of each reference object 10 is determined, preferably iteratively, for each reference object 10 by determining its respective position parameters with respect to the acquisition plane 2'. Preferably, this is done by subtracting characteristic parameters associated with that reference object 10 from the data of the acquired holographic image each time a reference object 10 appears, with the parameters being predetermined. In particular, a greedy type algorithm can be applied.

[0035] As long as three or more unaligned reference objects appear on the acquired holographic image, not only can the relative positions of the reference objects on the optical axis 5 (z-coordinate) with respect to the acquisition plane 2' be determined, but the orientation of the so-called reference surface on which these reference objects 10 are placed can also be determined. Therefore, determining the position of the surface of interest 6 may include determining the reference surface on which these reference objects 10 are placed, for example by interpolating the position coordinates of the reference objects 10 with respect to the acquisition plane 2', following the determination of the positions of the reference objects 10, and then determining the parametric coordinates of this reference surface on which the reference objects 10 are placed. Subsequently, the position of the surface of interest 6 can be determined with respect to this reference surface by correspondence between the reference surface and the surface of interest 6.

[0036] Figure 4 is a three-dimensional representation showing the positions of the seven reference objects 10 in the image of Figure 3, and with respect to the acquisition plane 2', these positions are determined by adjustments to the Mie model of optical diffraction, including the position of the ball, geometric and optical parameters, in this case the position coordinates (x,y,z), the radius of the ball, and their refractive indices. Each point represents the center of the ball. The coordinates are shown in units of tens of micrometers for both the optical z-axis 5 and the x and y coordinates perpendicular to the optical z-axis 5. Figure 4 shows that the reference objects 10 are well positioned on the reference surface, in this case on the surface 12a of the holder 12 on which they are placed.

[0037] In the example in Figure 4, the reference surface 6' whose position can be determined is the surface 12a of the holder 12 on which the balls are deposited. In a particularly advantageous special case, another reference surface could also be selected, such as the center of the balls, which is offset z from the holder 12 by a distance corresponding to the radius r of the balls. However, such a surface passing through the center of the balls may be irregular and non-planar if not all balls have exactly the same radius. Therefore, it is preferable to select the surface 12a of the holder 12 as the reference surface 6', which is in principle planar and does not depend on the regularity of the ball radii.

[0038] Once the position of the reference object 10 is determined relative to the acquisition plane 2', it is possible to determine the position of the surface of interest 6 relative to the acquisition plane 2' from the position parameters of the reference object 10 (usually its position coordinates), since the reference object 10 is located at a position corresponding to the position of the surface of interest 6. If the parametric coordinates of the reference surface 6' on which the reference object 10 is placed are determined, the position of the surface of interest 6 can be determined from these parametric coordinates of the reference surface 6'.

[0039] The position of the surface of interest 6 can coincide with the position of the reference object 10, which is the simplest correspondence. Therefore, in the example of Figure 4, if the surface of interest 6 coincides with the interface between the sample 1 and the holder 12, the position of the reference object 10 or the reference surface 6' derived therefrom will directly give the position of the surface of interest 6 relative to the acquisition plane 2'. The position of the surface of interest 6 may not coincide with the position of the reference object 10, but there is always a correspondence between the position of the surface of interest 6 and the position of the reference object 10. In this case, the position of the surface of interest 6 can be defined with respect to the position of the reference object 10. This correspondence can be, for example, a unidirectional offset, particularly along the optical axis 5.

[0040] In the example in Figure 4, the position of the surface of interest 6 corresponds to the midpoint of the reference object 10 on the surface 12a of the holder 12. In the example in Figure 4, the surface of interest 6, whose position can be determined from the position of the reference object 10, can therefore be defined with respect to the center of the ball by a simple translation along the optical axis 5, rather than the surface 12a of the holder 12 which forms the interface between the sample 1 and the holder 12. Considering the position of the ball on the surface 12a of the holder 2 as the reference surface 6', it is sufficient to offset the z coordinate by a value equivalent to once the radius r of the ball, which may have been determined as a geometric parameter forming the characteristic parameters associated with each reference object 10. It is easy to select another surface of interest 6. For example, if we want to know the position of the top of the ball which forms another surface of interest 6, it is sufficient to translate twice the radius of the ball from the surface 12a of the holder 12 which is the reference surface 6'.

[0041] For example, if sample 1 contains 2 μm thick microorganisms on holder 12, the surface of interest 6 can be selected to pass through the center of these bacteria. Thus, the surface of interest 6 is defined by a 1 μm offset from the surface 12a of holder 12, which is taken as the reference surface 6'. However, other offsets are also possible. Thus, if the surface of interest 6 coincides with the plane on which the center of the ball is located, the surface of interest 6 formed by the plane of the ball's center can be determined to be on the optical axis 5 at a distance of approximately 17.5 μm from the acquisition plane 2'. Advantageously, as in these examples, the correspondence between the position of the reference surface and the position of the surface of interest 6 may depend on the geometric parameters of the reference object 10. However, this is not essential. For example, with respect to the reference object 10, it may be desirable to image the surface of interest 6 located at a given distance, independent of the geometric parameters of the reference object. To image the surface of interest 6 5 μm from the surface 12a of holder 12, a 5 μm translation with respect to the z-coordinate of the bottom of the ball (along the optical axis 5) is sufficient.

[0042] If there is only one reference object 10, the only positional information that can be determined for focus adjustment is the position of the reference object 10, and therefore only the position of the surface of interest 6 along the optical axis 5 via the z-coordinate of the reference object 10. However, this is the most valuable information. This knowledge of the position of the surface of interest 6 on the optical axis 5 alone enables focus adjustment of the imaging system for image acquisition, that is, it is possible to reliably acquire an image of the surface of interest 6 at the focal plane of the holographic imaging system. Subsequently, the reference surface 6' is a reference surface perpendicular to the optical axis 5, and its z-coordinate is the z-coordinate of the reference object 10 (e.g., its center or side).

[0043] Preferably, sample 1 includes at least three reference objects within the field of view of the image sensor 2, and their position parameters with respect to the acquisition plane 2' are determined, preferably including at least five reference objects 10. Having position parameters for several reference objects 10 with respect to the acquisition plane 2' allows for improved accuracy in determining the position of the surface of interest 6 with respect to the acquisition plane by using a combination of position parameters, such as the position coordinates of several reference objects 10, for example by averaging their z coordinates. Having multiple position coordinates also makes it possible to determine possible inclinations of the reference surface where the reference objects 10 are positioned with respect to the acquisition plane 2', and therefore possible inclinations of the surface of interest 6, using x and y coordinates defined on a plane perpendicular to the optical axis 5. As mentioned above, the parametric coordinates of the reference surface can be determined for this purpose. Furthermore, multiple position coordinates allow for estimating the accuracy of the position determination by determining statistical scattering estimators of the results, such as the standard deviation.

[0044] Similarly, to further improve accuracy, it is possible to work with several holographic images acquired by the image sensor 2 at different positions in the acquisition plane 2'. For each holographic image, the position of at least one reference object 10 relative to the acquisition plane 2' of that holographic image is determined, and the determination of the position of the surface of interest 6 involves several positions of at least one reference object 10 relative to different positions in the acquisition plane 2'. To account for different optical effects depending on the wavelength of the illumination light, it is also possible to work with several holographic images acquired by the image sensor 2 at different wavelengths of illumination light.

[0045] Once the position of the surface of interest 6 relative to the acquisition plane 2' is determined, the focus adjustment for image acquisition can be performed based on the position of the surface of interest 6 relative to the acquisition plane 2'. Focus adjustment means aligning the position of the surface of interest 6 with the focal plane of the object in relation to the acquisition plane 2'.

[0046] Focus adjustment may involve displacing the sample 1 or image sensor 2 along the optical axis 5 over a displacement distance derived from the position of the surface of interest 6 with respect to the acquisition plane 2'. Focus adjustment is performed, for example, by displacing the sample 1 along the optical axis 5 using a rail or motor-driven stage that displaces the holder 12, so that the position of the surface of interest on the sample 1 coincides with the acquisition plane 2' of the image sensor 2. It is also possible to change the optical component of the imaging system that displaces the acquisition plane 2' to coincide with the position of the surface of interest. In this way, the subsequently acquired image is made so that the surface of interest is imaged in a focused state on this surface of interest. It is clear that this image acquired as a result of focus adjustment may advantageously be a holographic image, but it should be noted that this image acquired as a result of focus adjustment is not necessarily a holographic image. For example, it is possible to use a holographic image only for focus adjustment, and then acquire an image using another acquisition method, such as white light.

[0047] Another focusing mode, also known as holographic reconstruction, involves using a digital focusing method that transforms an acquired holographic image using a propagation model that includes the position of the surface of interest 6 relative to the acquisition plane 2'. Convolution of the image data with a complex function allows the focal plane to be displaced retrospectively or inductively (a posteriori) with respect to the imaged object. In particular, the Rayleigh-Sommerfeld propagation model can be used. For example, this model is described in Chapter 3 of Joseph W. Goodman's book "Introduction to Fourier Optics," second edition, McGraw Hill Higher Education, 1996. By digitally focusing already acquired images, a mechanical focusing system can be eliminated (to be dispensed with).

[0048] Figure 5 shows the result of digital (or "computer") focus adjustment applied to the image in Figure 3 by Rayleigh-Sommerfeld propagation reconstruction, where the focal plane is displaced to correspond to the surface of interest 6 (a reference surface corresponding to the surface 12a of the holder 12 on which the ball rests), which was selected to pass through the center of the ball. Although some artifacts (halo around the ball) can be observed, it should be noted that the ball appears clearly as if it had been placed directly in the acquisition plane 2' during image acquisition.

[0049] The results of the above method demonstrate that, in addition to the ability to acquire the initial holographic image, it is possible to adjust the focus to acquire an image of the surface of interest 6 of sample 1 with great accuracy without requiring specific equipment or imposing significant constraints on the imaging system. Therefore, this method can be applied to all existing imaging systems capable of acquiring holographic images.

[0050] The present invention is not limited to the embodiments described and represented in the accompanying drawings. However, modifications are possible without departing from the scope of protection of the present invention, particularly with respect to the configuration of various technical characteristics, or by substitution of technical equivalents.

[0051] [Note 1] A method for adjusting the focus to acquire an image of a surface of interest from a sample using an image sensor from an imaging system, - A step of placing the sample having the surface of interest within the field of view of the image sensor of the imaging system, wherein the sample includes at least one reference object having a known shape and described by characteristic parameters associated with the reference object, the characteristic parameters include at least a position parameter of the reference object, and the reference object is located at a position corresponding to the position of the surface of interest, - The steps of illuminating the sample with illumination light, and acquiring a holographic image of the interference pattern caused by the reference object in the acquisition plane, - A step of determining the position of the reference object with respect to the acquisition plane, using a light diffraction model of the at least one reference object, wherein the light diffraction model is based on the known shape of the reference object and includes characteristic parameters of the reference object, and at least the position parameters of the reference object are estimated by adjusting the light diffraction model, approximating the interference pattern caused by the reference object appearing in the holographic image, - A step of determining the position of the surface of interest relative to the acquisition plane from the position of the reference object defined by the position parameter, - A step of performing focus adjustment to acquire the image based on the position of the surface of interest with respect to the acquisition plane, A method that includes this. [Note 2] The reference object is either an artificial substance added to the sample, or a microorganism present in the sample. The method described in Appendix 1. [Note 3] The aforementioned reference object is spherical, cylindrical, or elliptical. The method described in Appendix 1. [Note 4] The aforementioned optical diffraction model is the Mie, generalized Mie, Thompson, or Rayleigh model. The method described in Appendix 1. [Note 5] The characteristic parameters related to the reference object also include geometric parameters that describe the known shape of the reference object. The method described in Appendix 1. [Note 6] The geometric parameters of the reference object are estimated by adjusting the optical diffraction model to approximate the interference pattern caused by the reference object in the holographic image. The method described in Appendix 5. [Note 7] The characteristic parameters associated with the reference object include the refractive index of the reference object, estimated by adjusting the optical diffraction model, and approximate the interference pattern caused by the reference object appearing in the holographic image. The method described in Appendix 1. [Note 8] The sample is placed in an immersion medium. The optical diffraction model includes the refractive index of the immersion medium estimated by adjusting the optical diffraction model, and approximates the interference pattern caused by the reference object appearing in the holographic image. The method described in Appendix 1. [Note 9] The sample includes at least three unaligned reference objects within the field of view of the image sensor, and their positional parameters relative to the acquisition plane are determined. The step of determining the position of the surface of interest includes the step of determining the position of the reference surface on which the reference object is located, The position of the reference surface is determined from the position of the reference surface, based on the correspondence between the position of the reference object and the position of the surface of interest. The method described in Appendix 1. [Note 10] The position parameter of the reference object includes position coordinates with respect to the acquisition plane, The position of the reference surface is determined by interpolation of the position coordinates, and then the parameter coordinates of the reference surface with respect to the acquisition plane are determined. The method described in Appendix 9. [Note 11] Multiple holographic images are acquired by the image sensor in the acquisition plane at different positions and / or with illumination light of different frequencies. For each of the holographic images, the position of at least one of the reference objects relative to the acquisition plane of the holographic image is determined. The determination of the position of the surface of interest includes multiple positions of the at least one of the reference objects. The method described in Appendix 1. [Note 12] The focus adjustment for image acquisition includes the relative displacement between the sample and the image sensor, at least along the optical axis of the imaging system, over a displacement distance derived from the position of the surface of interest relative to the acquisition plane. The method described in Appendix 1. [Note 13] The focus adjustment for image acquisition includes changing the optical component of the imaging system that displaces the acquisition plane. The method described in Appendix 1. [Note 14] The focus adjustment for the image acquisition includes transforming the acquired holographic image using a propagation model that includes the position of the surface of interest relative to the acquisition plane. The method described in Appendix 1. [Note 15] It is an imaging system: - An image sensor configured to acquire a holographic image in the acquisition plane within the field of view, - A light source configured to illuminate a sample placed within the field of view of the image sensor, wherein the sample includes a surface of interest and further includes at least one reference object having a known shape, described by characteristic parameters associated with the reference object, wherein the characteristic parameters include at least a position parameter, and the reference object is located at a position corresponding to the position of the surface of interest, - An automated data processing system configured to receive a holographic image acquired by the image sensor and determine the position of the reference object relative to the acquisition plane, wherein the system uses a light diffraction model of at least one reference object, the light diffraction model is based on the known shape of the reference object and includes characteristic parameters related to the reference object, at least the position parameter of the reference object is estimated by adjusting the light diffraction model to approximate the interference pattern caused by the reference object in the holographic image, the position of the surface of interest relative to the acquisition plane is determined from the position of the reference object defined by the position parameter, and the imaging system is configured to perform a focus adjustment method for acquiring an image of the surface of interest of a sample by the image sensor, as described in any one of appendices 1 to 14. An imaging system equipped with the following features.

Claims

[Claim 1] A method for adjusting the focus to acquire an image of a surface of interest from a sample using an image sensor from an imaging system, - A step of placing the sample having the surface of interest within the field of view of the image sensor of the imaging system, wherein the sample includes at least one reference object having a known shape and described by characteristic parameters associated with the reference object, the characteristic parameters include at least a position parameter of the reference object, and the reference object is located at a position corresponding to the position of the surface of interest, - A step of illuminating the sample with illumination light, and a step of acquiring a holographic image of the interference pattern caused by the reference object in the acquisition plane, - A step of determining the position of the reference object with respect to the acquisition plane, using a light diffraction model of at least one reference object, wherein the light diffraction model is based on the known shape of the reference object and includes characteristic parameters of the reference object, and at least the position parameters of the reference object are estimated by adjusting the light diffraction model, approximating the interference pattern caused by the reference object appearing in the holographic image, - A step of determining the position of the surface of interest relative to the acquisition plane from the position of the reference object defined by the position parameter, - A step of performing focus adjustment for acquiring the image based on the position of the surface of interest with respect to the acquisition plane, wherein the focus adjustment is to align the position of the surface of interest with the focal plane of the object in relation to the acquisition plane. A method that includes this.

Images