Determining defocus in image data relating to prepared blood sample

Abstract

A method for determining defocus in image data associated with a prepared blood sample by a blood analyzer is disclosed. Image data relating to a prepared blood sample arranged in the detection volume is obtained by an imaging system, the image data comprising data relating to at least one imaging plane corresponding to a depth of the prepared blood sample. The image data is analyzed by identifying a subject region in the obtained image data, the subject region comprising a set of pixels in the image data corresponding to a subject physically present in the prepared blood sample, and identifying an optical feature of the identified subject region, the optical feature originates from the refractive index difference between the subject and the medium of the prepared blood sample, and the optical feature is used as an additional artifact in the image data that does not represent the subject physically present in the prepared blood sample. At least a defocus direction in the image data is determined based on an appearance of the identified optical feature.

Description

Technical Field

[0001] The invention relates to a method for determining defocus in image data relating to a prepared blood sample. Background Art

[0002] The analysis of biological fluid samples (e.g., prepared blood samples) can be lengthy and require many steps, preparations, resources, and advanced equipment. Where the analysis is performed by an optical system (such as a microscope), the object being analyzed (such as blood cells) needs to be in focus in order to ensure proper, reliable, and rapid analysis.

[0003] US2020 / 0358946 Al discloses a system and method for capturing a whole slide image of a sample, wherein a camera is configured to capture a digital image of the sample. The system captures a bright field image of the sample and captures a digital image of the sample illuminated at a first wavelength from a first incident angle and at a second wavelength from a second incident angle. The system can determine whether the sample is defocused based on a transition offset between a first wavelength channel and a second wavelength channel of the captured digital image. Summary of the invention

[0004] It is an object of embodiments of the present invention to provide a method allowing an easy, fast and reliable determination of defocus in image data relating to a prepared blood sample.

[0005] The present invention provides a method for determining defocus in image data associated with a prepared blood sample by a blood analyzer comprising an imaging system and a detection volume, the method comprising the following steps:

[0006] - arranging the prepared blood sample in the detection volume of the blood analyzer,

[0007] - obtaining, by means of the imaging system of the blood analyzer, image data relating to the prepared blood sample, the image data comprising data relating to at least one imaging plane corresponding to a depth of the prepared blood sample,

[0008] - Analyze the acquired image data, comprising the following steps:

[0009] - identifying an object region in the obtained image data, the object region comprising a set of pixels in the image data, the set of pixels corresponding to an object physically present in the prepared blood sample,

[0010] - identifying an optical feature of the identified object region, said optical feature originating from a refractive index difference between said object and a medium of said prepared blood sample and said optical feature serving as an additional artifact in said image data which is not representative of an object physically present in said prepared blood sample, and

[0011] - determining at least a defocus direction in said image data based on the appearance of the identified optical feature.

[0012] Thus, the method according to the invention is a method for determining defocus in image data relating to a prepared blood sample. In the present context, the term "determining defocus" should be interpreted as covering determining whether a given portion of the image data is in focus, determining which portions of the image data can be considered to be in focus or out of focus, respectively, determining the degree to which a defocused portion of the image data is out of focus or how far a defocused portion is "out of focus", determining the direction of a defocused portion of the image data towards the focus, i.e. whether the imaging plane is in front of or behind the focal plane, and / or any other suitable type of determination providing information about the focus and defocus of the image data.

[0013] In this context, the term "prepared blood sample" should be interpreted as meaning a blood sample that has undergone one or more preparation steps to ensure that the blood sample can be properly analyzed. Such preparation steps may, for example, include dilution, application of chemicals, application of reagents, incubation, stabilization, etc.

[0014] In the present context, the term "image data" should be interpreted as meaning optical data originating from or relating to a prepared blood sample. Thus, the image data comprises information about the prepared blood sample being analyzed.

[0015] The method is performed by a blood analyzer, i.e., a device configured to perform an analysis of a prepared blood sample. The blood analyzer comprises an imaging system, i.e., a system configured to generate image data, and a detection volume, i.e., a volume configured to receive a prepared blood sample to be analyzed. The imaging system may, for example, be or include a microscope and / or a holographic imaging device.

[0016] In the method according to the invention, a prepared blood sample is initially arranged in a detection volume of a blood analyzer.

[0017] Next, image data related to the prepared blood sample is obtained by an imaging system of the blood analyzer. This may, for example, include directing a light beam toward the prepared blood sample and detecting light that leaves the prepared blood sample as a result thereof, for example including light scattered, reflected, deflected, diffracted, etc. by objects in the prepared blood sample. Alternatively or additionally, the obtained image data may include at least one holographic image.

[0018] The image data obtained in this way include data relating to at least one imaging plane corresponding to the depth of the prepared blood sample and thus to the distance from the detector of the imaging system. This will be described in further detail below.

[0019] Next, the obtained image data is analyzed in the following manner. An object region in the obtained image data is identified. The object region includes a group of pixels in the image data, which corresponds to an object physically present in the prepared blood sample. Therefore, the object is a real physical entity that is actually contained in the prepared blood sample to be analyzed, i.e., forms a part of the prepared blood sample to be analyzed. Therefore, the object region represents an image of the object physically present in the prepared blood sample in the image data, so that the part of the image data corresponding to the object region contains information about the object. Therefore, analysis about the object can be appropriately performed based on the part of the image data corresponding to the object region.

[0020] Furthermore, an optical signature of the identified object region is identified, the optical signature originating from the refractive index difference between the object and the medium of the prepared blood sample. The optical signature acts as an additional artifact in the image data. Thus, in contrast to the above-mentioned objects, the optical signature does not represent an object physically present in the prepared blood sample. Instead, the optical signature represents a purely optical phenomenon originating from the optical properties at the boundary between a real physical object visible in the image data and the surrounding medium, but is not an image of the actual physical object.

[0021] When light passes through an interface between two media having different refractive indices, it is refracted according to Snell's law, and this produces an optical feature that can be observed as an optical artifact in the image data. Therefore, when incident light strikes an object in a prepared blood sample, and the object has a refractive index that is different from the refractive index of the medium of the prepared blood sample and, therefore, the refractive index of the medium in which the object resides, it can be expected that a recognizable optical feature will appear as an optical artifact in the image data at or near the object region. However, as described above, such optical features or artifacts do not represent actual physical objects present in the prepared blood sample.

[0022] The inventors of the present invention have found that information about the defocus of the image data can be derived from such optical features or artifacts. For example, an optical feature caused by refraction can act as an additional or imaginary object or artifact in the image data, and the appearance of this additional artifact in the image data depends on whether a given observed imaging plane of the image data coincides with the focal plane of the imaging system, and in the case of defocus, also on whether the focal plane is closer to the objective or detector of the imaging system or farther away from the objective or detector of the imaging system. Thus, by observing the appearance of the optical feature in the image data, information related to the defocus in the image data can be retrieved.

[0023] Thus, based on the observation of the appearance of the identified optical features, at least the defocus direction in the image data is ultimately determined. Thus, at least it is determined whether a given imaging plane of the image data coincides with the focal plane of the imaging system, and if not, whether the focal plane is closer to or further away from the objective or detector of the imaging system than the imaging plane.

[0024] Thus, according to the present invention, at least the direction of defocus in the acquired image data is determined in an easy, fast, reliable and robust manner. Thus, possible defocus can be taken into account or handled in a fast, reliable and robust manner, thereby allowing a reliable and robust automatic analysis of the prepared blood sample. For example, the appearance of optical features can be derived from image data related to a single imaging plane, and thus at least the direction of defocus can be determined without the need for image data from several imaging planes, illumination of several wavelengths, etc.

[0025] In addition to being related to differences in refractive index, the observed optical features mentioned above can also be related to the shape of the object.

[0026] The step of obtaining image data associated with the prepared blood sample may include directing a light beam toward the prepared blood sample, the light beam having a numerical aperture that is less than a numerical aperture of an objective lens of the imaging system.

[0027] According to this embodiment, the light beam directed toward the prepared blood sample, i.e., the incident or illumination light beam, has a numerical aperture smaller than the objective lens of the imaging system, and thus smaller than the numerical aperture of the detection portion of the imaging system. In this context, the term "numerical aperture" should be interpreted as meaning a dimensionless number characterizing the angular range over which an optical system can emit or receive light.

[0028] Thus, according to this embodiment, the cone of incident light beams defined by the range of emission angles of the light source is smaller than the acceptance cone of the objective of the imaging system.

[0029] For example, the numerical aperture of the light beam may be less than 80% of the numerical aperture of the imaging system's objective, such as between 5% and 50% of the numerical aperture of the imaging system's objective, such as between 10% and 40% of the numerical aperture of the imaging system's objective.

[0030] By applying a numerical aperture of the light beam that is small compared to the numerical aperture of the objective of the imaging system, the entire light beam can be accommodated within the acceptance angle of the objective of the imaging system. Furthermore, by applying a small numerical aperture of the light beam, the prepared blood sample is illuminated with a small range of incident light angles, so that the determination of the defocus direction becomes very precise and robust.

[0031] The step of obtaining image data may comprise obtaining at least one holographic image. In such a holographic image, optical features of the above-mentioned type may be observed. Therefore, according to this embodiment, a subsequent analysis is performed on the at least one holographic image obtained and at least the defocus direction is determined based on the appearance of the identified optical features appearing in the holographic image.

[0032] The object may be a blood cell, such as a leukocyte, erythrocyte or platelet.According to this embodiment, the interface defining the transition from a region with one refractive index to a region with another refractive index may be formed by the membrane of the blood cell.

[0033] Blood cells have a shape that causes them to act as an optical lens. Therefore, when incident light strikes a blood cell, a lensing effect is created, thereby producing a recognizable optical feature in the resulting image data, acting as an additional artifact.

[0034] For example, for a red blood cell, the combination of the cell's refractive index and the cell's shape causes the red blood cell to act as a concave lens, defining a focal point behind the cell.

[0035] Similarly, for white blood cells, the combination of the cell's refractive index and the cell's shape causes the white blood cell to act as a convex lens, defining a focal point in front of the cell. This is due to the fact that the shapes of red blood cells and white blood cells are different from each other, i.e., red blood cells have a substantially concave shape, while white blood cells have a substantially convex shape.

[0036] Using image data relating to blood cells to determine the defocus of the image data is advantageous because thereby the defocus is determined based on objects that should be analyzed and that are already present in the prepared blood sample, so that no separate objects, such as beads or contrast agents, need to be added to the prepared blood sample. Furthermore, it is directly determined whether the cells to be analyzed are in focus.

[0037] As an alternative, the object may be any other suitable kind of object present in the prepared blood sample and for which analysis may be desired.As another alternative, the object may be an object present in the prepared blood sample whose sole purpose is to determine defocus in the image data.

[0038] Thus, the optical feature may result from a lensing effect produced by the object, for example as described above.

[0039] The step of identifying optical features of the identified object area may include identifying a first annular feature and a second annular feature, and the step of determining at least a defocus direction in the image data may include determining a relative position of the first annular feature and the second annular feature.

[0040] In the case where the object is of a type that produces a lens effect (e.g., blood cells as described above), this may produce an optical feature in the image data in the form of two annular objects arranged circumferentially relative to the object, i.e., an additional artifact, where one of the annular objects appears as a dark ring and the other annular object appears as a bright ring.

[0041] In the case where the object acts as a convex lens and the observed imaging plane is closer to the objective lens and the detector than the focal plane, the bright ring is arranged closer to the object than the dark ring. However, when the observed imaging plane is farther away from the objective lens and the detector than the focal plane, the dark ring is arranged closer to the object than the bright ring. Therefore, the position of the focal plane is the position where the dark ring and the bright ring switch positions.

[0042] Similarly, in the case where the object acts as a concave lens, the position of the annular features is reversed. Thus, in this case, if the observed imaging plane is closer to the objective and detector than the focal plane, the dark ring is arranged closer to the object than the bright ring, and when the observed imaging plane is farther away from the objective and detector than the focal plane, the bright ring is arranged closer to the object than the dark ring.

[0043] Thus, the mutual position of the annular object reveals whether the observed imaging plane is arranged closer or further away from the objective and detector than the focal plane, and the direction of defocus can thus be derived from this information alone. As described above, this may require only image data relating to a single imaging plane.

[0044] Furthermore, the further the observed imaging plane is from the focal plane, the more "blurry" the annular object becomes. Thus, information about the size or magnitude of the defocus can also be derived from the appearance of the annular object.

[0045] The step of determining at least the defocus direction in the image data may further be based on applying knowledge about the object. This may for example include applying knowledge about the optical properties of the object, such as whether the object produces a lens effect, and if so, whether it acts as a concave or convex lens.

[0046] As described above, red blood cells are expected to act as concave lenses, while white blood cells are expected to act as convex lenses. Therefore, if it is known whether the observation object is a red blood cell or a white blood cell, the defocus direction can be easily derived from the observation order of dark rings and bright rings in the image.

[0047] The method may further comprise the step of determining the size or magnitude of defocus in the image data. As described above, this may be performed, for example, based on how "blurry" the identified optical feature appears in the image data.

[0048] The image data may comprise a plurality of images, each image being associated with an imaging plane, the image data thereby forming an image stack associated with the plurality of imaging planes, and the method may further comprise the step of identifying, among the plurality of imaging planes, an imaging plane in which an image of the object is in focus.

[0049] According to this embodiment, the image data is in the form of a stack of images, each image being obtained at a specific depth of the prepared blood sample and thus corresponding to a specific imaging plane. Thus, the image stack includes several images of the same object at various depths, and therefore includes several images of the same object at various distances from the objective lens and the detector of the imaging system. The images of the object at the various imaging planes can be compared, in particular the appearance of the identified optical features on the imaging planes. Thereby, it can be determined which imaging planes are arranged closer to the objective lens and the detector than the focal plane, and which are arranged further away. Thus, the imaging plane closest to the focal plane can be identified, and therefore the imaging plane in which the object can be considered to be in focus can be identified.

[0050] For example, where the object is of a type that produces a lens effect (e.g., a blood cell) and the identified optical feature is of a type that includes a first annular feature and a second annular feature, then the imaging plane where the annular features appear to switch positions can be identified as the imaging plane in which the image of the object is focused.

[0051] Subsequently, analysis of the object may be performed based on the portion of the image data representing that the object is in focus (ie based on the image of the identified imaging plane).

[0052] The above described embodiments are well suited for settings where the prepared blood sample is substantially stationary and therefore several images can be obtained at several depths of the prepared blood sample.

[0053] As an alternative, the image data may comprise data from only one imaging plane, and the method may further comprise the step of adjusting at least one setting of the imaging system based on the determined defocus direction.

[0054] According to this embodiment, image data from only one imaging plane is obtained and based thereon, at least the defocus direction is determined in the manner described above (i.e., simply by observing the appearance of the identified optical features or artifacts). For example, in the case where the object is of a type that produces a lens effect (e.g., a blood cell) and the identified optical features are of a type that includes a first annular feature and a second annular feature, the direction of the defocus can be immediately determined simply by observing the sequence of the rings, as described above. Once the defocus direction is determined, at least one setting of the imaging system can be adjusted so as to bring the image of the object into focus. Subsequently, an analysis of the object can be performed based on the focused image.

[0055] This embodiment is well suited for settings where the prepared blood sample is moving, such as a flowing blood sample, and therefore obtaining several images of the same object at various depths in the prepared blood sample is not possible or practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The present invention will now be described in more detail with reference to the accompanying drawings, in which:

[0057] Figure 1 is a schematic diagram of a blood analyzer for performing a method according to an embodiment of the present invention,

[0058] Figure 2 shows image stacks associated with a plurality of imaging planes and obtained at various settings of the numerical aperture of the incident light beam;

[0059] Figure 3 Shown are simulations of images of blood cells at various settings of numerical aperture and image depth. DETAILED DESCRIPTION

[0060] Figure 1 is a schematic diagram of a blood analyzer 1 for performing a method according to an embodiment of the present invention. The blood analyzer 1 comprises a detection volume 2 holding a prepared blood sample 3. The blood analyzer 1 further comprises an imaging system 4, which comprises a light source 5 illuminating the prepared blood sample 3 via a focusing lens 6. Furthermore, the blood analyzer 1 comprises an objective lens 7, which collects light from the prepared blood sample 3 and provides the collected light to a detector 8. It should be noted that even if Figure 1 Although only one focusing lens 6 is shown in FIG. 4 , it is not excluded that the imaging system 4 includes two or more focusing lenses 6 .

[0061] Figure 1 The blood analyzer 1 can be operated in the following manner. The prepared blood sample 3 is illuminated by the light source 5 and via the focusing lens 6. Therefore, the light of the light source 5 is focused at the focal plane within the prepared blood sample 3. During this time, image data is obtained based on the light from the prepared blood sample 3 through the imaging system 4, i.e., through the objective lens 7 and the detector 8. The numerical aperture of the illumination beam is smaller than the numerical aperture of the objective lens 7, thereby ensuring that the illumination beam has the acceptance light cone of the objective lens 7.

[0062] The acquired image data may include a plurality of images, each image being associated with an imaging plane corresponding to a depth of the prepared blood sample 3. In this case, the acquired image data forms a stack of images representing various depths of the prepared blood sample 3. As an alternative, the acquired image data may be associated with only one imaging plane, corresponding to only one depth of the prepared blood sample 3.

[0063] The acquired image data is then analyzed in order to determine at least the defocus direction in the image data. This can be done in the following manner. An object region is identified in the acquired image data, wherein the object region comprises a set of pixels in the image data that correspond to an object, such as a blood cell, in the prepared blood sample 3. Thus, an image representation of an object is identified in the image data, the object being a real physical object physically present in the prepared blood sample 3.

[0064] An optical signature of the identified object region is then identified, wherein the optical signature originates from a refractive index difference between the object and the medium of the prepared blood sample 3. The optical signature may be caused, for example, by diffraction. The optical signature acts as an additional artifact in the image data and, in contrast to the object, does not represent a real physical object that is physically present in the prepared blood sample 3.

[0065] Finally, based on the appearance of the identified optical features, at least a defocus direction in the image data is determined.

[0066] Figure 2 is an illustration of image data in the form of an image stack obtained as part of an embodiment of the present invention. The image data may be obtained, for example, by Figure 1 The images were obtained at 15 different depths of the prepared blood sample, arranged in 15 columns, and with 7 different numerical aperture settings of the illumination beam, arranged in 7 rows. Figure 2 Each of the 105 images shown in represents the depth and numerical aperture, which correspond to the column and row, respectively, at which the image is located.

[0067] Each image shows an object region representing an object in the prepared blood sample in the form of a blood cell 9. Thus, the images all show the same cell 9, but are obtained at various depths in the prepared blood sample and with various settings regarding the numerical aperture of the illumination beam. The numerical aperture of the objective of the imaging system is kept constant.

[0068] The blood cells 9 define a refractive index that is different from the refractive index of the surrounding medium of the prepared blood sample. Furthermore, the shape of the blood cells 9, combined with the difference in refractive index, introduces a lens effect in the sense that the blood cells 9 act as a convex lens. This lens effect appears in the image data in the form of a pair of annular features 10 that are circumferentially arranged relative to the blood cells 9. Therefore, the pair of annular features 10 do not represent real physical objects that are physically present in the prepared blood sample, but are additional artifacts in the image data that originate from the optical properties at the boundary between the blood cells 9 and the surrounding medium of the prepared blood sample, as described in detail above.

[0069] The pair of annular features 10 includes a first ring that appears dark and a second ring that appears bright. It can be seen that for the depth of the prepared blood sample, which is arranged above the focal plane of the system, that is, closer to the objective lens and the detector than the focal plane, the bright ring is arranged closer to the blood cells 9 than the dark ring. On the contrary, for the depth of the prepared blood sample arranged below the focal plane (that is, farther away from the objective lens and the detector than the focal plane), the order of the rings is reversed, that is, the dark ring is arranged closer to the blood cells 9 than the bright ring. Therefore, the mutual position of the pair of annular features 10 reveals whether a given image is obtained at a depth above or below the focal plane of the system. Therefore, by studying only the pair of annular features 10, the defocus direction of the image data can be easily derived, and this can be achieved by studying only the depth of the prepared blood sample 10. Figure 2 This is done by looking at one of the 105 images shown in .

[0070] It can be further seen that the pair of annular features 10 appear more "blurry" the further away from the focal plane. From this, information about the size or magnitude of the defocus can also be derived from the image data. Finally, the focal plane of the system can be identified as the depth where the dark and light rings switch, corresponding to the image marked by the square.

[0071] Furthermore, it can be seen that in images obtained with a low numerical aperture of the illumination beam, the above-mentioned effect is significantly more pronounced than in images obtained with a higher numerical aperture of the illumination beam. This is due to the fact that at low numerical apertures, the angular range emitted from the light source is small, so that the light reaching the prepared blood sample almost only comprises a component that is substantially perpendicular to the prepared blood sample. This makes the lens effect of the blood cells 9 appear clearly. Therefore, it is advantageous to apply a low numerical aperture of the illumination beam relative to the numerical aperture of the objective of the imaging system, because this will allow a fast, accurate, reliable and robust determination of the defocus direction and possibly the position of the focal plane.

[0072] Once the defocus direction is determined, as described above, a focused image of the blood cells 9 can be quickly and easily obtained, for example, by selecting an image obtained at the focal plane or by adjusting the settings of the blood analyzer until the blood cells 9 are in focus. Subsequently, an automated analysis of the prepared blood sample can be performed based on the focused image, in particular an automated analysis of the blood cells 9. Since the image data used for the analysis is ensured to be in focus, the analysis will be reliable and robust.

[0073] Figure 3 shows a simulation of an image of a blood cell 9 at various settings of numerical aperture and image depth, and shows the above reference Figure 2 The effect described. The blood cell 9 is simulated as an object in the shape of a hollow convex lens. In the simulated image, the dark ring 10a and the bright ring 10b can be clearly seen. It can also be seen that the rings 10a, 10b switch positions at the focal plane and the effect is most obvious at low numerical apertures.

Claims

1. A method for determining defocus in image data associated with a prepared blood sample by a blood analyzer comprising an imaging system and a detection volume, the method comprising the steps of: - arranging the prepared blood sample in the detection volume of the blood analyzer, - obtaining, by means of the imaging system of the blood analyzer, image data relating to the prepared blood sample, the image data comprising data relating to at least one imaging plane corresponding to a depth of the prepared blood sample, - Analyze the acquired image data, comprising the following steps: - identifying an object region in the obtained image data, the object region comprising a set of pixels in the image data, the set of pixels corresponding to an object physically present in the prepared blood sample, - identifying an optical feature of the identified object region, said optical feature originating from a refractive index difference between said object and a medium of said prepared blood sample and said optical feature serving as an additional artifact in said image data which is not representative of an object physically present in said prepared blood sample, and - Based on the appearance of the identified optical feature, determining at least a defocus direction in the image data.

2. The method according to claim 1, wherein: The step of obtaining image data associated with the prepared blood sample includes directing a light beam toward the prepared blood sample, the light beam having a numerical aperture that is less than a numerical aperture of an objective lens of the imaging system.

3. The method according to claim 2, wherein: The numerical aperture of the light beam is less than 80% of the numerical aperture of the objective lens of the imaging system.

4. The method according to claim 3, wherein: The numerical aperture of the light beam is between 5% and 50% of the numerical aperture of the objective lens of the imaging system.

5. The method according to claim 1, wherein: The step of obtaining image data includes obtaining at least one holographic image.

6. A method according to any one of the preceding claims, wherein: The subject is a blood cell.

7. A method according to any one of the preceding claims, wherein: The optical characteristic results from a lens effect produced by the object.

8. A method according to any one of the preceding claims, wherein: The step of identifying optical features of the identified object area comprises identifying a first annular feature and a second annular feature, and wherein the step of determining at least a defocus direction in the image data comprises determining a relative position of the first annular feature and the second annular feature.

9. A method according to any one of the preceding claims, wherein: The step of determining at least a defocus direction in the image data is further based on applying knowledge about optical properties of the object.

10. A method according to any preceding claim, further comprising the step of determining the magnitude of defocus in the image data.

11. A method according to any one of the preceding claims, wherein: The image data comprises a plurality of images, each image being associated with an imaging plane, the image data thereby forming an image stack associated with a plurality of imaging planes, and wherein the method further comprises the step of identifying an imaging plane among the plurality of imaging planes in which the image of the object is in focus.

12. The method according to any one of claims 1 to 10, wherein: The image data comprises data from only one imaging plane, and wherein the method further comprises the step of adjusting at least one setting of the imaging system based on the determined defocus direction.

Images