SYSTEMS AND METHODS FOR IMAGING

Abstract

A system for imaging a non-opaque object is provided. The system may include a lighting device for illuminating the object from a variety of positions, a magnifying device for receiving light that has passed through the object, an image sensor for imaging the object and generating an image of the object, and a processor. The processor can process the image data to compensate for the directional illumination of the object, compare the image data and retain only pixels whose values ​​are within a predefined tolerance, and generate an image of the object in a selected plane from the retained pixels. Furthermore, a processor-controlled method for processing image data of a non-opaque object is provided. Additionally, a calibration device for calibrating a system for imaging a non-opaque object is provided.

Description

1. AREA OF TECHNOLOGY

[0001] The invention relates to systems and methods for imaging. In particular, the invention relates to systems and methods for imaging non-opaque objects. 2. TECHNOLOGICAL BACKGROUND

[0002] The most important techniques for real-time imaging of living tissue are differential interference contrast microscopy and Hoffman modulation contrast microscopy. The strengths of these microscope designs lie in their ability to allow the operator to better visualize highly transparent living objects and their internal structures, such as blood cells, oocytes, and protozoa. A limitation of these microscopes is their high light energy consumption. For example, Hoffman modulation microscopes used for oocyte dissection are typically powered by a 100-watt white-light halogen lamp.

[0003] Laser confocal microscopy and light-sheet microscopy support 3D imaging, and fluorescence microscopy enables subdiffractive deconvolution and 3D imaging. Unfortunately, lasers and fluorescent dyes cause bleaching and cell damage, while light-sheet microscopy operates at a lower frame rate than HMC.

[0004] Overcoming the limitation of the large amount of light energy required for illumination in most forms of microscopy has generally been difficult, as only a small amount of light reaches the eye or camera. This is due to light loss in the optical path and the subtle attenuation or phase changes in the biological sample. This dim light for the viewer means that the image will exhibit noise and lower frame rates. Another major challenge is that computational photogrammetry relies on accurate per-pixel data captured from an object. The techniques used to enhance contrast in DIC and HMC render these methods unsuitable. Furthermore, deconvolution of the point scattering function, as performed computationally in fluorescence microscopy, for example, is impossible with these artifacts.An analytical challenge when using diffuse light sources, even with area light, in-phase or coherent light that is not point light, is that it is not possible to trace the path of the photons. Techniques such as GLIM are variants of quantitative phase imaging, which are slow to acquire and are not suitable for subdiffractive techniques. 3. SUBJECT OF TECHNOLOGY

[0005] The goal of the technology is to provide an improved system and / or method for imaging. Alternatively, a goal of the technology is to provide an improved computer-implemented method for processing image data of an object. Alternatively, a goal of the technology is to provide the public with at least a useful choice. 4. SUMMARY OF THE TECHNOLOGY

[0006] In imaging transparent objects using a light source that is essentially a single-point light source, it is known that all light incident on the sensor originates from this point. This allows the image to be used as a photometric source of information about all photons absorbed, reflected, or refracted across the surface of an imaged object. The Helmholtz reciprocity principle means that the photon path tracing concepts used for image reproduction can be applied in reverse. By switching the position of the light source, we can learn more about the path that the photons take through the molecules in the object on their way to the image sensor. For example, how any change in z through the object results in x / y changes in the image plane. In this way, we can identify the parts of the image below, on, and / or above the focal plane.This digital deconvolution can be used to improve HMC microscopes and mimic the focal plane sharpness of a confocal laser microscope or a light sheet microscope.

[0007] According to one aspect of the technology, a system for imaging an object, for example, a non-opaque object, is provided. The system may include one or more light sources, which can be considered point light sources for practical purposes. The system may include a light source that can be moved relative to the object, or multiple light sources in different positions relative to the object. The system may further include an image sensor, such as a camera, positioned on the side of the one or more light sources opposite the object. The system may also include a magnifying device positioned between the object and the image sensor. The system may be configured to allow movement between the object and the system.

[0008] According to one aspect of the technology, a method for imaging an object, for example, a non-opaque object, is provided. The method can include illuminating the object with one or more light sources and capturing images of the illuminated object with an image sensor.

[0009] According to one aspect of the technology, a computer-implemented method is provided for generating an image of an object, for example, a non-opaque object. The method may involve receiving a multitude of images of the object, with each image being captured by an image sensor as the object is illuminated by a light source in a different position. The method may further involve processing the images to generate an image of a plane within the object, for example, a portion of the object lying on a focal plane of a magnifying device.

[0010] According to one aspect of the technology, a system for imaging a non-opaque object is provided. The system may include an illumination device configured to illuminate the object from any of a multitude of positions on one side of the object. The system may further include a magnification device positioned to receive light from the illumination device that has passed through the object. The system may also include an image sensor configured to be positioned, when in use, to image the object through the magnification device. The image sensor may be configured to generate a multitude of image data sets. Each of the image data sets can represent an image of the object when illuminated from a particular position among the multitude of positions. The system may further include a processor.The processor can be configured to process multiple sets of image data to compensate for the directional illumination of the object. The processor can further be configured to compare these multiple sets of image data and retain only pixels whose values ​​are within a predetermined tolerance in each set. Finally, the processor can be configured to generate an image of the object in a selected plane from the retained pixels.

[0011] In certain embodiments, the processor can further be configured to apply a respective offset to each pixel in each of the sets of image data from a reference in the image before the comparison step. For each set of image data, the respective offset can be calculated from: 1) the respective position of the plurality of positions from which the object is illuminated to generate the respective set of image data; and 2) a distance of the selected plane from a focal plane of the magnifying device.

[0012] In certain embodiments, the respective offset can be zero if the selected plane is the focal plane of the magnification device.

[0013] In certain embodiments, the processor can also be configured to combine images of the object in a multitude of selected planes to create a three-dimensional image of the object.

[0014] In certain embodiments, the processor can also be configured to generate a density change map from the multitude of image data sets.

[0015] In certain embodiments, the lighting device can comprise a plurality of stationary light sources, each of the stationary light sources being located at one of the plurality of positions.

[0016] In certain embodiments, the lighting device may include one or more light sources configured to be moved to a multitude of positions.

[0017] In certain embodiments, the lighting device may comprise an arm and a plurality of light sources attached to the arm. The arm may be configured to rotate in order to move the plurality of light sources into different positions.

[0018] In certain embodiments, the illumination device can be configured such that one of the multitude of positions lies essentially on an optical axis of the magnifying device.

[0019] In certain embodiments, the lighting device can be configured to illuminate the object with essentially monochromatic light.

[0020] In certain embodiments, the lighting device can be configured to illuminate the object with essentially monochromatic light at a variety of different frequencies.

[0021] In certain embodiments, the system can be configured to change the position of a focal plane of the magnification device relative to the object.

[0022] In certain embodiments, the system can be configured to change the distance between the magnification device and the object.

[0023] According to one aspect of the technology, a processor-controlled method for processing image data of a non-opaque object is provided. The method can include receiving a multitude of image data sets. Each of the image data sets can represent an image of the object as captured by an image sensor when the object is illuminated from one of a multitude of positions. The method can further include processing the multitude of image data sets to compensate for the directional illumination of the object. The method can also include comparing the multitude of image data sets and retaining only those pixels whose values ​​are within a predetermined tolerance of each other in each set. The method can further include generating an image of the object in a selected plane from the retained pixels.

[0024] In certain embodiments, the method may further include, prior to the comparison step, applying a respective offset to each pixel in each of the sets of image data from a reference in the image. For each set of image data, the respective offset can be calculated from: 1) the respective position of the plurality of positions from which the object is illuminated to generate the respective set of image data; and 2) a distance of the selected plane from a focal plane of the magnifying device.

[0025] In certain embodiments, the respective offset can be zero if the selected plane is the focal plane of the magnification device.

[0026] In certain embodiments, the method may further include combining images of the object in a multitude of selected planes to create a three-dimensional image of the object.

[0027] In certain embodiments, the method may also include generating a map of density change from the multitude of sets of image data.

[0028] According to one aspect of the technology, a calibration device is provided for use in calibrating a system for imaging a non-opaque object. The calibration device may comprise a plurality of layers arranged horizontally in parallel. Each layer may be composed of one or more substantially transparent materials. Each layer may have a predetermined thickness. The calibration device may further comprise a plurality of markers. Each marker may be assigned to a specific layer within the plurality of layers, such that adjacent markers are vertically spaced by the thickness of one of the layers and horizontally offset from each other. The dimensions of each marker in the horizontal direction and the vertical position of each marker within the calibration device may be predetermined.

[0029] In certain embodiments, the thickness of one of the layers can differ from the thickness of another of the layers.

[0030] In certain embodiments, each of the markings can be configured so that it can be automatically identified by an optical recognition system.

[0031] In certain embodiments, each of the markings can comprise a different surface pattern.

[0032] In certain embodiments, each of the markings can be formed from a different material than the one or more materials from which the multitude of layers are formed.

[0033] Further aspects of the technology, which should be taken into account in all its novel aspects, will become clear to the person skilled in the art when reading the following description, which provides at least one example of a practical application of the technology. 5. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] One or more embodiments of the technology are described below by way of example only and without limitation with reference to the following drawings, in which: Fig. 1 a schematic representation of a system for imaging an object according to an embodiment of the technology; Fig. 2A-B are schematic representations of refraction in a medium and an object according to another embodiment of the technology; Fig. 3A-B are schematic representations of transmission through a medium containing an object according to another embodiment of the technology; Fig. 4A-C are schematic illustrations of refraction in a medium and an object from multiple light source positions according to another embodiment of the technology; Fig. Figures 5A-B are schematic illustrations showing circles of confusion in certain embodiments of the technology; Fig. Figures 6A-C are schematic illustrations showing circles of confusion in other embodiments of the technology; Fig. 7 is a schematic illustration of depicted planes according to a further embodiment of the technology; Fig. 8 is a schematic illustration of a lighting device according to a further embodiment of the technology; Fig. 9 a schematic illustration of a system for imaging an object and exemplary generated image data according to another embodiment of the technology; Fig. 10 is a schematic illustration of an exemplary image processing system according to another embodiment of the technology; Fig. 11 is an illustration of an exemplary method for processing image data according to a further embodiment of the technology; Fig. 12 is an illustration of two exemplary objects and exemplary image data generated from them; Fig. 13 is an illustration of an exemplary system for imaging an object according to a further embodiment of the technology; Fig. 14 an illustration of the exemplary system for imaging an object from Fig. 13 is; and Fig. Figure 15 is a schematic illustration of a calibration device for calibrating a system for imaging an object according to a further embodiment of the technology. 6. DETAILED DESCRIPTION OF EXAMPLE EXECUTIONS OF THE TECHNOLOGY

[0035] In summary, embodiments of the technology provide an imaging system 200 for illuminating an object 100 from a variety of positions with one or more light sources 212 and using a mathematical approach to solve the refractive light transport function per pixel. Digital deconvolution for subdiffractive imaging and a computational approach using multiple light passes can be used to isolate the focal plane, which can be combined with Z-stacking to create 3D voxel images of the object.

[0036] Implementations of the technology can use refractive imaging. Here, the change in the density of object 100 causes the light rays to bend towards the normal line as they travel from the surrounding medium to the denser object 100. The opposite occurs when the rays travel from the denser object back into the medium, and the light rays bend back again. Depending on the geometry of the object, this can lead to lensing effects, where the path of the light rays changes due to the changing shape or density of the object, resulting in a bending of the output rays and a reduction in light intensity in some areas. This is evident in the Fig. 2A and Fig. 2B shown schematically. Fig. 2A The light beam (black arrow) misses object 100 and penetrates the medium in which the object is suspended without any deviation other than refraction at the boundaries of the medium. Fig. 2B the light beam (black arrow) is deflected due to refraction when entering the denser object 100 and when exiting the object 100.

[0037] Refractive imaging is potentially easier to set up than brightfield imaging (LED, probe, magnifying lens, camera). In refractive imaging, the light is not refocused onto the image plane; instead, computational techniques are used to create a path tracing of the light source back to its point source origin and isolate the focal plane. Refractive imaging is primarily based on phase shifting and results in very high contrast with only minute phase changes. It can produce a 3D effect that accurately reflects the topology and, with coded illumination, potentially trace different paths through the molecules in the sample to create complex 3D voxel maps of changes in object density. Refractive imaging can also measure the amplitude (opacity) and correct these parts of the object to prevent them from distorting density calculations. 6.1. Object

[0038] The exemplary embodiments of the technology described in this specification can be used to image an object 100. Any suitable object 100 can be imaged, and unless expressly stated otherwise, the embodiments of the technology are not limited to a particular object or object type.

[0039] In certain embodiments, object 100 may not be opaque. That is, the object may be able to transmit light through at least parts of it. In some embodiments, the object may be transparent, i.e., it may transmit light without significant scattering, with the photons essentially obeying Snell's law. In other embodiments, the object may be translucent, i.e., it may transmit light, although some scattering may occur at the object's interfaces or within it. In some embodiments, the object may have some areas that are essentially transparent and some areas that are essentially translucent. It should be noted that some parts of the object may be opaque, even though a substantial part of the object may transmit light.

[0040] Examples of non-opaque objects that can be imaged using certain embodiments of the technology include biological material and living tissue. Imaging biological material and living tissue presents particular challenges due to the high transparency of cells, their three-dimensional structure, and their sensitivity to electromagnetic radiation such as ultraviolet and visible light. The core objectives are typically: 1) the individual imaging of individual cells and 2) Imaging of cells in situ, directly accessible through the pupil without surgical intervention, up to complex cells, neuronal photoreceptors and blood vessels.

[0041] To image largely transparent biological objects such as blood cells, egg cells and protozoa at high speed in three dimensions, the goal could be to use illumination energy that is three orders of magnitude lower and to have the capability to support fluorescence imaging.

[0042] Specific examples of biological material that can be imaged using certain embodiments of this technology include blood cells, egg cells, embryos, protozoa, tears, saliva, seminal fluid, urine, bile and other body fluids.

[0043] Implementations of the technology can also be used for imaging non-biological material, for example glass, optical components such as lenses, and lithography in transparent optical materials such as lithium niobate or silicon nitride, which are used in photonics. 6.2. Exemplary Systems

[0044] Embodiments of the technology provide a system 200 that may include an illumination device 210 and an imaging device 250. The imaging device 250 may include a magnification device 260 and an image sensor 270. Exemplary embodiments are described in the Fig. Figures 1, 4A to 4C, 5A, 5B and 8 illustrate this. These components will now be described in more detail using the figures. 6.2.1. Lighting device

[0045] In certain embodiments of the technology, the illumination assembly 210 is configured such that the object 100 can be illuminated from any of a plurality of positions on at least one side of the object. The plurality of positions can extend over an area in a plane perpendicular to the optical axis of the imaging device 250, which may be parallel to a focal plane of the magnifying device 260. For the purposes of this discussion, this plane can be considered to extend in the xy-plane, while the z-axis can be the optical axis. Illumination from a plurality of positions in the xy-plane can be achieved in various ways, for example: 1. The lighting device 210 can comprise a plurality of stationary light sources 212, each of the light sources 212 being located at one of the plurality of positions. In examples, the stationary light sources 212 can be arranged in a grid, each of the light sources having a known (x, y) position. In some embodiments, the light sources 212 can be arranged at regular intervals, which can facilitate the computation of the image data generated by the system. This arrangement requires no moving parts and allows for a rapid change of the lighting position, but requires more light sources, and the spatial resolution of the arrangement of light sources limits the number of positions from which the object 100 can be illuminated. 2. The lighting device 210 can comprise a single light source 212, and the lighting device 210 can be configured to move the single light source 212 to each of the plurality of positions. For example, the light source 212 can be mounted on a system configured to move the light source in the xy-plane. The system can be configured to move the light source 212 to a plurality of positions in a regular spatial pattern to support the computation. This arrangement can make it possible to illuminate the object 100 from any position in the xy-plane within the system's range of motion, but the change in the lighting position can be relatively slow compared to embodiments with a plurality of stationary light sources 212. 3. The lighting device 210 can comprise a plurality of light sources 212, and the lighting device 210 can be configured to move the plurality of light sources 212 to the plurality of positions. For example, the light sources 212 can be mounted on a system configured to move the light sources in the xy-plane. This option can provide some of the advantages of each of the previous options. An example of such an arrangement is shown schematically in Fig. Figure 8 illustrates this. In this embodiment, a plurality of light sources 212 are linearly mounted on a device in the form of an arm. The arm is configured such that it can rotate about a pivot point, which may be located essentially at one end of the arm. Consequently, each of the light sources 212 can be moved into a plurality of different positions. In the example of Fig. 8. The arm can be moved into eight different positions, arranged at intervals of 45° to each other. Consequently, the position of each light source in the xy-plane is known for each position of the arm. In other embodiments, the arm can be moved into a different number of positions, and a different number of light sources 212 can be mounted on the arm. In other embodiments, the arm can be moved in a different way, for example, it can be displaced in the xy-plane. In other embodiments, the light sources 212 can be mounted on a device having a different shape; and 4. The lighting device 210 may comprise a combination of stationary light sources 212, as described above in point 1, and movable light sources 212, as described above in point 2 or 3.

[0046] In certain embodiments of the technology, the plurality of positions for illuminating the object 100 can include a position that lies substantially on the optical axis of the imaging device 250, for example, the magnifying device 260. For example, a stationary light source 212 can be positioned substantially on the optical axis, or the illumination device 210 can be configured to move a light source 212 for illuminating the object onto the optical axis. In addition to providing another point from which the object 100 can be illuminated, a light source 212 located substantially on the optical axis can help calibrate the system 200 for the overall absorption of the light from the illumination device 210 by the object 100.

[0047] The lighting device 210 can be configured to illuminate the object 100 successively from each of the plurality of positions. That is, the object 100 can only be illuminated from one position at a time. In some embodiments, the lighting device can be configured to illuminate each of the light sources 212 in a predetermined sequence to achieve this.

[0048] In certain embodiments, the lighting device 210 can be configured to illuminate the object 100 with essentially monochromatic light. For example, the one or more light sources 212 can be essentially monochromatic. The monochromatic light sources may or may not be spatially coherent. In some embodiments, one or more light sources 212 can be used that emit light at a precisely known frequency.

[0049] In some embodiments, the illumination device 210 can be configured to illuminate the object with essentially monochromatic light at a variety of different frequencies. For example, the one or more light sources 212 can be configured to emit monochromatic light at adjustable frequencies. This can make it possible to capture multiple identical images of the object 100 at these different frequencies. Different light frequencies absorb and reflect light differently, so this approach can be used to categorize the atomic composition of some molecules and organic compounds.

[0050] In other embodiments, the lighting device 210 can be configured to illuminate the object 100 with light of a specific frequency range, for example, white light. The light can be passed through one or more filters to illuminate the object 100 with light of predetermined frequencies. For example, the lighting device 210 can include one or more frequency-pass beam splitters to isolate specific frequencies in the incident light. In such embodiments, the image sensor 270 can be configured to detect the relevant frequency ranges.

[0051] In certain embodiments, the one or more light sources 210 can closely resemble a point light source, i.e., the light sources 210 can be physically small. For example, LEDs can be used, or several switchable LED light sources can be used. In other embodiments, the one or more light sources 210 can comprise a special multi-frequency light source, such as a comb laser with controllable frequency isolation, and a diffraction grating to diffuse the photon beam into a cone to illuminate the object 100 and the image sensor 270.

[0052] The imaging mechanism of embodiments of the technology can be refractive imaging. In refractive imaging, the one or more light sources can provide extremely low illuminance levels (e.g., 1000 mcd or 0.2 lumens at 30 degrees), yet sufficient light can be supplied to the image sensor 270 because, unlike conventional imaging methods, the light sources 212 can be positioned to illuminate the image sensor 270 directly through the object 100. 6.2.2. Magnifying device

[0053] As shown in some of the figures, the system 200 can include a magnifying device 260. During the use of the system 200 to image an object 100, the magnifying device 260 can be positioned to receive light from the illumination device 210 that has passed through the object 100. For example, as shown in the Fig. As shown in Figures 1, 4A to 4C, 5A, 5B and 8, the magnifying device 260 can be positioned on one side of the object 100 opposite the illumination device 210, so that light passes through the object 100 to reach the magnifying device 260.

[0054] The magnifying device 260 can comprise any combination of optical components that serve to focus the object 100 onto the plane of the image sensor 270, for example, lenses, prisms, mirrors, and / or gratings. In certain embodiments, the magnifying device 260 can comprise a holographic element, a reflective, or a refractive magnifying device. In the examples of Fig. In 1, 4A to 4C, 5A, 5B and 8, a simple convex lens is used, although more complex devices may be used for magnification devices 260 in other embodiments. The magnification device 260 can be configured to focus the image with a suitable magnification to meet the required pixels per square dimension of the object 100. For example, a magnification that maps 230 square nanometers of the object 100 onto a square pixel detector in an image sensor may be the maximum magnification for blue light with a wavelength of 460 nm before the diffractive effects of reflective imaging (Abbé's law of diffraction) become dominant.

[0055] The magnifying device 260 can define a focal plane 262. In certain embodiments of the technology, the system 200 is configured to change the position of the focal plane 262 of the magnifying device 260 relative to the object 100. In some embodiments, this can be achieved by changing the distance between the magnifying device 260 and the object 100. For example, either the magnifying device 260 or the object 100, or both, can be mounted on a movable part, and the system 200, or a user thereof, can control the movable part to achieve a desired relative position between the magnifying device 260 and the object 100. It should be noted that the system 200 can be configured so that the image sensor 270 can be moved along with the magnifying device 260.In other embodiments, the focal plane 262 can be moved in a different way, for example by changing one or more optical components of the magnifying device 260, e.g. by replacing lenses or mirrors. 6.2.3. Image sensor

[0056] In certain embodiments of the technology, for example as in the Fig. As shown in Figures 1, 4A to 4C to 6 and 8, the system 200 can include an image sensor 270 to image the object 100 during use. Any suitable image sensor 270 can be used that detects light in the relevant frequency range(s) and produces image data that is representative of the detected light. The image sensor 270 can, for example, be a CCD or CMOS sensor. It should be noted that, unless the context clearly indicates otherwise, reference to an image in this specification may mean that the data recorded by the image sensor 270 constitutes an image and / or the visual representation of an image.

[0057] The image sensor 270 can be configured to generate a multitude of image data sets. Each of the image data sets can represent an image of the object 100 when illuminated by a light source 212 of the lighting device 210 in one of the multitude of positions from which the object 100 can be illuminated.

[0058] Reducing the integration time of the 270 image sensor can be a trade-off. This reduces the number of photons sampled by the 270 image sensor, which decreases the potential image quality, but more images can be captured faster, thus shortening the imaging time and also reducing object movement.

[0059] One feature of the system 200, according to certain embodiments of the technology, is that the illumination beneath the object 100 transmits the photons largely directly through the object 100 to the image sensor 270. This means that many photons can be collected, enabling a very short integration time. By using an illumination device comprising a multitude of stationary light sources 212 (as previously explained), it may be possible to switch light sources 212, such as a standard 26 milliampere LED, on and off within nanoseconds, e.g., about 5 nanoseconds. This is a much shorter time than an image sensor 270 is likely to be able to operate, but it means that it might be possible to achieve the maximum readout rates supported by the image sensor.For example, some CMOS image sensors might be able to achieve readout rates of 1 ms, allowing 1000 images per second to be sampled.

[0060] With a sufficient number of GPU units for image processing and Tensor Flow units for machine learning inference, it may be possible to process image stacks into voxel sets in real time. For a 4k sensor and a nine-disc voxel (e.g., four discs above and four discs below the focal plane) with eight rotation angles (more rotation angles increase x / y accuracy, but eight can be considered a good balance between computational power and accuracy) and four radial offset distances (more sampled radial offset distances increase vertical accuracy, and as a rule of thumb, the number of discs below the focal plane can be considered equal to 262), as in Fig. As shown in Figure 7, 32 images with a resolution of 3072 x 4096 pixels are generated. With an integration time of 1 ms, such an image stack can be captured in 32 ms. Alternatively, a 3D density map can be calculated at 31.25 images per second. 6.2.4. Image processing system

[0061] Fig. Figure 10 is a schematic representation of an exemplary image processing system 400 according to certain embodiments of the technology. The image processing system 400 can include a hardware platform 402 that manages the collection and processing of the data generated by the image sensor 270. The hardware platform 402 can include a processor 404, a memory 406, and other components typically found in such computer devices. The hardware platform 402 can be located locally at the image sensor 270 or remotely from the image sensor 270 and receive the image data via a suitable communication link, such as the network 416.In the illustrated exemplary embodiment of the technology, the memory 406 stores information that the processor 404 can access, the information comprising instructions 408 that can be executed by the processor 404 and data 410 that can be retrieved, processed, or stored by the processor 404. The memory 406 can be any suitable means known in the art that is capable of storing information in a manner accessible to the processor 404, including a computer-readable medium or any other medium that stores data that can be read by means of an electronic device.

[0062] The processor 404 can be any suitable device known to a person skilled in the art. Although the processor 404 and the memory 406 are illustrated as part of a single unit, it should be noted that this is not intended to be a limitation and that the functionality described herein can be performed by any number of processors and memories, which may or may not be located apart from each other or from the processing system 400. The instructions 408 can include any set of instructions suitable for execution by the processor 404. For example, the instructions 408 can be stored as computer code on the computer-readable medium. The instructions can be stored in any suitable computer language or computer format. Data 410 can be retrieved, stored, or modified by the processor 404 according to the instructions 410.The data 410 can also be formatted in any suitable computer-readable format. Again, it should be noted that although the data is illustrated as being stored in a single location, this is not to be understood as a limitation—the data can be stored in multiple storage locations or in multiple locations. The data 410 can also include a data set 412 containing control routines for aspects of the system 400.

[0063] The hardware platform 402 can communicate with a display device 414 to show the results of data processing. The hardware platform 402 can communicate via a network 416 with one or more other devices (e.g., user devices such as a tablet computer 418a, a personal computer 418b, or a smartphone 418c, or other devices including sensors) or with one or more server devices 420 that have associated memory 422 for storing and processing data collected by the local hardware platform 402. It should be noted that the server 420 and the memory 422 can take any suitable form known in the art, for example, a cloud-based distributed server architecture.The 416 network can encompass various configurations and protocols, including the Internet, intranets, virtual private networks, wide area networks, local area networks, private networks using proprietary communication protocols of one or more companies, whether wired or wireless, or a combination thereof.

[0064] After analyzing the data representing the movement of the eye 101, the image processing system 400 can be configured to output certain information obtained from the processing, examples of which are described in more detail below. The information can be output by the image processing system 400 by sending it over the network 416 or by outputting it via an output device, such as a display device 414, a tablet computer 418a, a personal computer 418b, or a smartphone 418c. Alternatively, the information can be stored in a memory, such as one or both of the memories 406 or 422, for later output by the image processing system 400.

[0065] In certain embodiments, the hardware platform 402 of the image processing system 400 can comprise a computer device, for example, a laptop or a PC. In other embodiments, the hardware platform 402 can comprise a plurality of computer devices configured to work together to perform the image processing. 6.3. Principles of Refraction

[0066] The following describes some principles of the refraction of light passing through a non-opaque object 100 and used in certain embodiments of the technology.

[0067] In Fig. 1a A light source 212 is used to illuminate an object 100 from one side, e.g., from below. This object 100 is then imaged from the other side, e.g., from above, using an imaging device 250, which includes an image sensor 270 and a magnification device 260, for example, to focus the object 100 onto the sensor plane with a suitable magnification, as explained previously. Fig. 3A and Fig. Figure 3B shows how the movement of the light source 212 in the x or y direction (i.e. perpendicular to the optical axis of the imaging device 260 / parallel to a focal plane of the imaging device 250) can change the angle of the light rays that interact with the object 100 with respect to the focal plane of the imaging device 250.

[0068] Furthermore, the Fig. 4A, Fig. 4B and Fig. 4C, how the movement of the light source 212 in the x- or y-direction (i.e., through positions a, b, and c) can change the perception of the three different sections of the object 100, labeled 1, 2, and 3, by the image sensor 270. Object sections 1 and 3 are somewhat "out of focus" because any object not located on the focal plane 262 will appear less sharp. For this reason, it may be advantageous for the imaging device 250 to include a relatively small aperture in order to provide a larger area of ​​acceptable sharpness in the vertical direction (z-axis).

[0069] When light source a is illuminated, the image generated on image sensor 270 results in object section 1 being located to the left of object section 2 and object section 3 being located to the right of object section 2. With the same vertical offset (Z-axis) between object sections 1 and 2, and between object sections 2 and 3, the apparent horizontal distance (xy-plane) between object section 1 and object section 2 is less than the apparent horizontal distance between object section 3 and object section 2. When light source c is illuminated, the image generated on image sensor 270 results in object section 1 being located to the right of object section 2 and object section 3 being located to the left of object section 2.With the same vertical offset (Z-axis) between object sections 1 and 2, as well as between object sections 2 and 3, the apparent horizontal distance (xy-plane) between object section 1 and object section 2 is again less than the apparent horizontal distance between object section 3 and object section 2.

[0070] This means that when object 100 is illuminated by light sources 212 whose positions in the xy-plane vary, the apparent horizontal distance for the same offset in the z-axis between features in object 100 that are closer to the imaging device 250 than the focal plane 262 and features in the focal plane 262 is smaller than the apparent horizontal distance between features that are farther from the imaging device 250 than the focal plane 262 and features in the focal plane 262. This can be described as the focal plane 262 acting as the pivot point for the formation of the refractive image on the image sensor 270 from the perspective of the light source 212.

[0071] Some non-opaque objects 100 can have different densities, and the changing densities can cause refraction of a light source 212. For example, organic objects such as cattle eggs / embryos are largely translucent and contain various molecular clusters such as lipids, proteins, polysaccharides, and membranes. These clusters can have different molecular densities, and the changing densities of these structures cause refraction. These clusters exist as three-dimensional structures stacked on top of each other, so that the light is refracted multiple times as it passes through the object 100, which can result in a final image on the image sensor 270 that can represent this multiple refraction as bright and dark areas. By changing the position of the light source, as in the Fig. 4A, Fig. 4B and Fig. As 4C is observed, these visual features change because the angle of the light passing through object 100 follows a different path. The number of samples required to reconstruct the layers of the structure depends on the depth and complexity of object 100. If the medium in which the object is suspended is known, a 3D voxel map of the estimated relative object densities can be extracted using a deconvolution approach and color-coded for the specific molecular density shown. This approach can rely on the fact that the contrast of the refraction resolved in the focal plane 262 is the highest contrast, and then estimate the relative motion of the subsequently blurred planes based on their displacement as the position of the light source 212, and thus the illumination angle, changes.

[0072] The Fig. 5A and Fig. Figure 5B shows how the imaging device 250 can be configured to reduce the convergence angle, for example, by selecting a magnification device 260 that focuses the light onto the image sensor 270 at a shallower angle. Increasing the distance between the magnification device 260 and the image sensor 270 allows a larger portion of the object 100 to remain in focus along the vertical (z-axis) direction. This is because, for the same vertical offset along the z-axis, the circle of confusion is smaller when the convergence angle is smaller.

[0073] Fig. Figure 6 shows that the circle of confusion on the image sensor 270 can be the same when the image sensor 270 is positioned the same amount closer to or further away from the focal plane 262 of the magnifying device 260; that is, the distances a and c are equal when the focal plane 262 is at the same vertical distance above or below the image sensor 270. In some embodiments of the technology, it may be useful to distinguish the layers above and below the focal plane 262, and a property of the focal plane pivot point can be used, as in the Fig. 2A and Fig. Figure 2B shows the image of the refraction occurring above the focal plane 262, resolved on the image sensor 270, which can be shifted in a positive relationship to the position of the point light source 212 and the resulting illumination angle. The image of the refraction occurring below the focal plane 262, also resolved on the image sensor 270, can be shifted by a larger ratio and inversely to the position of the point light source 212 and the resulting illumination angle. Since the circle of confusion increases linearly with the vertical distance (z) from the focal plane 262, the accuracy of the resolved image can be achieved closest to the focal plane 262, where the circle of confusion is smallest.

[0074] In some forms of technology, the goal may be to isolate the focal plane 262 as the most accurate representation possible and then to sample the underlying layers with increasingly larger vertical sampling, since there may be no computational value in sampling vertically less than the resolution of the current horizontal circle of confusion.

[0075] The result is that while the voxel x / y ratio is 1:1 to the pixels in the image sensor 270, this horizontal accuracy decreases with the circle of confusion, causing point sources to be shifted into larger patches. This can be useful in some cases, however, as deconvolution may be possible for fine structures. Meanwhile, the voxel height may be non-linear due to the nature of the larger patch size starting at the focal plane 262, with half the depth-of-field height determined by the angle of convergence. The height used to calculate the voxel image may be twice the previous vertical volume to accommodate the increase in the circle of confusion. Once the circle of confusion for a volume exceeds approximately 5% of the image area, the refraction image may become too blurry to be usable. Fig. Figure 7 shows the focal plane and voxel stack when the additional layers are added above and below.

[0076] In some shapes with a narrow convergence angle, it may be possible to slice eight or more vertical layers above and below the central focal plane layer. Furthermore, it may also be possible to configure the imaging device 250 to adjust the focal plane 262. By moving the focal plane 262 and repeating the process, additional voxel maps can be computed and subsequently combined to generate a voxel map with much higher x / y and vertical spatial accuracy. 6.4. Exemplary procedures

[0077] Exemplary methods for carrying out image processing according to certain forms of technology are now presented with reference to Fig. Figure 11 describes an exemplary procedure 500. Unless otherwise specified, it is understood that the processing steps can be performed by the image processing system 400 or a part thereof, for example the processor 404.

[0078] As a preparatory step for the in Fig. In the 11 illustrated method 500, the image sensor 270 images the object 100 through the magnification device 260 in the manner described above with reference to the exemplary systems 200. Consequently, the image sensor 270 can generate several sets of image data, each set representing an image of the object 100 as it is illuminated from one of the many positions from which the light sources 212 can illuminate the object. In step 501, the sets of image data are received by the image processing system 400, for example, by the processor 404. In step 502, the processor 404 can perform a process to compensate for the differences in the directional illumination of the object 100. This process will be described in more detail later. In step 503, the processor 404 can apply an offset to the pixels in each of the sets of image data.Whether this step is performed depends on which layer of the object is being mapped. If the focal plane 262 is being mapped, no offset is applied, and step 503 is not performed. In other words, the offset is zero. In step 504, processor 404 can compare the sets of image data and retain only pixels whose values ​​are within a predefined tolerance of each other in each set. In step 505, the image of object 100 in the selected layer can be generated based on the pixels retained in step 504. In step 506, the previous steps can be repeated for other selected layers in object 100. Steps 503 through 505 will now be explained in more detail. 6.4.1. Layer isolation

[0079] Fig. Figure 9 graphically supports the explanation of how to implement steps 503 and 504 in practice. This is shown in Fig. The system 200 shown in Figure 9 comprises an imaging device 250, which itself includes a magnifying device 260 and an image sensor 270. The system 200 further comprises an illumination device 210, which includes a plurality of light sources 210 at different positions on the xy-plane. An object 100 is positioned such that a section of it, referred to as the focal plane section 110, lies on the focal plane 262 of the imaging device 250. Fig. Figure 9 also shows two further sections of object 100, located above and below the focal plane 262 (i.e., optically closer and farther from the image sensor 270 than the focal plane 262), which are designated as upper voxel plane section 112 and lower voxel plane section 114, respectively. Sections 110, 112, and 114 may, for example, contain molecular structures.

[0080] Fig. Figure 9 illustrates how the sets of image data representing the focal plane section 110 and the upper and lower voxel plane sections 112 and 114 are isolated from the sets of image data acquired by the image sensor 270 by acquiring multiple images, each with a different position of the light source 212. This process utilizes the previously described pivot point around the focal plane 262, whereby molecular clusters causing refraction at a specific height above or below the focal plane 262 are mapped in the acquired image with a predictable horizontal x / y offset. By taking these offsets into account and combining the image data, it is possible to enhance the information about the refraction effects of the sections in the relevant plane. Only pixels in the image that correlate across all images (taking refraction effects into account) are considered to be present in that plane.

[0081] In certain embodiments of the technology, isolating the focal plane can be the first step. The position of the focal plane section 110 in the images does not change between images illuminated by light sources 212 in different positions (regardless of subtle changes in the angle of refraction). This is shown in image stack 2 in the bottom row of Fig. Figure 9 illustrates this. In this scenario, the image of focal plane section 110 can be the clearest and most focused part of the image. By examining the enhancing pixels across the set of images and removing all uncorrelated pixels (i.e., unenhanced values ​​of the upper and lower circles of confusion), focal plane section 110 can be isolated, and an image of focal plane section 110 can be generated. The step of retaining correlated pixels and removing uncorrelated pixels may involve applying a tolerance to the process of determining the correlated and uncorrelated pixels. It is understood that the pixel values ​​may refer to the intensity and / or color of the pixels, which in some forms may be the brightness of the pixel in grayscale and in others may be represented as a three-color vector, e.g., red-green-blue (RGB).The value of the predetermined tolerance applied to these pixel values ​​can be selected by the user based on the desired accuracy of the imaging process. The selected tolerance can take into account various relevant factors, such as the characteristics of the 270 image sensor, the noise generated in the image data, etc.

[0082] In some embodiments, the image of the focal plane section 110 can then be subtracted from the combined image data stacks and further layers (e.g., the upper and lower voxel planes 112 and 114) that were isolated in the same way as in image stacks 1 and 3 in Fig. 9 shown.

[0083] In certain embodiments, the process of isolating a section of the object 100 on a plane, such as the focal plane section 110, the upper voxel plane section 112, or the lower voxel plane section 114, may require applying a suitable x / y offset to each image from a reference in the image. The reference in the image may be the center of the image, for example, a center line or the midpoint. More precisely, the reference may correlate with the position of the optical axis in the imaging device 250. The position of the reference in the image may be determined from the orientation of the imaging device 250 with other components of the system 200.The appropriate offset to be applied to each pixel can be calculated from: 1) the position of the light source 212 used to illuminate the object 100 when the set of image data is acquired; and 2) the distance of the relevant plane from the focal plane 262 in the z-direction (i.e., parallel to the optical axis / perpendicular to the focal plane). Alternatively, a calibration process can be used to determine the appropriate offsets, as described later. In the special case of isolating the focal plane 262, no offset is required.

[0084] Consequently, for each plane selected for imaging, each image in the set of image data can be adjusted with respect to the x / y light angle and the z-offset of the plane from the focal plane 262 and then combined. Subsequently, a comparison between sets of image data can be performed, retaining only those pixels whose values ​​are within a predetermined tolerance of each other (i.e., step 504), in order to isolate the image of the selected plane section of object 100. 6.4.2. Compensation for directional lighting

[0085] Although steps 503 to 505 have been described, in some embodiments of the technology the insulation process may still need to consider the x / y position of the illumination. This is because if a structure in object 100, which has a different density than the surrounding material, is illuminated from one side, this will cause the light to be concentrated on one side of the structure and dispersed on the other. Which side is concentrated and which is dispersed can depend on both the relative density of the structure compared to the surrounding medium and the direction of the illumination. A less dense structure can be considered a downward slope of density, while a denser structure can represent an upward slope of density—thus changing which side of the "slope" is illuminated depending on the direction of the incident light.This can result in a bright edge on one side of the image in terms of structure and a dark edge on the other side of the image.

[0086] In some embodiments, this step 502 includes converting the light intensity in the acquired image from refraction to a map of relative density. Since this step may rely on photometry, where the intensity of pixels is measured and, in this case, correlated with the degree of positive or negative refraction, it may be important to correct for any opacities in the object 100. Some materials in the object 100 may be largely opaque and therefore reduce the intensity by blocking light transmission (rather than by changing the refraction). In some embodiments of the technology, one way to estimate the opacity of the object 100 on a pixel basis may be to illuminate the object 100 with a diffuse light source placed directly beneath it.This diffuse light penetrates the transparent sections of object 100, but causes the opaque sections to appear as shadows, resulting in a reduction of intensity.

[0087] Once the image has been compensated for the effects of opacity, from the perspective of the point light source, rays traced across the object image, where the underlying pixels are darker than the surrounding image level, can indicate an area of ​​increasing density, and pixels brighter than the surrounding image can indicate an area of ​​decreasing density. These intensity changes can become visible in the image at the boundary between two different densities of matter when illuminated by a point light source. With homogeneous density of matter, no change can occur, and the pixel values ​​can remain at the surrounding level. Fig. Figure 12 illustrates two objects 100 that can be imaged using certain embodiments of this technology: a cell wall in saline solution (image on the left) and a mineral oil bubble in water (image on the right). The dotted horizontal line in Fig. 12 marks a cross-section of objects 100. The lower graphics in Fig. Figure 12 shows the density of the object along the cross-section, and the upper graphs illustrate the change in refraction as a gradient of the lower density graphs. In some embodiments, the graphs are used to calculate the refraction in object 100.

[0088] The intensity in the captured images can be converted into a change in the refraction / density value by tracing the light rays from the illumination source across the object and creating a new density map. In this map, values ​​increase when the "slant" is upward and decrease when the "slant" is downward. This means that this relative density map produces an illumination-independent representation of the relative densities that resulted in the captured image. This illumination direction compensation step can be performed in step 502 in some embodiments of the technology. From the perspective of the traced ray, there can be a gradient representing the change in the object's density. The overall length and intensity of the change from the color in the middle region to edge dispersion or edge concentration can be fed into the perturbation.The more intense the change in color in the middle area, the higher the upward or downward slope, and the longer the change, the longer the slope.

[0089] In some embodiments, a general approach is to identify the position of the light sources 212 in space relative to the object 100. This is typically several orders of magnitude away from the focal plane section 110 of the object 100 (e.g., the focal plane section 110 may be 3 micrometers deep, and the LED may be, for example, 5000 micrometers below it and 5000 micrometers horizontally offset from the focal plane), so that the light may have already traveled a long distance relative to the object, and the rays may be essentially parallel. For these calculations, the difference in z across the object may be minimal, and x / y may be the only component that matters in this analysis.

[0090] As in Fig. Figure 13 shows a light source 212, which is positioned on one side of the object 100 (in the case of Fig. 13 above the sample), light rays illuminating object 100 from this position outside the screen. Then, each light source 212 and its position in the image set that needs to be compensated for is taken into account; this consideration can be carried out, for example, from any light source direction and any light source distance from object 100.

[0091] For a selected image trace, conceptual rays are cast across the captured image, taking into account the light position relative to the object (in some embodiments, there may be a counter per pixel, so that pixels are not processed more than once). Since the algorithm considers every image pixel along each ray, a parallel value can be updated in an array to store the relative density map. In this map, the height can be increased for a dark area and decreased for a bright area, based on the relative change from the mean color of the image. This process of addition and subtraction can perturb the pixel grid. The change in the density map can then be further processed to relax the map. This can effectively blend values ​​with neighboring pixels, and there are many suitable relaxation strategies that can be used, such as...The process involves averaging the NSEW pixels, averaging the entire 8 pixels surrounding the pixels, or even larger Gaussian kernels. This process can then be iterated and repeated, effectively "bloating up" the density values. The number of iterations can be chosen for each specific implementation, but in some embodiments, the values ​​asymptotically approach stability once the distance to the greatest slope is reached.

[0092] In some embodiments, this process can inflate the values ​​of the density change map, so that areas of the object that are denser than the surrounding medium may have a higher value, and areas that are less dense may have a lower value. Areas that have the mid-color of the undisturbed light may not change in density.

[0093] This density change map now consists of relative values, with the elements in the scene no longer exhibiting directional illumination. Elements outside the focal plane are still shifted in the X / Y direction based on the illumination angle and their distance from the focal plane. This compensated map may therefore be suitable for performing the subsequent steps in the plane isolation process (i.e., steps 503 to 506 described earlier). This is because, instead of considering the directional effects of density change from different illumination angles, the underlying density change was calculated in these embodiments, and this data was made comparable regardless of direction.

[0094] When this data is combined with the change in the density / refraction map, the focused portion of the object is enhanced 100%, but the features outside the focal plane remain unchanged because they move in the image data when the angle of the illumination light changes.

[0095] Fig. Figure 14 shows a further illustration of the process just explained. 6.4.3. Calibration procedure & device

[0096] It was explained above that in certain embodiments of the technology, an x / y offset can be applied to each image to isolate a selected plane, for example, if the selected plane is not the focal plane 262. While the offset can be calculated, in some embodiments a calibration process is performed to determine the appropriate offsets. The calibration process can be performed instead of, or in addition to, calculating the offsets in various embodiments.

[0097] To support the calibration process, a calibration device 300 can be provided, exemplified in Fig. Figure 15 shows that the calibration device 300 can be designed so that the system 200 has the necessary information to measure the effects of the illumination configuration, the imaging assembly (e.g., a microscope objective), the light path, and the image sensor. This allows complex theoretical calculations to be avoided and calibration parameters to be measured empirically.

[0098] In some embodiments, such as the one illustrated, the calibration device 300 can take the form of a calibration slide. The calibration slide can be considered a three-dimensional ruler. The calibration slide can comprise a body which itself includes a plurality of horizontally parallel layers, for example, three layers 310a, 310b, 310c. Each layer 310 can be formed of a substantially transparent material (for example, epoxy resin or glass), and the material used to form each layer can be the same or different from each other. Each layer 310 can have a known thickness, for example, 10 micrometers. In some embodiments, the thickness of each layer 310 can be different.

[0099] Each layer 310, for example on, below, or within each layer, can be assigned a marker 320a, 320b, 320c. Each marker 320 can be assigned to one of the layers 310 such that adjacent markers 320 are vertically spaced by the thickness of one of the layers. Each marker 320 can have one or more properties. An example property is that each marker 320 can have predetermined dimensions in the x and / or y direction (i.e., parallel to the focal plane 262 during use). Each marker 320 can also, or alternatively, be configured, i.e., it can include a predetermined shape and / or decoration, so that it can be easily recognized either by a human user and / or automatically by an optical recognition system. For example, the markers 320 can be square in the top view (i.e., in the xy direction).The markings 320 may also or alternatively comprise a surface pattern, for example, a microdot pattern, with predetermined properties, and each marking 320 may comprise a different pattern to identify the marking (and the associated layer). Each marking 320 may also or alternatively be formed from a material with a predetermined density, which may be a material with a different density than the material from which the layers 310 of the slide are made, for example, epoxy resin on glass. The markings 320 associated with each layer 310 may be offset in the x / y direction by a predetermined amount. The markings 320 may be spaced apart in the z direction by the predetermined thickness of the respective layers 310.

[0100] To calibrate the system 200, the imaging device 250 (either automatically or manually) can be focused on the marker 320b of the middle layer 310b, for example, using the edges of the marker and / or its pattern, which it may have as a contrast detection feature, while adjusting the position of the focal plane in the z-direction to provide contrast for the middle marker. Next, the calibration device 300 is illuminated from various positions, for example, the light sources 212 are illuminated sequentially, and the change in position of the markers 320a and 320b of the upper and lower layers 310a and 310b can be determined. Since the exact dimensions (i.e., z, x, and y details) of the calibration device 300 are known, the ratios for the position of each light source 212 relative to the pixels can be calculated.The System 200 can then be used to scan objects other than Objects 100 and to understand exactly how large features are in the x / y plane and how much z-offset a given x / y deflection causes for a feature (and vice versa).

[0101] It should be noted that the form and configuration of the above described and in Fig. The calibration devices 300 shown in Figure 15 are only examples. Other configurations, such as different patterns and shapes, multiple shapes, and multiple layers, can also be provided in a calibration device 300 in other embodiments of the technology. In some embodiments, a calibration device 300 with n markers can be formed with the same number of layers, i.e., n layers, as shown in Figure 15. Fig. 15 described. In other embodiments, a calibration device 300 with n markings can be configured with one fewer layer than the number of markings, i.e., n - 1 layers, wherein one of the markings is positioned on the surface opposite one of the layers to another marking. In this way, the markings can still be separated by the thickness of one of the layers. 6.4.4. Multi-plane imaging

[0102] Regarding step 503 in Fig. Section 11 explained how an offset can be applied to the pixels in the captured images to isolate an image of a section of object 100 that is offset in the z-direction relative to the focal plane 262, e.g. the upper voxel plane section 112 or the lower voxel plane section 114.

[0103] In some embodiments of the technology, one or more additional sections of object 100 can be imaged by alternatively or additionally changing the position of the focal plane 262 of the magnifying device 260 relative to object 100 such that, in the new position, the intended section of object 100 lies on the focal plane 262. The method for imaging the focal plane section 110 of object 100 can then be applied to image this additional section of object 100. Exemplary options for changing the position of the focal plane 262 of the magnifying device 260 relative to object 100 have been described above.Consequently, taking into account factors such as accuracy, speed and size, different sections of an object 100 can be imaged by isolating a section of the object on the focal plane 262, isolating sections of the object 100 offset from the focal plane 262 and / or changing the position of the focal plane 262 relative to the object 100. 6.4.5. Combining images from multiple layers

[0104] In certain embodiments, after repeated execution of steps 501 to 506 of method 500, a number of images of different sections of object 100 in the z-direction are obtained, the different sections corresponding to the selected planes for imaging. In some embodiments, these images can be combined in step 507 to generate a three-dimensional image of object 100 or of sections thereof. Fig.Figure 7 conceptually illustrates how these image layers can be arranged to form a voxel stack. 6.5. Other remarks

[0105] Unless the context clearly requires otherwise, the terms “include”, “comprehensive” and similar terms throughout the description and claims are to be understood in an inclusive sense and not in an exclusive or exhaustive sense, i.e., in the sense of “including but not limited to”.

[0106] All disclosures of all above and below-mentioned applications, patents and publications, if any, are hereby incorporated by reference.

[0107] References to the prior art in this description are not to be understood as an acknowledgment or indication that this prior art is part of the general knowledge in this field in any country of the world.

[0108] The technology can also be described more generally as the parts, elements and features that are individually or collectively referred to or specified in the description of the application, in any or all combinations of two or more of these parts, elements or features.

[0109] Where the foregoing description refers to integers or components with known equivalents, these integers are included herein as if they were listed individually.

[0110] It should be noted that various changes and modifications to the currently preferred embodiments described herein will be obvious to those skilled in the art. Such changes and modifications can be made without departing from the spirit and scope of the technology and without impairing its associated advantages. It is therefore intended that such changes and modifications be included in the present technology.

Claims

[1] System for imaging a non-opaque object, the system comprising: a lighting device configured to illuminate the object from any of a multitude of positions on one side of the object; a magnifying device positioned to receive light from the illumination device that has passed through the object; an image sensor configured to be positioned during use to image the object through the magnifying device, wherein the image sensor is configured to generate a plurality of sets of image data, each set of image data representing an image of the object when illuminated from a respective position of the plurality of positions; and a processor configured to: a) processes the multitude of sets of image data to compensate for the directional illumination of the object; b) compares the multitude of sets of image data and retains only pixels whose values ​​lie within a predetermined tolerance to each other in each of the sets; and c) creates an image of the object in a selected layer from the retained pixels. [2] System according to claim 1, wherein the processor is further configured to apply a respective offset to each pixel in each of the sets of image data from a reference in the image prior to step b), wherein for each set of image data the respective offset is calculated from: 1) the respective position of the plurality of positions from which the object is illuminated to generate the respective set of image data; and 2) a distance of the selected plane from a focal plane of the magnification device. [3] System according to claim 2, wherein the respective offset is zero when the selected plane is the focal plane of the magnification device. [4] System according to any one of claims 1 to 3, wherein the processor is further configured to combine images of the object in a plurality of selected planes to generate a three-dimensional image of the object. [5] System according to any one of claims 1 to 4, wherein the processor is further configured to generate a density change map from the plurality of sets of image data. [6] System according to any one of claims 1 to 5, wherein the lighting device comprises a plurality of stationary light sources, each of the stationary light sources being located at one of the plurality of positions. [7] System according to any one of claims 1 to 6, wherein the lighting device comprises one or more light sources configured to be movable to a plurality of positions. [8] System according to claim 7, wherein the lighting device comprises an arm and a plurality of light sources attached to the arm, the arm being configured to rotate in order to move the plurality of light sources into different positions. [9] System according to any one of claims 1 to 8, wherein the illumination device is configured such that one of the plurality of positions lies substantially on an optical axis of the magnification device. [10] System according to any one of claims 1 to 9, wherein the lighting device is configured to illuminate the object with substantially monochromatic light. [11] System according to claim 10, wherein the lighting device is configured to illuminate the object with substantially monochromatic light at a variety of different frequencies. [12] System according to any one of claims 1 to 11, wherein the system is configured to change the position of a focal plane of the magnification device relative to the object. [13] System according to claim 12, wherein the system is configured to change the distance between the magnification device and the object. [14] Processor-implemented method for processing image data of a non-opaque object, the method comprising: a) Receiving a plurality of sets of image data, each set of image data representing an image of the object as captured by an image sensor when the object was illuminated from a particular position of a plurality of positions; b) Processing the multitude of sets of image data to compensate for the directional illumination of the object; c) Comparing the multitude of sets of image data and retaining only those pixels whose values ​​lie within a predetermined tolerance of each other in each of the sets; and d) Generating an image of the object in a selected layer from the retained pixels. [15] Method according to claim 14, wherein the method further comprises, prior to step c), applying a respective offset to each pixel in each of the sets of image data from a reference in the image, wherein for each set of image data the respective offset is calculated from: 1) the respective position of the plurality of positions from which the object is illuminated to generate the respective set of image data; and 2) a distance of the selected plane from a focal plane of the magnification device. [16] Method according to claim 15, wherein the respective offset is zero when the selected plane is the focal plane of the magnification device. [17] Method according to any one of claims 14 to 16, wherein the method further comprises combining images of the object in a plurality of selected planes to produce a three-dimensional image of the object. [18] Method according to any one of claims 14 to 17, wherein the method further comprises generating a map of density change from the plurality of sets of image data. [19] Calibration device for use in the calibration of a system for imaging a non-opaque object, the calibration device comprising: a plurality of horizontally parallel layers, each layer being formed from one or more substantially transparent materials and each layer having a predetermined thickness; and a plurality of markings, each marking being assigned to a respective layer of the plurality of layers, such that adjacent markings are vertically spaced apart by the thickness of one of the layers and adjacent markings are horizontally offset from each other, wherein the dimensions of each of the markings in the horizontal direction and the vertical position of each of the markings are predetermined in the calibration device. [20] Calibration device according to claim 19, wherein the thickness of one of the layers differs from the thickness of another of the layers. [21] Calibration device according to one of claims 19 to 20, wherein each of the markings is configured to be automatically identified by an optical recognition system. [22] Calibration device according to one of claims 19 to 21, wherein each of the markings comprises a different surface pattern. [23] Calibration device according to one of claims 19 to 22, wherein each of the markings is formed from a material other than the one or more materials that form the plurality of layers.

Images