Imaging device and method
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- LEICA MICROSYSTEMS CMS GMBH
- Filing Date
- 2023-07-25
- Publication Date
- 2026-08-03
AI Technical Summary
Fluorescence microscopy is time-consuming and stressful for samples due to sequential imaging of different fluorophores, which requires repeated exposures.
An imaging device that excites multiple fluorophores simultaneously and uses image segmentation to distinguish features marked with the same fluorophore, reducing exposure and imaging time.
Faster imaging with reduced stress on the sample by illuminating once and using image segmentation to differentiate features marked with the same fluorophore, resulting in clearer images with less computational effort.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an imaging device. Furthermore, the present invention relates to a method and a computer program product for imaging a sample. [Background technology]
[0002] In fluorescence microscopy, fluorophores are used to mark features of a sample, such as cell walls or cell nuclei. Fluorophores can be excited by excitation light, typically light of a particular wavelength, to emit fluorescence. This fluorescence may be used to generate an image of the sample. By marking a feature in the sample with a fluorophore, the feature is made visible for fluorescence imaging. Typically, each feature of the sample is marked with a different fluorophore to distinguish them. For example, all cell walls in a sample are marked with one fluorophore and all cell nuclei are marked with a different fluorophore. Different fluorophores have different emission properties, which allow the fluorophore, and thus the feature to which it is attached, to be identified.
[0003] Typically, the different fluorophores are imaged sequentially, meaning that each of the different fluorophores is excited and imaged individually in a separate imaging step. However, sequential imaging takes time and causes stress on the sample due to the repeated exposures. Another way to distinguish between the different fluorophores is to separate the fluorescence emitted by the sample into different spectral channels, where each spectral channel contains a different wavelength band. However, this does not reduce the stress on the sample, because each of the different fluorophores still needs to be excited individually. Summary of the Invention [Problem to be solved by the invention]
[0004] It is therefore an object of the present invention to provide an imaging device and a method for imaging a sample which allows for fluorescence imaging of a sample faster and with less stress caused to the sample. [Means for solving the problem]
[0005] The above mentioned object is achieved by the subject matter of the independent claims. Preferred embodiments are defined in the dependent claims and the following description.
[0006] The proposed imaging device for imaging a sample includes an excitation unit configured to emit excitation light for exciting fluorophores attached to a first feature of the sample and at least one second feature of the sample. A detection unit of the imaging device is configured to receive fluorescence from the excited fluorophores and generate at least one fluorescence image from the received fluorescence. The imaging device further includes a controller. The controller is configured to determine, based on the image segmentation, a first image area of the fluorescence image corresponding to the first feature and a second image area of the fluorescence image corresponding to the second feature, generate the first image based on the first image area, generate the second image based on the second image area, and / or generate a composite image including at least the first image area and the second image area.
[0007] The imaging device is configured to image the first feature and the second feature using fluorescence imaging. To image the first feature and the second feature, the first feature and the second feature are first marked with the same fluorophore. The fluorophore is then excited with excitation light to emit fluorescence. The fluorescence is received by a detection unit, which generates a fluorescence image. The fluorescence image includes both the first feature and the second feature. To distinguish the first feature and the second feature, an image segmentation is performed. Based on this image segmentation, the controller determines a first image area corresponding to the first feature and a second image area corresponding to the second feature. The controller then generates a first image including the first feature and a second image including the second feature. Alternatively or additionally, the controller generates a composite image including both the first feature and the second feature. In the proposed imaging device, the first feature and the second feature are imaged in a single imaging step. This is significantly faster than imaging the first feature and the second feature consecutively. Furthermore, the sample is illuminated only once with the excitation light, resulting in less light exposure and less stress on the sample.
[0008] The imaging device may be configured to image additional features of the sample. Some of the additional features may be marked with the same fluorophore as the first feature and the second feature. For each additional feature marked with the same fluorophore, the controller is configured to determine an additional image region corresponding to the additional feature based on the image segmentation.
[0009] According to one embodiment, the controller is configured to perform image segmentation based on prior knowledge of the first feature and / or the second feature. In this embodiment, the features of the sample are identified by known characteristics. For example, the features may be identified by their shape, size and / or structure. The features may be identified by the intensity of the emitted fluorescence, since some biological structures affect the emission properties of fluorophores. In particular, the controller is configured to perform image segmentation using a database comprising known characteristics of at least the first feature and the second feature. Advantageously, the controller is configured to perform image segmentation without using machine learning. Performing image segmentation based on known characteristics is significantly faster and requires less computational effort than other methods of image segmentation.
[0010] According to another embodiment, the detection unit is configured to detect a plurality of fluorescence emission signals from the received fluorescence. The controller may be configured to perform image segmentation by selectively assigning each of the fluorescence emission signals to a first image area associated with the first feature or a second image area associated with the second feature. Each of the fluorescence emission signals corresponds to a plurality of photons of the received fluorescence originating from the same vicinity of the sample. The fluorescence image may be generated by combining the fluorescence emission signals. In this embodiment, each of the first image area and the second image area includes a portion of the plurality of fluorescence emission signals. The image segmentation is performed by identifying which fluorescence emission signal corresponds to the fluorescence emitted from the first structure and which fluorescence emission signal corresponds to the fluorescence emitted from the second structure. This identification may in particular be based on which image area the adjacent fluorescence emission signals correspond to. This identification may be based on an emission characteristic of the fluorescence, for example intensity.
[0011] According to another embodiment, the fluorescent image includes a plurality of pixels. Each pixel can correspond to one or more of the fluorescent emission signals. The controller may be configured to determine, based on the image segmentation, which pixels of the fluorescent image correspond to the first image region and which pixels of the fluorescent image correspond to the second image region. In this embodiment, each of the first image region and the second image region includes a collection of pixels of the fluorescent image. Since each pixel may correspond to more than one fluorescent emission signal, some of the pixels may be part of both the first image region and the second image region. In such a case, the controller may determine that the pixel in question is assigned to either the first image region or the second image region. The assignment may be based, for example, on the intensity or photon count of the fluorescent emission signal. The assignment may also be based on which image region the neighboring pixels correspond to. A plurality of pixels is a simple way of storing image data in an easily retrievable manner. However, alternatives exist. For example, the fluorescent image may be a volumetric image including voxels or a point cloud, where each point of each voxel or point cloud corresponds to one or more fluorescent emission signals.
[0012] According to another embodiment, the first image and the second image are false color images. The first image has a first color and the second image has a second color, which is different from the first color. In this embodiment, the first image and the second image are provided in different colors. This helps the user to distinguish the first feature from the second feature, thereby facilitating the use of the imaging device.
[0013] According to another embodiment, the composite image is a false color image. A first image area in the composite image has a first color and a second image area in the composite image has a second color. A first feature has a first color in the composite image, whereas a second feature has a second color in the composite image. This assists a user to distinguish between the first feature and the second feature in the composite image, thereby facilitating use of the imaging device.
[0014] According to another embodiment, the excitation unit is configured to emit a second excitation light for exciting a second fluorophore attached to a third feature of the sample. The detection unit is configured to separate the received fluorescence into at least two spectral channels, a first spectral channel corresponding to a first wavelength band comprising at least a portion of an emission spectrum of the first fluorophore and a second spectral channel corresponding to a second wavelength band comprising at least a portion of an emission spectrum of the second fluorophore. The detection unit is further configured to generate a first fluorescence image based on the fluorescence received in the first spectral channel and to generate a second fluorescence image based on the fluorescence received in the second spectral channel. Preferably, the first wavelength band and the second wavelength band do not overlap.
[0015] In this embodiment, the imaging device is configured to also image a third feature using fluorescence imaging. The first feature and the second feature are marked with the same first fluorophore, while the third feature is marked with a second fluorophore different from the first fluorophore. The two fluorophores are then excited by an excitation unit to emit fluorescence. The fluorescence is received by a detection unit and separated into a first spectral channel and a second spectral channel. A first fluorescence image is generated based on the first spectral channel and includes the first feature and the second feature. A second fluorescence image is generated based on the second spectral channel and includes the third feature. From the first fluorescence image, a first image region and a second image region are determined based on image segmentation. This allows for a distinction between the first feature and the second feature even if they are imaged in the same spectral channel. Preferably, image segmentation is not performed for the second channel. The first image region and the second image region form, so to speak, two virtual spectral channels. This allows the imaging device to optimally use the available spectral channels. Even though the sample is illuminated twice, the light exposure is still less than if the first and second features were marked with different fluorophores, because the two different fluorophores must be excited by different excitation light.
[0016] The imaging device according to this embodiment may in particular be used to observe a third feature in the second spectral channel using the first and second features as background. For example, when a user is observing a sample, many features of the sample, such as cell walls, cell nuclei and certain organelles, are already known and of little interest. However, these known features, i.e., the first and second features, can serve as background for a feature of interest, i.e., a third feature that may be observed. This makes the feature of interest easily localizable within the sample.
[0017] According to another embodiment, the optical detection unit includes at least two detector elements and a beam splitter configured to direct the received fluorescence having a wavelength within a first wavelength band to the first detector element and to direct the received light having a wavelength within a second wavelength band to the second detector element. In this embodiment, the beam splitter, the first detector element and the second detector element are used as a means for generating the first and second spectral channels. Compared to other means for generating the first and second spectral channels, the use of a beam splitting element and the first and second detector elements is easy to implement, cost-effective and reliable.
[0018] According to another embodiment, the controller is configured to generate a composite image including at least a first image area, a second image area, and based on the second fluorescent image. The first image area has a first color, the second image area has a second color, and the second fluorescent image has a third color, the third color being different from the first color and the second color. Each of these features is present in a different color in the composite image. This assists a user to distinguish different features in the composite image, thereby facilitating the use of the imaging device.
[0019] According to another embodiment, the controller is configured to determine a background region of the fluorescence image based on at least the first image region and the second image region, and remove the background region from the first image, the second image, the composite image, the first fluorescence image, and / or the second fluorescence image. The background region may be a region of the sample that is less interesting to the user. The background region may also be a region in the first fluorescence image and / or the second fluorescence image that is highly noisy. In this embodiment, the background region is removed based on the first feature and the second feature. In particular, the controller removes all fluorescence emission signals or pixels that do not correspond to either the first feature or the second feature. For example, a certain polymer used as a substrate for mounting a sample emits fluorescence in the same wavelength range as a certain fluorophore. For example, polyethylene naphthalate (PEN) foil is used as a substrate for laser microdissection. However, PEN emits fluorescence in the same wavelength range as DAPI when using illumination light having a wavelength of 365 nm. By identifying the first feature and the second feature stained with DAPI in the first fluorescent image and removing the background, the contribution of the autofluorescence of the PEN substrate to the first fluorescent image is removed. This allows the imaging device to use a given polymer substrate in fluorescent imaging, making it even more versatile. Overall, removing the background regions results in a clearer image with less clutter.
[0020] According to another embodiment, the imaging device is a microscope. Preferably, the optical detection unit includes at least one microscope objective lens oriented to the target area and configured to receive fluorescence from the sample. The microscope provides a magnification that allows a user to see details of the sample that are not visible to the naked eye. However, the imaging device is not limited to a microscope. For example, the imaging device may be a slide scanner, a light sheet system, a flow cytometer, or any other imaging device suitable for fluorescence imaging.
[0021] The invention also relates to a method for imaging a sample, the method comprising the steps of: exciting fluorophores attached to a first feature of the sample and at least one second feature of the sample, receiving fluorescence from the excited fluorophores and generating at least one fluorescence image from the received fluorescence, determining a first image region of the fluorescence image corresponding to the first feature and a second image region of the fluorescence image corresponding to the second feature based on image segmentation, generating a first image based on the first image region, generating a second image based on the second image region, and / or generating a composite image including at least the first image region and the second image region.
[0022] The method has the same advantages as the imaging device described above and can be supplemented using the features of the dependent claims directed to the imaging device.
[0023] According to one embodiment, the method comprises the additional steps of: The method includes the steps of staining the first feature and the second feature with a first fluorophore and staining the third feature with a second fluorophore, separating the received fluorescence into at least two spectral channels, a first spectral channel corresponding to a first wavelength band including at least a portion of the emission spectrum of the first fluorophore and a second spectral channel corresponding to a second wavelength band including at least a portion of the emission spectrum of the second fluorophore, generating a first fluorescence image based on the fluorescence received in the first spectral channel and generating a second fluorescence image based on the fluorescence received in the second spectral channel. Preferably, image splitting is not performed for the second channel. The first image area and the second image area form two virtual spectral channels, which increases the number of available spectral channels. In this embodiment, optimal use of the available spectral channels is made.
[0024] According to another embodiment, the method is part of an experiment, where possible changes in the first and second features are less relevant to the experiment than changes in the third feature. In this embodiment, the third feature is observed in the second spectral channel using the first and second features as background. For example, the first and second features may already be known and / or may be of less interest to the experiment. However, the first and second features may still be used as background for the feature of interest, i.e. the third feature to be observed. In this way, the first and second features provide, so to speak, a roadmap that allows the third feature to be easily located in the sample.
[0025] The invention further relates to a computer program product comprising program code configured to perform the above-mentioned method, when the computer program product is executed on a processor.
[0026] This computer program product has the same advantages as the imaging device and method described above and can be supplemented using the features of the dependent claims directed to the imaging device and the method for controlling an imaging device, respectively.
[0027] In the following, specific embodiments will be described with reference to the drawings. [Brief description of the drawings]
[0028] [Figure 1] 1 is a schematic diagram of an imaging device according to one embodiment. [Diagram 2] 2 is a flow chart of a method for imaging a sample using the imaging device according to FIG. 1 . [Diagram 3] 2 is a schematic diagram of a first fluorescent image produced by the imaging device according to FIG. 1; [Figure 4] FIG. 4 is a schematic diagram of a first image area of a first fluorescent image according to FIG. 3; [Diagram 5] FIG. 4 is a schematic diagram of a second image area of the first fluorescent image according to FIG. 3; [Figure 6] 2 is a schematic diagram of a second fluorescent image produced by the imaging device according to FIG. 1; [Figure 7] 2 is a schematic diagram of a composite image produced by the imaging device according to FIG. 1; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0029] FIG. 1 is a schematic diagram of an imaging apparatus 100 for imaging a sample 102, according to one embodiment.
[0030] The sample 102 is illustratively shown as a biological sample. The sample 102 may be a tissue section, such as a liver or brain section, among others. A first fluorophore is attached to a first feature 104 of the sample 102 and a second feature 106 of the sample 102. In FIG. 1, the first feature 104 is illustratively selected from a cell wall, and the second feature 106 is illustratively selected from a cell nucleus. A second fluorophore is attached to a third feature 108 of the sample 102.
[0031] The imaging device 100 includes an excitation unit 110 configured to emit an excitation light 111. In particular, the excitation unit 110 is configured to emit a first excitation light for exciting a first fluorophore and a second excitation light for exciting a second fluorophore. In FIG. 1, the first and second excitation lights are indicated by the same reference number 111 for clarity. The excitation unit 110 may include, in particular, one or more laser light sources for generating the excitation light 111. For example, one laser light source each may be provided for the first and second excitation lights 111. Alternatively, a single broadband laser light source configured to generate broadband laser light may be provided. The first and second excitation lights 111 may then be generated from the broadband laser light, for example by means of a filter, in particular by means of an acousto-optical tunable filter (AOTF). The excitation unit 110 may further include an optical element configured to direct the first and second excitation lights 111 to the sample 102. In particular, the excitation unit 110 may include a scanning unit configured to selectively direct the first and second excitation lights 111 to different regions of the sample 102 .
[0032] The detection unit 112 of the imaging device 100 includes a microscope objective lens 114, a beam splitter 116, a first detector element 118a, and a second detector element 118b. The microscope objective lens 114 is directed toward the sample 102 and configured to receive the fluorescence 120 from the excited fluorophore and direct the fluorescence 120 to the beam splitter 116. The beam splitter 116 is exemplarily formed as a dichroic beam splitter. The beam splitter 116 is configured to direct the fluorescence 120 having a wavelength within a first wavelength band to the first detector element 118a and direct the fluorescence 120 having a wavelength within a second wavelength band to the second detector element 118b. The first wavelength band includes at least a portion of the emission spectrum of the first fluorophore. Preferably, the first wavelength band includes an emission maximum of the first fluorophore. Similarly, the second wavelength band includes at least a portion of the emission spectrum of the second fluorophore. Preferably, the second wavelength band includes the emission maximum of the second fluorophore. Two spectral channels are formed by splitting the fluorescent light 120 between the first detector element 118a and the second detector element 118b. More spectral channels may be easily formed by providing more beam splitters and detector elements.
[0033] Because most of the fluorescence 120 emitted by the first fluorophore is captured by the first detector element 118a, the first feature 104 and the second feature 106 are imaged by the first detector element 118a. Similarly, most of the fluorescence 120 emitted by the second fluorophore is captured by the second detector element 118b, and the third feature 108 is imaged by the second detector element 118b. In other words, the first feature 104 and the second feature 106 are imaged in the first spectral channel, and the third feature 108 is imaged in the second spectral channel.
[0034] The fluorescence 120 detected by the first detector element 118a and the second detector element 118b is converted into a fluorescence emission signal. Each fluorescence emission signal corresponds to one or more photons of the fluorescence 120. The detection unit 112 forms a first fluorescence image 300 (see FIG. 3) from the fluorescence emission signal generated by the first detector element 118a, and forms a second fluorescence image 600 (see FIG. 6) from the fluorescence emission signal generated by the second detector element 118b. The first fluorescence image 300 includes a first feature 104 and a second feature 106. The second fluorescence image 600 includes a third feature 108. The first and second fluorescence images 600 are particularly monochrome images. In this embodiment, the first fluorescence image 300 and the second fluorescence image 600 include a plurality of pixels, each pixel corresponding to one or more of the fluorescence emission signals. The first fluorescent image 300 and the second fluorescent image 600 are described in more detail below with reference to Figures 3 and 6, respectively.
[0035] In this embodiment, the imaging device 100 is exemplarily formed as a microscope, more specifically as a wide-field microscope configured to simultaneously image the entire observation area of the sample 102. The imaging device 100 may also be formed as a scanning microscope configured to scan the observation area of the sample 102 in successive imaging steps. In particular, the imaging device 100 may be formed as a confocal microscope. However, the imaging device 100 is not limited to being a microscope.
[0036] 1, the beam path of the excitation light 111 and the beam path of the fluorescence light 120 do not overlap. However, the imaging device 100 may be configured such that the beam path of the excitation light 111 and the beam path of the fluorescence light 120 at least partially coincide. For example, the excitation light 111 may be directed to the sample through a microscope objective lens 114.
[0037] The imaging device 100 further includes a controller 122 and an output unit 124. The controller 122 includes a memory element 126 and is configured to control the excitation unit 110, the detection unit 112 and the output unit 124. The controller 122 is further configured to execute a method for imaging the sample 102. The method includes, in particular, the controller 122 determining a first image area 104' corresponding to a first feature 104 in the first fluorescence image 300 and a second image area 106' corresponding to a second feature 106 in the first fluorescence image 300. The method is described in more detail below with reference to Figures 2 to 7.
[0038] In FIG. 1 , the output unit 124 exemplarily displays a composite image 128 generated by the controller 122 based on the first image region 104′, the second image region 106′, and the second fluorescent image 600. The first image region 104′ corresponds to the first feature 104, i.e., the cell wall of the sample 102, and is shown in FIG. 1 by a dashed line. The second image region 106′ corresponds to the second feature 106, i.e., the cell nucleus of the sample 102, and is shown in FIG. 1 by a dotted line. The third feature 108 is exemplarily a filament of the sample 102, and is shown in FIG. 1 by a solid line.
[0039] FIG. 2 is a flow chart of a method for imaging a sample 102 using the imaging device 100 according to FIG.
[0040] The method may in particular be part of an experiment where possible changes in the first feature 104 and the second feature 106 are less relevant to the experiment than changes in the third feature 108. In such an experiment, the first feature 104 and the second feature 106 may already be well known and / or of less interest compared to the third feature 108. In such a case, the first feature 104 and the second feature 106 are used as a background against which changes in the third feature 108 are observed.
[0041] The process starts in step S200. In step S202, the first feature 104 and the second feature 106 are stained with a first fluorophore, and the third feature 108 is stained with a second fluorophore. In step S204, the controller 122 controls the excitation unit 110 to emit first and second excitation lights 111, thereby exciting the first fluorophore and the second fluorophore. The first and second excitation lights 111 may be emitted simultaneously or sequentially in any order. In step S206, the controller 122 controls the detection unit 112 to receive fluorescence 120 from the excited fluorophore, for example by focusing the microscope objective lens 114 on the sample 102. In step S208, the detection unit 112 separates the received fluorescence 120 into a first spectral channel and a second spectral channel. In step 210, the controller 122 controls the detection unit 112 so that a first fluorescent image 300 is generated from the fluorescent light 120 detected by the first detector element 118a, and a second fluorescent image 600 is generated from the fluorescent light 120 detected by the second detector element 118b. If the imaging device 100 is a wide-field microscope, the first fluorescent image 300 and the second fluorescent image 600 are images captured by the first detector element 118a and the second detector element 118b, respectively, in a single imaging step. If the imaging device 100 is a scanning microscope, the first fluorescent image 300 and the second fluorescent image 600 are generated from fluorescent emission signals generated in multiple consecutive imaging steps.
[0042] In step S212, the controller 122 performs image segmentation in the first spectral channel based on prior knowledge of the first feature 104 and the second feature 106. In this step, the first feature 104 and the second feature 106 are identified by known characteristics, such as their shape and / or size. In this embodiment, these known characteristics constitute the prior knowledge. The controller 122 can look up these known characteristics in a database, which may be stored in the storage element 126 of the controller 122. The database may also be stored in one or more remote storage elements and accessed via a computer network. The controller 122 does not use machine learning algorithms to perform the image segmentation. Using known characteristics for image segmentation is faster, less computationally intensive, and does not require training. The image segmentation may be performed on the first fluorescence image 300. Alternatively, the controller 122 performs image segmentation on the fluorescence emission signal generated by the first detector element 118a. In the latter case, step S212 may be performed before the first fluorescence image 300 is generated in step S210.
[0043] In step S214, the controller 122 determines a first image region 104' and a second image region 106' based on the image segmentation. The first image region 104' corresponds to the first feature 104, and the second image region 106' corresponds to the second feature 106. In step S216, the controller 122 generates a first image 400 (see FIG. 4) based on the first image region 104' and generates a second image 500 (see FIG. 5) based on the second image region 106'. The first image 400 shows the first feature 104 and will be described in more detail below with reference to FIG. 4. The second image 500 shows the second feature 106 and will be described in more detail below with reference to FIG. 5. The first image 400 and the second image 500 may be generated by the controller 122 as monochrome images or as false color images. Alternatively or additionally, in step S216, the controller 122 generates a composite image 128 including at least the first image region 104' and the second image region 106'. The composite image 128 may include the second fluorescent image 600 such that the composite image 128 shows the first feature 104, the second feature 106 and the third feature 108. In optional step S218, the controller 122 determines a background region and removes the background region from the first image 400, the second image 500, the composite image 128, the first fluorescent image 300 and / or the second fluorescent image 600. In particular, the controller 122 assigns all fluorescent emission signals detected by the first detector element 118a that do not correspond to either the first feature 104 or the second feature 106 to the background region. In another optional step S220, the controller 122 displays the first fluorescence image 300, the second fluorescence image 600, the first image 400, the second image 500 and / or the composite image 128 to a user via the output unit 124. The process ends in step S222.
[0044] FIG. 3 is a schematic illustration of a first fluorescent image 300 produced by the imaging device 100 according to FIG.
[0045] The first fluorescent image 300 includes a plurality of pixels, each pixel corresponding to a fluorescent emission signal detected by the first detector element 118a. In FIG. 3, the first fluorescent image 300 is inverted for clarity. This means that darker pixels correspond to stronger fluorescent emission signals. For the same reason, in FIG. 3, not all instances of the first feature 104 and the second feature 106 are marked with reference characters. The first detector element 118a detects only the fluorescent light 120 emitted by the first fluorophores attached to the first feature 104 and the second feature 106. Thus, as can be seen in FIG. 3, the first fluorescent image 300 includes the first feature 104, exemplarily shown as a cell wall, and the second feature 106, exemplarily shown as a cell nucleus.
[0046] FIG. 4 is a schematic illustration of a first image area 104' of the first fluorescent image 300 according to FIG.
[0047] The first image 400 was generated by the controller 122 based on the first image area 104' and shows only the first feature 104, i.e. the cell wall. The first image 400 includes pixels of the first fluorescent image 300 that correspond to the first image area 104'. In FIG. 4, the first image 400 is inverted for clarity. For the same reason, not all instances of the first feature 104 are marked with a reference number in FIG. 4. Although FIG. 4 is a monochrome image, the first image 400 may be a false color image, preferably a false color image having a single color.
[0048] FIG. 5 is a schematic illustration of the second image area 106' of the first fluorescent image 300 according to FIG.
[0049] The second image 500 has been generated by the controller 122 based on the second image area 106' and shows only the second feature 106, i.e. the cell nuclei. The second image 500 includes pixels of the first fluorescent image 300 that correspond to the second image area 106'. In FIG. 5, the second image 500 has been inverted for clarity. For the same reason, in FIG. 3, not all instances of the second feature 106 are marked with reference numbers. Although FIG. 5 is a monochrome image, the second image 500 may be a false color image, preferably having a single color different from the single color of the first image 400.
[0050] FIG. 6 is a schematic illustration of a second fluorescent image 600 produced by the imaging device 100 according to FIG.
[0051] The second fluorescence image 600 includes a plurality of pixels, each pixel corresponding to a fluorescence emission signal detected by the second detector element 118b. In FIG. 6, the second fluorescence image 600 is inverted for clarity. For the same reason, in FIG. 6, not all instances of the third feature 108 are marked with a reference number. The second detector element 118b detects only the fluorescence 120 emitted by the second fluorophore attached to the third feature 108. Thus, as can be seen in FIG. 3, the second fluorescence image 600 includes the third feature 108, exemplarily shown as an organelle.
[0052] FIG. 7 is a schematic illustration of a composite image 700 produced by the imaging device 100 according to FIG.
[0053] The composite image 700 is generated by the controller 122 based on the first image area 104', the second image area 106' and the second fluorescent image 600. In other words, the composite image 700 shows the first feature 104, the second feature 106 and the third feature 108. In FIG. 7, the composite image 700 is inverted for clarity. For the same reason, in FIG. 3, not all instances of the first feature 104, the second feature 106 and the third feature 108 are marked with reference numbers. Although FIG. 7 is a monochrome image, the composite image 700 is preferably a false color image. For example, the first feature 104 may be shown in blue, the second feature 106 in red, and the third feature 108 in yellow.
[0054] In all figures, identical or similar operating elements are provided with the same reference numerals. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ". All individual features of the embodiments and combinations of individual features of the embodiments with each other are considered to be disclosed as well as in combination with individual features or groups of features in the foregoing description and / or claims.
[0055] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a step or feature of a step, and similarly, aspects described in the context of a step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
[0056] Some embodiments relate to a microscope including a system as described in relation to FIG. 1. Alternatively, the microscope may be part of or connected to a system as described in relation to FIG. 1. FIG. 1 shows a schematic diagram of a system 100 configured to perform the methods described herein. The system 100 includes a microscope 112 and a computer system 122. The microscope 112 is configured to capture images and is connected to the computer system 122. The computer system 122 is configured to perform at least some of the methods described herein. The computer system 122 may be configured to execute machine learning algorithms. The computer system 122 and the microscope 112 may be separate entities, but may be integrated in one common housing. The computer system 122 may be part of a central processing system of the microscope 112 and / or the computer system 122 may be part of a subordinate part of the microscope 112, such as a sensor, actor, camera or lighting unit of the microscope 112.
[0057] The computer system 122 may be a local computing device (e.g., a personal computer, laptop, tablet computer, or mobile phone) with one or more processors and one or more storage devices, or may be a distributed computing system (e.g., a cloud computing system with one or more processors and one or more storage devices distributed at various locations, such as local clients and / or one or more remote server farms and / or data centers). The computer system 122 may include any circuit or combination of circuits. In one embodiment, the computer system 122 may include one or more processors, which may be of any type. As used herein, a processor may contemplate any type of computing circuit, such as, but not limited to, a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), a multi-core processor, a field programmable gate array (FPGA), or any other type of processor or processing circuit, for example, of a microscope or a microscope component (e.g., a camera). Other types of circuits that may be included in computer system 122 may be custom circuits, application specific integrated circuits (ASICs), such as one or more circuits (such as communications circuits) used in wireless devices such as cell phones, tablet computers, laptop computers, two-way radios, and similar electronic systems. Computer system 122 may also include one or more storage devices, which may include one or more memory elements suitable for a particular application, such as main memory in the form of random access memory (RAM), one or more hard drives and / or one or more drives handling removable media, such as compact discs (CDs), flash memory cards, digital video discs (DVDs), and the like.Computer system 122 may also include a display device, one or more speakers and a keyboard and / or controller which may include a mouse, a trackball, a touch screen, a voice recognition device, or any other device that enables a user of the system to input information to and receive information from computer system 122.
[0058] Some or all of the steps may be performed by (or using) a hardware apparatus, such as, for example, a processor, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, any one or more of the crucial steps may be performed by such an apparatus.
[0059] Depending on certain implementation requirements, the embodiments of the present invention can be implemented in hardware or software. The implementation can be performed by a non-transitory recording medium, such as a digital recording medium, for example a floppy disk, a DVD, a Blu-ray, a CD, a ROM, a PROM and EPROM, an EEPROM or a FLASH memory, on which electronically readable control signals are stored, which cooperate (or can cooperate) with a programmable computer system to implement the respective methods. Thus, the digital recording medium can be computer readable.
[0060] Some embodiments of the present invention include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to perform any of the methods described herein.
[0061] Generally, embodiments of the present invention can be implemented as a computer program product comprising program code which is operable to perform any of the methods when the computer program product is run on a computer, the program code may for example be stored on a machine readable carrier.
[0062] Another embodiment comprises the computer program for performing any of the methods described herein, stored on a machine readable carrier.
[0063] In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing any of the methods described herein, when the computer program runs on a computer.
[0064] Therefore, another embodiment of the present invention is a recording medium (or data carrier or computer readable medium) containing a computer program stored thereon for performing any of the methods described herein when executed by a processor. The data carrier, digital recording medium or recording medium is typically tangible and / or non-transitory. Another embodiment of the present invention is an apparatus as described herein, including a processor and a recording medium.
[0065] A further embodiment of the invention is therefore also a data stream or a sequence of signals representing the computer program for performing any of the methods described herein, the data stream or the sequence of signals being for example adapted to be transmitted via a data communication connection, for example the Internet.
[0066] Another embodiment comprises a processing means, for example a computer, or a programmable logic device configured to or adapted to perform any of the methods described herein.
[0067] Another embodiment comprises a computer having the computer program installed thereon for performing any of the methods described herein.
[0068] Another embodiment of the invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for implementing any of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a storage device, etc. The apparatus or system may, for example, include a file server to transfer the computer program to the receiver.
[0069] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform any of the methods described herein. In general, the methods are advantageously performed by any hardware apparatus. [Explanation of symbols]
[0070] 100 Imaging device 102 Sample 104,106,108 Features 104',106' Image area 110 Excitation Unit 111 Excitation light 112 Detection Unit 114 Microscope Objective Lens 116 Beam splitter 118a, 118b Detector element 120 Fluorescence 122 Controller 124 output units 126 Memory Element 128 Composite Images 300,400,500,600,700 images
Claims
1. An imaging device (100) for imaging a sample (102), wherein the imaging device (100) is An excitation unit (110) configured to emit excitation light (111) for exciting a phosphor attached to a first feature (104) and at least one second feature (106) of the sample (102), A detection unit (112) is configured to receive fluorescence (120) from an excited phosphor and generate at least one fluorescence image from the received fluorescence (120), Controller (122) and Includes, The controller (122) is configured to determine, based on image segmentation, a first image region (104') of the fluorescence image corresponding to the first feature (104) and a second image region (106') of the fluorescence image corresponding to the second feature (106), generate a first image (400) based on the first image region (104'), generate a second image (500) based on the second image region (106'), and / or generate a composite image (128, 700) including at least the first image region (104') and the second image region (106'). Imaging device (100).
2. The controller (122) is configured to perform the image segmentation based on prior knowledge of the first feature (104) and / or the second feature (106). The imaging device (100) according to claim 1.
3. The detection unit (112) is configured to detect multiple fluorescence emission signals from the received fluorescence (120), The controller (122) is configured to perform the image segmentation by selectively assigning each of the fluorescence emission signals to the first image region (104') associated with the first feature (104) or the second image region (106') associated with the second feature (106). The imaging device (100) according to claim 1 or 2.
4. The fluorescence image includes a plurality of pixels, each of which corresponds to one or more of the fluorescence emission signals, and the controller (122) is configured to determine, based on the image division, the pixels of the fluorescence image corresponding to the first image region (104') and the pixels of the fluorescence image corresponding to the second image region (106'). The imaging device (100) according to claim 3.
5. The first image (400) and the second image (500) are false-color images, the first image (400) has a first color, and the second image (500) has a second color, the second color being different from the first color. The imaging device (100) according to claim 1 or 2.
6. The composite image (128,700) is a false-color image, the first image region (104') in the composite image (128,700) has a first color, and the second image region (106') in the composite image (128,700) has a second color. The imaging device (100) according to claim 1 or 2.
7. The excitation unit (110) is configured to emit a second excitation light (111) to excite a second phosphor attached to the third feature (108) of the sample (102), The detection unit (112) is configured to separate the received fluorescence (120) into at least two spectral channels: a first spectral channel corresponding to a first wavelength band including at least a portion of the emission spectrum of a first phosphor, and a second spectral channel corresponding to a second wavelength band including at least a portion of the emission spectrum of a second phosphor; generate a first fluorescence image (300) based on the fluorescence (120) received in the first spectral channel; and generate a second fluorescence image (600) based on the fluorescence (120) received in the second spectral channel. The imaging device (100) according to claim 1 or 2.
8. The optical detection unit (112) At least two detector elements, A beam splitter (116) configured to orient received fluorescence (120) having a wavelength within the first wavelength band to a first detector element (118a), and to orient received light having a wavelength within the second wavelength band to a second detector element (118b), Includes, The imaging device (100) according to claim 7.
9. The controller (122) is configured to generate the composite image (128,700) based on the second fluorescence image (600) and includes at least the first image region (104') and the second image region (106'), wherein the first image region (104') has a first color, the second image region (106') has a second color, and the second fluorescence image (600) has a third color, the third color being different from the first and second colors. The imaging device (100) according to claim 7.
10. The controller (122) is configured to determine the background region of the fluorescence image based on at least the first image region (104') and the second image region (106'), and to remove the background region from the first image (400), the second image (500), the composite image (128, 700), the first fluorescence image (300), and / or the second fluorescence image (600). The imaging device (100) according to claim 1 or 2.
11. The imaging device (100) is a microscope. The imaging device (100) according to claim 1 or 2.
12. A method for imaging a sample (102), the method comprising the following steps, namely, A step of exciting the phosphor attached to the first feature (104) and at least one second feature (106) of the sample (102), The steps include receiving fluorescence (120) from the excited phosphor and generating at least one fluorescence image from the received fluorescence (120), A step of determining, based on image segmentation, a first image region (104') of the fluorescence image corresponding to the first feature (104) and a second image region (106') of the fluorescence image corresponding to the second feature (106), A step of generating a first image (400) based on the first image region (104'), generating a second image (500) based on the second image region (106'), and / or generating a composite image (128,700) including at least the first image region (104') and the second image region (106'), A method that includes this.
13. The above method includes the following additional steps, namely, The steps include staining the first feature (104) and the second feature (106) with a first phosphor, and staining the third feature (108) with a second phosphor, The steps include separating the received fluorescence (120) into at least two spectral channels, namely a first spectral channel corresponding to a first wavelength band including at least a portion of the emission spectrum of the first phosphor, and a second spectral channel corresponding to a second wavelength band including at least a portion of the emission spectrum of the second phosphor, The steps include generating a first fluorescence image (300) based on the fluorescence (120) received in the first spectral channel, and generating a second fluorescence image (600) based on the fluorescence (120) received in the second spectral channel, including, The method according to claim 12.
14. The above method is part of the experiment, and the possible changes in the first feature (104) and the second feature (106) are less relevant to the experiment than the changes in the third feature (108). The method according to claim 13.
15. It is a computer program, The computer program includes program code configured to perform the method described in any one of claims 12 to 14 when executed on a processor, Computer program.