Method for recording merged microscope image, microscope system and computer program product
Patent Information
- Application Number
- JP2024059074
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-04-03
- Filing Date
- 2024-04-01
- Publication Date
- 2024-11-28
- Estimated Expiration
- 2044-04-01
AI Technical Summary
Existing microscopes suffer from limited image section and shallow depth of field due to optical limitations, leading to blurred images from out-of-focus radiation, with existing methods like confocal microscopy being time-consuming and HiLo microscopy requiring multiple image captures.
A method involving structured and Gaussian illumination to capture two images, one with structured and one with uniform illumination, which are fused to create a composite image, allowing simultaneous or sequential imaging to enhance image clarity and coverage.
This approach enables fast, cost-effective imaging with high optical quality and large image section, discarding out-of-focus information to achieve sharp, high-resolution images quickly, especially suitable for thick samples.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a method, a microscope system, and a computer program product for recording a composite microscope image. [Background technology]
[0002] A large number of different microscopes are known for a very wide range of applications, which can differ considerably in terms of their optics and their performance.
[0003] A drawback of microscopes is that the image section that can be captured in one image is very small.
[0004] Also, a commonality that optical systems generally have is that the depth of field of microscopic imaging is very shallow due to the laws of optics, which may lead to image capture being disturbed by radiation occurring outside the focal plane, which is later perceived as a blurred luminous phenomenon on the captured image, even though only a single focal plane is observed.
[0005] Various approaches to solving this problem are already known in the state of the art.
[0006] In a confocal microscope, only a part of the object to be imaged is illuminated at a time. The sample is then scanned one by one. The reflected or fluorescent light from the sample is measured to its position, and an entire sample is constructed based on it. A pinhole aperture is attached to the beam path of the reflected or fluorescent light, which passes only the light from the sharply imaged area. This effectively reduces the effects of interference from radiation from layers outside the focal plane. The disadvantage of this setup is that it takes a long time to create a single image by scanning the object.
[0007] On the other hand, HiLo microscopy is also known as an alternative method. This method is described, for example, in Lim et al., Optics Letters-33:1819-1821 (2008) and Jerome Mertz, Nature Methods-8:811-819 (2011). The principle behind this imaging method is that two images are imaged. In HiLo microscopy, the first image with so-called speckle illumination is effectively used as a filter to depict the low frequencies (Lo) in the image and therefore the background. The second image (Hi), created with Gaussian illumination, contains the image information that is in focus. By using the first image as a filter, it is possible to eliminate interfering radiation from planes outside the focal plane. Summary of the Invention [Problem to be solved by the invention]
[0008] The object of the present invention is to at least partially eliminate the above-mentioned drawbacks known from the prior art, in particular to provide a method, a microscope system and a computer program product for recording composite microscope images, which allow fast and cost-effective imaging with a particularly high optical quality, including in particular image sharpness, and at the same time with as large an image section as possible, in particular an image section of the entire sample. [Means for solving the problem]
[0009] The above problem is solved by a method having the features of claim 1, a microscope system having the features of claim 14 and a computer program product having the features of claim 15. Further features and details of the invention will become apparent from the respective dependent claims, the description and the drawings. Features and details explained in relation to the method according to the invention naturally also apply in relation to the microscope system according to the invention and / or the computer program product according to the invention and vice versa. Thus, reference to the individual aspects of the invention in relation to the present disclosure may be made or always made mutually.
[0010] According to a first aspect of the present invention, there is provided a method for recording a synthetic microscope image of a sample, the method comprising: microscopic imaging, in particular by means of an optical imaging device, of at least the first fused image and the second fused image, - acquiring a first image, the sample being illuminated in a structured manner, in particular by a structured illumination device; taking a second image, the sample being illuminated uniformly, in particular by a Gaussian illumination device; - fusing a first image and a second image, the first image being used for low frequency image information located at the focus by applying a low pass filter and the second image being used for high frequency image information located at the focus by applying a high pass filter; wherein the second fused image depicts a different location on the sample than the first fused image; combining, particularly by a computing device, at least the first fused image and the second fused image to form a combined microscopic image; Includes.
[0011] The method steps / stages may be performed at least partially simultaneously and / or sequentially, whereby the order of the method steps / stages is not limited by the order specified, and individual steps / stages may be performed in different orders. Furthermore, individual or all steps / stages may be performed repeatedly.
[0012] In the context of the present invention, reference is made to both first and second images and first and second fused images, where first and second refer only to the number of images or fused images and not to the order of imaging.
[0013] In particular, the method may be a computer-implemented method.
[0014] In the context of the present invention, microscopic image recording can be understood as the measures necessary to obtain image information in a microscopic region of a sample.
[0015] A synthetic microscope image can be understood as a combination of several images with at least partially overlapping fields of view and can in particular be generated by the synthesis of segmented panoramic or high-resolution images.
[0016] It is contemplated that a definition of the imaging parameters is provided. The imaging parameters may consist of at least an imaging area, a resolution or a color channel.
[0017] An imaging area may be defined, resulting in a composite microscopic image. The imaging area may be divided into fields of view in which microscopic imaging is performed. The imaging area may include at least one area in the z-direction, the x-direction, or the y-direction. An area imaged in the z-direction, the x-direction, or the y-direction may also be referred to as a block.
[0018] Additionally, at least one definition of the resolution in at least the x, y, or z direction can be provided. In particular, the z resolution can be specified as a number of faces in the xy plane. This has the advantage that the user can easily record how many layers the composite image will contain.
[0019] The imaging area may in particular be divided into blocks based on a defined resolution, the size of the blocks being in particular defined as a function of at least the number of fields of view, the number of xy planes or the memory size of a memory device of the microscope system.
[0020] Furthermore, at least one color channel can be defined. If multiple color channels are defined, the order of the color channels can be defined during microscope imaging. In particular, at least one exposure time or filter depth can be defined for each color channel individually.
[0021] It is also conceivable that a microscopic pre-imaging is provided. Pre-imaging can be understood to mean that imaging is performed according to defined imaging parameters, but is limited in terms of at least the imaging area, resolution, filter depth or computation quality compared to the microscopic imaging. This allows the user to quickly get a full picture of the effect of defining the imaging parameters, without having to take the corresponding time to capture a complete image for each adjustment.
[0022] In principle, the sample can be any object that can be observed through the microscope used. In particular, the sample can be a biological and / or medical sample, in particular a cell, a tissue or an organ. The sample can be arranged on an object holder. The object holder can hold one or more samples.
[0023] In the context of the present invention, microscopic imaging can be understood as generating, with the aid of an imaging device, an image containing information on the microscopic scale of a sample.
[0024] The fused image can be composed of information from at least two microscope images, in particular, the fused image can be taken at the same location on the sample.
[0025] The second image includes both blurred and sharp content. The blurred content includes substantially localized low frequency components. A high pass filter can be used on the second image to discard the blurred image information. The high pass filter can be configured to select only the high spatial frequency components and filter out the low frequency components.
[0026] In the first image, the structured illumination causes negligible contrast in the imaged modulation relative to the object signal occurring outside the focus, i.e. the measure of the local contrast of the imaged modulation is a measure of how well the object is in focus.
[0027] A measure of local modulation contrast of the first image may be provided. The measure of modulation contrast of the first image may be performed as a measure of at least local variance, single sideband modulation, or double sideband modulation. The measure of local modulation contrast may be coarse-grained.
[0028] The low frequency image information derived from the first image can be configured to be supplemented with high frequency image information derived from the second image. Fusing the high and low frequency image content results in a fully resolved, sharp image that includes all frequency content within the frequency bandwidth of the microscope system. This essentially corresponds to a HiLo microscopy approach.
[0029] Structured illumination can be understood as illumination that is configured to generate radiation from the sample in response to the illumination, mainly in the focal plane. In particular, the intensity of the structured illumination can vary depending on the location. The variation in intensity can be generated by a diffuser illuminated by a light source, in particular a coherent light source.
[0030] Uniform illumination (also known as Gaussian illumination) can be defined as illumination that exposes the imaged image area with a substantially uniform intensity. There will always be some degree of non-uniformity, which is unavoidable due to the characteristics of the light source and the optical system, even in uniform illumination. Uniform illumination can be achieved by a rotating diffuser illuminated by a light source, in particular a coherent light source. It is particularly advantageous that the diffuser is the same as the diffuser used to provide the structured illumination, so that the diffuser is stationary when the structured illumination is provided. It is therefore possible to switch between structured illumination and uniform illumination by stopping or rotating the diffuser.
[0031] Furthermore, the imaging order of the first and second images may be configured such that the first image is followed by a further first image after the position on the sample has been moved, or the second image is followed by a further first image after the position on the sample has been moved, which has the advantage that the imaging mode of the imaging device does not need to be changed for each imaging, allowing for faster imaging.
[0032] The synthesis of fused images (also called stitching) can be understood as the combination of image information contained in the fused images, which are at least partially overlapping, whereby, in particular when the images are synthesized, at least the overlapping regions can be identified and the images can be aligned or their appearance can be adjusted, in particular at least with respect to brightness, color or contrast.
[0033] Different positions on the sample may mean that the first fused image and the second fused image have at least one area that is not included in both images. In other words, this means that the two images may be imaged at different positions. The microscopic imaging and compositing may be performed several times, in particular once, for a given area of the image, in particular a block. A block may have one or more images at xy positions and in the xy plane. The compositing may first be performed using one set of fused images from the first block and the further block, using the fused image with the highest contrast, whereby further fused images may be composited using this ratio. This allows the compositing process to minimize errors in the images.
[0034] The method may comprise at least one of the following steps / stages, particularly in the order presented:
[0035] At least the sample or the lens is moved to a first block. A first xy-plane of the block is microscopically imaged, where a first and a second image are imaged and fused, and a next xy-plane of the first block is microscopically imaged, where a first and a second image are imaged and fused until all xy-planes of the first block have been microscopically imaged. The microscopic imaging of the first block is repeated for one or more additional color channels. All fused images of all xy-planes of the first block are intermediately stored.
[0036] At least the sample or the lens is moved to a further block. A first xy plane of the further block is microscopically imaged, whereby a first and a second image are imaged and fused, and a next xy plane of the further block is microscopically imaged, whereby a first and a second image are imaged and fused until all xy planes of the block are microscopically imaged. The microscopic imaging of the next block is repeated for one or more further color channels. All fused images of all xy planes of the first block are intermediately stored.
[0037] The fused image of the first block, in particular of the xy-plane with the highest contrast in the first color channel, is combined with the fused image of the second block from the same xy-plane. "Same xy-plane" can in particular refer to at least partially overlapping image content. Alternatively or additionally, it can also mean at least the same position in the z-direction relative to the lens or the sample. All further fused images of the first block are also combined according to the relationship of the first combined image.
[0038] At least the sample or the lens can then be moved to the next block and the steps / stages repeated. The composite images of the blocks can be combined with each other to create a composite microscope image of the sample.
[0039] Overall, the advantage of the method according to the invention is that it allows fast and cost-effective imaging with a particularly high optical quality, including in particular image sharpness, and at the same time with the largest possible image section, in particular an image section of the entire sample. The fusion of the first and second image means that image information from the out-of-focus plane of the lens can be discarded, thereby creating an image with increased image sharpness very quickly. This method is particularly fast compared to scanning images as in confocal microscopy, since it is not necessary to scan the image, but only two images need to be captured. By combining at least two, in particular several, fused images, the imaging speed can be exploited to stitch together several images and thus generate images with a particularly large amount of information, for example segmented panoramic images or high-resolution images with a large depth of field.
[0040] Within the scope of the present invention it is also conceivable that the structured illumination is configured at least as a speckle illumination, a periodic grating or a checkerboard pattern when capturing the first image.
[0041] Speckle illumination can be configured to generate scattered points of light. Speckle can be understood as individual point-like locations on the sample where the illumination intensity is high relative to the rest of the sample. Speckle can be generated by interference. For this purpose, a monochromatic coherent light source combined with a diffuser can be provided.
[0042] The advantage of speckle illumination is that it can produce particularly sharp fused images, since the intensity of backscattered light outside the focal plane is significantly reduced.
[0043] Periodic grating or checkerboard patterns can also be provided with the aid of interference generating optics.
[0044] Furthermore, in the method according to the invention it may be advantageous for the illumination during the acquisition of the first and second images to be configured as fluorescent illumination, in particular having at least two color channels.
[0045] In other words, the first and second images can be fluorescence images, in particular multi-channel fluorescence images. Fluorescence images are particularly susceptible to radiation that does not originate from the focal plane, since these radiations can lead to a certain blurring effect, blurring otherwise sharp structures against a fairly dark background. Therefore, the method according to the invention in combination with fluorescence illumination is particularly advantageous.
[0046] Furthermore, in the method according to the invention, it is conceivable that the first and second images are imaged at least in the first and second color channels, whereby in particular the sample is first illuminated in a structured homogeneous manner in the first color channel and then in a structured homogeneous manner in the second color channel. When imaging several color channels, it may be particularly advantageous that the illumination of the first or second image always starts from the illumination at which the imaging of the previous color channel was terminated. For example, if there are three color channels, red, green and blue, the sequence would be as follows: structured illumination of red, uniform illumination of red, uniform illumination of green, structured illumination of green, structured illumination of blue, uniform illumination of blue. This means that there is no need to frequently accelerate and decelerate the diffuser, which on the one hand saves time in the imaging and on the other hand reduces interferences in the form of vibrations.
[0047] If there are several color channels and microscopic imaging in several xy-planes, it is possible to first image each xy-plane in each color channel and then switch to the next xy-plane. This has the advantage that a particularly high image quality is achieved when combining the individual channels, since the image is taken without moving the sample or the lens. It is also possible that each xy-plane is traversed by a single color channel.
[0048] Furthermore, in the method according to the invention, it may be advantageous that the position on the sample varies at least in the sample z-direction, the sample x-direction or the sample y-direction, in particular that the position of the sample in at least the x-direction or the y-direction can be moved by an xy-stage on which the sample can be positioned.
[0049] In the context of the present invention, the z-direction can be understood as the depth of the sample that lies on the optical axis of the imaging optics in the sample region. The x- and y-axes therefore extend in a plane perpendicular to the optical axis in the sample region.
[0050] By varying the z-position, the composite microscope image can essentially correspond to a tomographic image: the individual layers can be displayed with particular clarity due to the defined depth of field.
[0051] The sample can be mapped by merging fused images in the x and / or y directions. The entire sample can be merging from the fused images in at least the x, y or z directions. This has the advantage that a particularly large amount of information is available for evaluation, since the sample is completely recorded from one or more directions.
[0052] In the context of the present invention, it is also conceivable that the position of the sample in the second fused image differs from the position of the sample in the first fused image by 500 nm to 500 μm, in particular 1 μm to 300 μm in the z direction.
[0053] In other words, very thick samples can be measured with an optical microscope. With thick samples, conventional optical microscopes have the problem that radiation from planes that do not correspond to the focal plane is added together, blurring the composite image. This is prevented by the method according to the invention, so that samples in a specified thickness range can also be imaged with particularly high quality.
[0054] It is also conceivable within the scope of the present invention to adjust the depth of field of the individual captured fusion images.
[0055] This allows the fusion effect to be optimally adapted to the current imaging situation. A deeper depth of field allows more information to be obtained from areas that are not exactly in the focal plane. This prevents the elimination of object information in the case of objects that are far apart in the z-plane when the distance between the fused images to be combined is large or when the focal plane is not set perfectly correctly. On the other hand, a shallow depth of field ensures particularly sharp images and a good overall picture with a small distance between the images to be combined in the z-direction.
[0056] It is also conceivable that the method according to the invention also provides for the output of a composite microscope image, whereby in particular the z-depth can be set during the output.
[0057] This allows the user to select the optimal depth of field for the situation, which, as mentioned above, can vary even within a sample. Customizability therefore always guarantees an optimal user experience and allows for accurate evaluation of the composite microscope image.
[0058] It is further conceivable that in the method according to the present invention, knowledge is provided regarding a lens configured to capture at least the first image and the second image, and in particular the depth in the z direction is set based on knowledge regarding the lens.
[0059] By matching it to the respective lens, it is possible to set the z-depth that represents the optimum compromise between image sharpness and depth of field.
[0060] It is also conceivable in the method according to the invention that the microscopic imaging of at least the first fused image and the second fused image is performed successively.
[0061] Taking the images to be fused successively has the advantage that the taken images can be generated particularly quickly.
[0062] In the context of the present invention, it is further contemplated that shading correction is provided when at least the first fused image and the second fused image are combined to form a combined microscope image.
[0063] Shading correction can achieve particularly good optical quality, especially in the transition between the two images to be combined.
[0064] It is also conceivable in the method according to the invention that at least the first fused image or the second fused image is displayed as a live image, thereby setting at least one depth in the z-direction.
[0065] The live image allows the user to immediately see the effect of the z-depth setting and therefore set the optimum value for the measurement. The immediate feedback effectively prevents the initiation of a lengthy imaging process only to discover later that the settings were not optimal.
[0066] Preferably, the method according to the invention comprises at least Loading microscope slides into the microscope system, Overview scans of microscope slides, Recording label information on microscope slides; Detection of samples on microscope slides, Identification of image recording areas based on detection of the sample; Defining the imaging parameters; Microscope preview imaging with predefined imaging parameters, in particular microscope preview imaging according to the invention, Microscopic imaging with predefined imaging parameters, in particular microscopic imaging according to the invention, · Creating a focus map; Movement of the xy stage to the image recording area, Correction of lens defects, storing at least the first image, the second image, the first fused image, the second fused image, or the combined microscopic image; and Movement of the xy stage to the next specimen or microscope slide Further possibilities include:
[0067] By recording the label information, the microscope can be optimally adjusted before the image is taken, e.g., the z-depth can be adapted to the label information. For example, the label may contain information about the thickness or type of the sample, so that a certain depth is found to be advantageous.
[0068] An overview scan of a sample can be understood as a microscopic imaging with at least lower resolution, fewer or different channels, a larger image area, or a lower magnification compared to the microscopic imaging, such an overview scan allows for rapid identification of the sample and the area of interest for high resolution imaging.
[0069] The next sample can be picked up quickly by moving the xy stage to the next sample. Furthermore, the next microscope slide, which may in particular have one or more samples, can also be picked up in this way. The next microscope slide can also be inserted into the xy stage via the microscope slide mechanism.
[0070] Loading the microscope slide into the microscope system, overview scanning the microscope slide, identifying the image recording area, defining the imaging parameters, microscope imaging, saving at least the first image, the second image, the first fused image, the second fused image or a composite microscope image, or moving the xy stage to the next sample or the next microscope slide may be performed in a specifically specified order.
[0071] According to a further aspect of the invention there is provided a microscope system, in particular for carrying out the method according to the invention, said system comprising: an optical imaging device adapted to provide microscopic imaging of at least the first fused image and the second fused image, a camera and a lens configured to capture a first image and a second image; a structured illumination device configured to illuminate the sample in a structured manner when the first image is captured; a Gaussian illuminator adapted to uniformly illuminate the sample when the second image is captured; and an optical imaging device comprising: a computing device adapted to fuse a first image and a second image, the first image being used for low frequency focused image information and the second image being used for high frequency focused image information, the second fused image depicting a different location on the sample than the first fused image, and combining at least the first fused image and the second fused image into a combined microscope image; Equipped with.
[0072] The microscope system according to the invention therefore has the same advantages as those described above with reference to the method according to the invention.
[0073] The structured illumination device may comprise at least one coherent light source, in particular a laser, or a diffuser, whereby the diffuser is illuminated by laser light and generates a speckle pattern.
[0074] Furthermore, the diffuser may be configured to be rotatable, and when illuminated by a Gaussian illuminator, may be rotated to such an extent that a uniformly illuminated image is captured on the time scale of microscopic imaging. Such an arrangement allows the first and second images to be generated in particularly rapid succession, thereby significantly reducing the overall imaging time of the composite microscopic image.
[0075] According to a further aspect of the invention there is provided a computer program product comprising instructions which cause a computer to carry out the method according to the invention when the program is executed by a computer, in particular a computing device of a microscope system according to the invention.
[0076] The computer program product according to the invention therefore has the same advantages as those described above with reference to the method according to the invention and / or the microscope system according to the invention.
[0077] Further advantages, features and details of the invention will become apparent from the following description in which an embodiment of the invention is described in detail with reference to the drawings. The features recited in the claims and in the specification may be essential to the invention individually or in any combination. [Brief description of the drawings]
[0078] [Figure 1] FIG. 2 illustrates a method according to the present invention. [Diagram 2] FIG. 13 is a diagram showing an example of synthesizing a fusion image. [Diagram 3] FIG. 13 is a diagram showing another example of synthesizing a fusion image. [Figure 4] FIG. 1 illustrates a microscope system according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0079] In the following description of several embodiments of the present invention, the same reference signs are used for the same technical features even in different embodiments.
[0080] FIG. 1 shows a method 100 for recording a synthetic microscope image 200 of a sample 300. The method 100 includes: A step 110 of microscopically capturing at least a first fusion image 210 and a second fusion image 220, A step 111 of capturing a first image 230, in which the sample 300 is illuminated in a structured manner, capturing 112 a second image 240, where the sample 300 is uniformly illuminated; and A step 113 of fusing a first image 230 and a second image 240, where the first image 230 is used for low frequency image information located at the focus and the second image 240 is used for high frequency image information located at the focus, wherein the second fused image 220 depicts a different location on the sample 300 than the first fused image 210; A step 120 of combining at least the first fused image 210 and the second fused image 220 into a combined microscopic image 200; Includes.
[0081] Overall, the method 100 according to the invention achieves the advantage that it allows fast and cheap imaging with a particularly high optical quality, including in particular image sharpness, and at the same time with as large an image section as possible, in particular an image section of the entire sample 300. The composition of the first image 230 and the second image 240 means that an image can be created very quickly, in which image information from planes that are not in the focus of the lens 412 can be discarded, thereby increasing the image sharpness. This method is particularly fast compared to scanning through images in a confocal microscope, since it does not require scanning of the image, but only takes two images 230, 240. By combining at least two, in particular several, fused images 210, 220, the imaging speed can be exploited to stitch together several images, so that a particularly information-rich image 200 can be generated, for example a segmented panoramic image or an image 200 with a high resolution image with a large depth of field.
[0082] Furthermore, further steps / stages may be provided in the method according to the invention.
[0083] For example, the output 130 of the composite microscope image 200 may be provided, in particular on a screen. In particular, the depth in the z-direction at which those image information from the composite microscope image 200 are discarded may further be adjustable during the output 130. In this way, the user can optimally adapt the depth to the evaluation and / or imaging requirements.
[0084] 2 and 3 show how the fused images 210, 220 can be combined 120.
[0085] 2 is a top view of the sample 300. The dashed lines indicate the points at which a first fused image 210 and a second fused image 220 are captured. Each dotted square corresponds to a first image 230 and a second image 240, which are then combined 113. In other words, the microscope system 400 captures the two images 230, 240 four times in the illustrated example. The fused images 210, 220 overlap in each case. The combined 120 of the fused images 210, 220 then produces a combined microscope image 200. This combined microscope image 200 covers a much larger area of the sample 300 in the xy plane than the individual fused images 210, 220. In particular, it may be possible to cover the entire sample 300 and / or a preselected section of the sample with the combined microscope image 200.
[0086] Fig. 3 shows a further variation for synthesizing 120 the fused images 210, 220. In the example of Fig. 2, the positions of the fused images 210, 220 on the sample vary in the xy plane, whereas in the example of Fig. 3, the variation in the z plane is shown. In other words, it is also possible to synthesize 120 the fused images 210, 220 that vary in the z axis parallel to the optical axis of the imaging device 410 (see Fig. 4).
[0087] Shown in dashed lines is the z-depth at which image information from the second image 240 that is not in focus of the second image 240 is discarded. This depth may be adjustable. The distance between the fused images 210, 220 may be selected as shown such that the fused images 210, 220 do not overlap in the depth direction. The fused images 210, 220 may also at least partially overlap at these depths.
[0088] Of course, it is also possible to provide a composite microscopic image 200 by combining 120 fused images 210, 220 that vary in the xy plane as well as fused images 210, 220 that vary on the z axis. As a result, the user of the microscope system 400 has particularly a lot of image information available for evaluation, whereby the method 100 according to the invention creates a composite microscopic image 200 very quickly, despite the high amount of information.
[0089] FIG. 4 shows a microscope system 400 for carrying out the method 100 according to the invention. The microscope system 400 comprises: an optical imaging device 410 adapted to provide microscopic imaging 110 of at least a first fusion image 210 and a second fusion image 220, a camera 411 and a lens 412 adapted to capture 111 a first image 230 and a second image 240; a structured illumination device 413 adapted to illuminate the sample 300 in a structured manner when the first image 230 is captured, and A Gaussian illuminator 414 adapted to uniformly illuminate the sample 300 when the second image 240 is captured. an optical imaging device 410 including: a computing device 420 adapted to fuse 113 a first image 230 and a second image 240, where the first image 230 is used for low frequency focused image information and the second image 240 is used for high frequency focused image information, the second fused image 220 depicting a different location on the sample 300 than the first fused image 210, and the computing device 420 adapted to combine at least the first fused image 210 and the second fused image 220 into a combined microscope image 200; Equipped with.
[0090] 4 is a view of the microscope system in the xz plane. Light is directed from a coherent light source 430 through a structured illuminator 413 and / or a Gaussian illuminator 414, through a beam splitter 450 and a lens 412 onto a sample 300 placed on a stage 415. The reflected or fluoresced light is reflected back through the beam splitter 450 and then onto the camera 411. In the illustrated embodiment, the structured illuminator 413 and the Gaussian illuminator 414 have a diffuser 440 configured to be rotatable. If the diffuser does not rotate, speckle illumination is generated by the structured illuminator 413 to capture 111 a structured illumination image 230. If the diffuser 440 is configured to rotate, a uniformly illuminated image 240 can be captured 112. The controller 420 may control at least the imager 410 , the camera 411 , the lens 412 , the structured illuminator 413 , the Gaussian illuminator 414 , the stage 415 , the coherent light source 430 or the diffuser 440 .
[0091] The microscope system 400 according to the invention therefore has the same advantages as those described above with reference to the method 100 according to the invention.
[0092] The structured illumination device 413 may comprise at least one coherent light source 430, in particular a laser, and a diffuser 440, which is illuminated by the laser light and generates a speckle pattern.
[0093] Furthermore, the diffuser 440 is configured to be rotatable, and when illuminated by the Gaussian illuminator 414, may be rotated to such an extent that a uniformly illuminated image is captured on the time scale of the microscope imaging 110. Such an arrangement allows the structured and uniformly illuminated images 230, 240 to be generated particularly quickly in succession, such that the overall imaging time of the composite microscope image 200 is significantly reduced.
[0094] The above description of the embodiments is intended to illustrate the invention purely by way of example, and it goes without saying that the individual features of the embodiments can be freely combined with one another, where technically expedient, without departing from the scope of the invention.
Claims
1. A method (100) for recording a synthetic microscope image (200) of a sample (300), comprising: Microscopic imaging (110) of at least a first fusion image (210) and a second fusion image (220), comprising: A step (111) of capturing a first image (230), wherein the sample (300) is illuminated in a structured manner; capturing (112) a second image (240), wherein the sample (300) is uniformly illuminated; - a step (113) of fusing the first image (230) and the second image (240), the first image (230) being used for low frequency image information located at the focal point and the second image (240) being used for high frequency image information located at the focal point; wherein the second fused image (220) depicts a different location on the sample (300) than the first fused image (210); combining (120) at least the first fused image (210) and the second fused image (220) into the combined microscopy image (200); Including, Method (100).
2. The structured illumination in the step (111) of capturing the first image (230) is implemented as at least a speckle illumination, a periodic grating or a checkerboard pattern.
2. The method of claim 1 (100).
3. The illumination in the step (112) of capturing the first image (230) and the second image (240) is configured as a fluorescent illumination, in particular having at least two color channels.
3. The method (100) of claim 1 or 2.
4. the steps (111) of taking the first image (230) and (112) of taking the second image (240) are performed using at least a first and a second color channel, in particular characterized in that the sample (300) is first illuminated in a structured and uniform manner using the first color channel and then the sample (300) is illuminated in a structured and uniform manner using the second color channel, 3. The method (100) of claim 1 or 2.
5. the positions on the sample (300) vary at least in the z-direction of the sample (300), the x-direction of the sample (300) or the y-direction of the sample (300), characterized in that the position of the sample (300) at least in the x-direction or the y-direction is moved by an xy-stage (415) on which the sample (300) can be positioned, 3. The method (100) of claim 1 or 2.
6. characterised in that the position of the sample (300) in the second fused image (220) differs from the position of the sample (300) in the first fused image (210) by a range of 500 nm to 500 μm, in particular 1 μm to 300 μm in the z direction.
3. The method (100) of claim 1 or 2.
7. The depth of field of each of the first fusion image (210) and the second fusion image (220) captured can be set.
3. The method (100) of claim 1 or 2.
8. An output (130) of the composite microscope image (200) is further provided, in particular characterized in that the z-depth can be set in the output (130), 3. The method (100) of claim 1 or 2.
9. characterised in that recognition of a lens (412) configured for performing at least the step (111) of taking the first image (230) or for performing the step (112) of taking the second image (240) is provided, in particular the z-depth being set based on recognition of the lens (412), 3. The method (100) of claim 1 or 2.
10. wherein the step (110) of microscopically imaging at least the first fusion image (210) and the second fusion image (220) is performed consecutively.
3. The method (100) of claim 1 or 2.
11. characterised in that in the step (120) of combining at least the first fused image (210) and the second fused image (220) to form the combined microscopic image (200), shading correction is also provided.
3. The method (100) of claim 1 or 2.
12. At least the first fusion image (210) or the second fusion image (220) is displayed as a live image, and at least one depth in the z-direction is adjustable.
3. The method (100) of claim 1 or 2.
13. Furthermore, at least Loading the microscope slide into the microscope system (400); an overview scan of said microscope slide (310); recording the label information of said microscope slide (310); - detection of said sample (300) on said microscope slide (310); - Identification of an image recording area based on the detection of said sample (300); - Creating a focus map, - moving an xy stage (415) to the image recording area; - Correction of lens defects, storing at least the first image (230), the second image (240), the first fused image (210), the second fused image (220) or the composite microscopic image (200); Movement of the xy stage (415) to the next sample (300) or to the next microscope slide (310) The present invention is characterized in that:
3. The method (100) of claim 1 or 2.
14. A microscope system (400) for carrying out the method (100) in particular according to claim 1 or 2, comprising: an optical imaging device (410) adapted to provide microscopic imaging (110) of at least a first fusion image (210) and a second fusion image (220), comprising: a camera (411) and a lens (412) adapted to capture (111) a first image (230) and a second image (240); a structured illumination device (413) adapted to illuminate the sample (300) in a structured manner when the first image (230) is taken; and a Gaussian illuminator (414) adapted to uniformly illuminate the sample (300) when the second image (240) is captured; an optical imaging device (410) including: a computing device (420) adapted to fuse (113) the first image (230) and the second image (240), where the first image (230) is used for low frequency focused image information and the second image (240) is used for high frequency focused image information, and where the second fused image (220) depicts a different location on the sample (300) than the first fused image (210); and fusing at least the first fused image (210) and the second fused image (220) into a composite microscope image (200); Equipped with A microscope system (400).
15. The program comprises instructions which, when executed by a computer, in particular a computing device (420) of a microscope system (400) according to claim 14, cause the computer to carry out the method according to claim 1 or 2, Computer program products.
