Method for providing a combined image using a digital microscope, digital microscope system and program for providing a combined image using a digital microscope

By receiving the user-selected region of interest, generating individual images in full-resolution or reduced-resolution modes, and combining secondary sampling and downsampling techniques, the optimization problem between image quality and generation speed in digital microscopy is solved, achieving fast and efficient combined image generation.

CN114730069BActive Publication Date: 2025-12-05PRECIPOINT GMBH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202080078233.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-11-15
Filing Date
2020-11-13
Publication Date
2025-12-05
Estimated Expiration
2040-11-13

AI Technical Summary

Technical Problem

Existing digital microscope operating modes cannot effectively adapt to different usage scenarios, especially when users have different regions of interest in the sample, making it difficult to optimize between image quality and generation speed.

Method used

By receiving the user's selection of the region of interest, a separate image is generated using either full-resolution or reduced-resolution mode. Combined with secondary sampling and image data downscaling techniques, the image processing speed and quality are optimized to meet user needs.

Benefits of technology

It enables the rapid generation of high-quality combined images based on user selections when different regions of interest are defined, improving the efficiency and convenience of user sample analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114730069B_ABST
    Figure CN114730069B_ABST
Patent Text Reader

Abstract

A method of providing a combined image using a digital microscope (2) having an optical system (25), an image sensor (120) having a predefined number of image pixels and a stage (10) for holding a sample (12), the stage (10) being movable relative to the optical system (25) and the image sensor (120), comprising receiving a user selection regarding a region of interest (74) of the sample (12), the user selection indicating a position and an extension of the region of interest (74), selecting one of a full resolution mode and a reduced resolution mode depending on the user selection, wherein in the full resolution mode a single image having the predefined number of image pixels is generated, wherein in the reduced resolution mode a single image having a reduced number of image pixels compared to the predefined number of image pixels is generated, moving the stage (10) relative to the optical system (25) and the image sensor (120) and generating a single image of the region of interest (74) according to the selected one of the full resolution mode and the reduced resolution mode, and combining the single images into a combined image representing the region of interest (74).
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present invention is in the field of digital microscopes. In particular, the present invention is in the field of generating a digital image of a part of a sample under observation by a digital microscope. BACKGROUND

[0002] Recent developments in technology aim at replacing traditional analog microscopes by digital microscopes. Current digital microscopes are typically operated in one of two operating modes. In one operating mode, a digital microscope is operated like a traditional analog microscope. A stage holding a sample to be observed is driven to a desired position, a single image is taken by a camera of the digital microscope, the single image corresponding to the position of the stage. The single image is presented to a user. In another operating mode, a user requests an image of a sample, the microscope performs a line-by-line or column-by-column scan of the sample, the image of the sample being composed from the individual images taken during the scan. These operating modes of current digital microscopes do not well accommodate all use scenarios of digital microscopes.

[0003] Therefore, it would be advantageous to provide a digital microscope system and a method of using a digital microscope to provide a composed image that allows for an improved handling by a user. SUMMARY

[0004] An exemplary embodiment of the present invention comprises a method of providing a composed image using a digital microscope having an optical system, an image sensor having a predefined number of image pixels and a stage for holding a sample, the stage being movable relative to the optical system and the image sensor, the method comprising receiving a user selection regarding a region of interest of the sample, the user selection indicating a position and an extension of the region of interest; selecting one of a full resolution mode and a reduced resolution mode depending on the user selection, in the full resolution mode individual images having the predefined number of image pixels are generated, in the reduced resolution mode individual images having a reduced number of image pixels compared to the predefined number of image pixels are generated; moving the stage relative to the optical system and the image sensor and generating individual images of the region of interest according to the selected one of the full resolution mode and the reduced resolution mode; and combining the individual images into a composed image representing the region of interest.

[0005] Exemplary embodiments of the present invention allow adaptation of the generation of individual images, which are eventually combined into a combined image, to a user selection relating to a region of interest of a sample. An image of a portion of the sample that is of real interest to the user can be provided by an optimized trade-off between image quality and speed of provision. In particular, by providing a selection between a full resolution mode and a reduced resolution mode for generating individual images, in case the user selection indicates that a relatively small resolution of the required sample image is sufficient, then faster image processing of the individual images with a reduced number of image pixels can be utilized. For example, in case the user selection indicates that the user is interested in a substantial portion of the sample, i.e. in case the user selection indicates that the extension of the region of interest is substantial, then the reduced resolution mode can be selected and a substantially larger number of individual images required for generating a representation of said substantial portion of the sample can be implemented at a higher speed compared to using the full resolution mode. In other words, based on the assumption that a user interested in a substantial portion of a sample is satisfied with a combined image of less than maximum resolution, the reduced resolution mode can be selected, increasing the speed of generating the combined image without affecting the resolution required by the user for the instant use case. In case the user selection indicates a small region of interest, the full resolution mode can be selected. In this case, the relatively small number of individual images required for generating the combined image can still be provided at a comparably fast operation and provides the user with a high image quality of the selected small region of interest.

[0006] The digital microscope has an optical system. The optical system can be any kind of optical system suitable for providing a desired magnification of the digital microscope. In particular, the optical system can comprise an objective and a tube lens. Due to the specific optical system arranged in the digital microscope, the magnification can be a set value. In particular, the magnification can be set according to the specific objective arranged in the digital microscope.

[0007] The digital microscope has a stage for holding a sample. The sample can be provided in the form of a glass slide. The glass slide can be placed on the stage or put into a device specifically designed for holding a glass slide.

[0008] The stage is moved relative to the optical system and the image sensor. In particular, the digital microscope can have a stage drive assembly driving the stage relative to the optical system and the image sensor. The optical system and the image sensor can be essentially stationary in a microscope reference frame. The stage can be moved in two dimensions by the stage drive assembly, such that a two-dimensional scan of the sample can be performed. The stage can also be moved towards / away from the optical system and the image sensor. This movement dimension can be comparably small and can only be used for focusing purposes.

[0009] The digital microscope has an image sensor. The image sensor can be part of a digital camera. It can thus also be said that the digital microscope has a digital camera with an image sensor. The digital camera can have additional components as usual in cameras, such as a shutter, an image sensor driver, etc.

[0010] The method comprises receiving a user selection regarding a region of interest of the sample. The region of interest can also be described as a field of view of interest to the user. The field of view can in particular be a field of view that the user wants to be displayed on a screen that cooperates with the digital microscope. The sample region of interest of interest to the user corresponds to a particular part of the platform on which said region of interest of the sample is located. Due to this correspondence between the region of interest of the sample and the respective part of the platform, the platform can be moved in a certain way to generate individual images of the region of interest.

[0011] The term "generate individual images of the region of interest" means generate individual images of individual parts of the sample in the region of interest. The individual images can be generated in a scan of the region of interest. The individual images of adjacent parts of the region of interest can have an overlap.

[0012] The user selection indicates the location and extension of the region of interest. In particular, the user selection can implicitly specify the location and extension of the region of interest. For example, the user can make the user selection by zooming and navigating laterally through an image preview of the sample. This zooming and lateral navigation can be done by means of a touch screen or by means of operation of a mouse or by means of any other suitable input device. The location and extension of the region of interest can result from the zooming and navigation with respect to the image preview. It is noted that the user can select the region of interest of the sample in any suitable way, as long as the method receives a user selection, the information of which allows the derivation of the location and extension of the region of interest.

[0013] In the full resolution mode, the individual images are generated using the predefined number of pixels of the image. In contrast thereto, in the reduced resolution mode, individual images are generated with a reduced number of image pixels. The reduced number of image pixels allows the image data processing of the individual images to be carried out at an increased speed. For example, any type of post-processing filter that can be applied to the image data of the individual images can work faster with a reduced number of image pixels. Furthermore, when combining the individual images into a combined image, any stitching operation can process the individual images faster with a reduced number of image pixels. Thus, the processing speed of the image data can be traded off against the resolution of the combined image, thereby adapting the provision of the combined image to the user selection. The user's convenience can be increased in an adaptive manner and the speed of the user's analysis of the sample can be increased.

[0014] According to another embodiment, the individual image with reduced number of image pixels is generated by sub-sampling the predefined number of image pixels of the image sensor. Sub-sampling is a particularly effective way of reducing the predefined number of image pixels of the individual image. In particular, sub-sampling reduces the number of image pixels right at the source, i.e. right at the image sensor, so that all subsequent post-processing is done with reduced number of image pixels. The term "sub-sampling" refers to reading out a reduced number of measurement values from the image sensor compared to the total number of measurement values available. In other words, sub-sampling refers to deliberately ignoring part of the image data generated at the image sensor. Sub-sampling can be done by reading out every nth image pixel of the image sensor, e.g. every second or third image pixel, which can be applied to both dimensions of the image sensor. By applying sub-sampling, the read-out time of the image sensor can be reduced and the processing time of all subsequent image processing can be reduced. The speed of the entire image processing chain can be improved.

[0015] According to another embodiment, the individual image with reduced number of image pixels is generated by downscaling of the image data generated by the image sensor. Downscaling of the image data is another way of reducing the number of image pixels. In contrast to sub-sampling as described above, all sensed image data is used. However, the image data of a given group of neighboring pixels can be combined into a single pixel. For example, a single resulting pixel can be calculated from a window of 2x2 original pixels or 3x3 original pixels or 4x4 original pixels or 5x5 original pixels or the like. In this way, the number of image pixels can be reduced while the image data of the reduced number of image pixels is based on all information sensed by the image sensor. Downscaling can typically be performed at any point in the image processing chain. Downscaling of the image data can be performed immediately after reading out the image data from the image sensor. In this way, the speed increase due to the reduced number of pixels can be exploited in a large part of the image processing chain. Downscaling can be performed according to any suitable downscaling algorithm. Downscaling algorithms are known to the skilled person per se.

[0016] According to another embodiment, the platform is moved in a at least partially continuous manner in the reduced resolution mode, wherein the image sensor captures image data of individual images while the platform is moving. In this way, the speed of generating the combined image can be further increased in the reduced resolution mode. Since the image sensor captures image data of individual images while the platform is moving, there is no need to perform time-consuming platform start / stop operations for capturing image data of individual images. Thus, the image data of individual images is generated quickly consecutively, allowing to provide complete image data of the combined image within a rather short time frame. The capturing of image data of individual images during movement of the platform is particularly beneficial in combination with the sub-sampling of the predefined number of image pixels of the image sensor. Since sub-sampling allows to read out the image data captured by the image sensor faster, the image sensor can be quickly ready to capture more image data. In this way, a continuous image capturing operation can be achieved during movement of the platform without creating any problems that disturb the read-out of the image sensor. The term "at least partially continuous manner" refers to a generally continuous movement, which can be stopped at certain positions. For example, there can be a stop after a predefined number of image data capturing operations in order to reach a well-defined starting point of all components of the digital microscope again. There can also be a stop when changing the direction of movement of the platform, for example when capturing a new row or a new column of individual images. However, it is also possible that substantially all or all image data is captured while the platform is moving. It can also be said that at least a substantial part of the image data of the individual images is captured while the platform is moving. This does not exclude that some image data is captured while the platform is stopped.

[0017] According to another embodiment, the speed of movement of the platform is chosen such that the blurring of the image data is limited to at most two pixels, in particular at most one pixel. In other words, the speed of movement of the platform is chosen such that each point of the sample influences at most three pixels, in particular at most two pixels, in the captured image data. In this way, the speed of movement of the platform is adapted to the rest of the digital microscope system. The speed of movement of the platform and the degree of blurring of the image data is another dimension in which a trade-off between the speed of providing the combined image and the image quality can be made. Thus, the handling of the digital microscope can again be adapted to the user's choice. The term "blurring of the image data" can refer to the degree of blurring in the combined image.

[0018] According to another embodiment, the platform is moved in an intermittent manner in the full resolution mode, the image sensor capturing image data of a separate image when the platform is in a stopped position. This way, the quality of the captured image data is particularly high, since there is no risk of blurring due to movement of the platform when the image data is captured. The term "moving the platform in an intermittent manner" refers to starting / stopping movement of the platform, which can also be referred to as driving / stopping movement of the platform. The platform driving assembly can be synchronized with the image sensor, in particular with the exposure time of the image sensor, so as to accurately capture the image data when the platform is in a stopped position.

[0019] According to another embodiment, the full resolution mode is selected if the extension of the region of interest is smaller than a first threshold. In other words, the full resolution mode is selected if the user is interested in a portion of the sample that is smaller than a threshold size. In this case, it is assumed that the user is interested in a high level of detail. Therefore, a high resolution is selected and, as a consequence, a high image quality of the combined image. The first threshold can be a one-dimensional threshold, for example applied to the larger dimension of the region of interest, or can be a two-dimensional threshold, or can be a region threshold.

[0020] According to another embodiment, the reduced resolution mode comprises a plurality of reduced resolution sub-modes, and the step of selecting one of the full resolution mode and the reduced resolution mode comprises selecting one of the full resolution mode and the plurality of reduced resolution sub-modes. In this way, more than two modes of generating separate images can be provided, thereby enabling the selected mode of generating separate images to be adapted more finely to the user needs. In particular, depending on the extension of the region of interest, a suitable one of the full resolution mode and the plurality of reduced resolution sub-modes can be selected. For this selection, a plurality of thresholds can be provided. Each threshold can be a one-dimensional threshold or a two-dimensional threshold or a region threshold, as described above with respect to the first threshold.

[0021] According to another embodiment, each of the plurality of reduced resolution sub-modes has a respective sub-mode specific reduced number of image pixels for the separate image. In other words, the plurality of reduced resolution sub-modes differ from each other in the reduced number of image pixels. Each reduced resolution sub-mode has a different reduced number of image pixels for the separate image. The selection of one of the full resolution mode and the plurality of reduced resolution sub-modes can be a monotonic function between the extension of the region of interest and the number of image pixels of the separate image in the respective mode. In other words, the smaller the extension of the region of interest, the higher the number of image pixels of the separate image can be.

[0022] According to a further embodiment, the plurality of reduced resolution sub-modes differs in at least one of the level of sub-sampling of the predefined number of image pixels of the image sensor and the degree of reduction of the image data generated by the image sensor. In a particular embodiment, the plurality of reduced resolution sub-modes can differ in both the level of sub-sampling of the predefined number of image pixels of the image sensor and the degree of reduction of the image data generated by the image sensor. It is also possible that the plurality of reduced resolution sub-modes differ with respect to the capturing of image data being performed when the platform moves in an at least partially continuous manner or when the platform is in a stopped position during intermittent movement. Furthermore, the plurality of reduced resolution sub-modes can differ with respect to the speed of movement of the platform in case the platform moves while capturing the image data. In this way, the speed of generation of individual images and thus the trade-off between the speed of generation of the combined image and the quality of the combined image can be adapted to the plurality of reduced resolution sub-modes in a particularly refined manner.

[0023] According to another embodiment, the plurality of reduced resolution sub-modes has two or three or four or five or six reduced resolution sub-modes. A larger number of reduced resolution sub-modes is also possible.

[0024] According to another embodiment, the user selection further indicates a representative resolution, which indicates a desired resolution of the combined image. If applicable, the selection of the full resolution mode, or of one of the reduced resolution modes, or of the plurality of reduced resolution sub-modes can be based on the desired resolution of the user. In this way, the resulting combined image can be more directly based on the user's wishes. It is also possible that the selection is made in accordance with technical properties of the system surrounding the digital microscope, such as a screen on which the combined image is depicted. In a particular embodiment, the representative resolution indicates a screen resolution of a screen on which the combined image is depicted. In this way, taking into account the screen resolution and the degree of zooming of the sample, the selection of the full resolution mode or of the reduced resolution mode can allow to provide a high quality image.

[0025] According to a further embodiment, the method further comprises displaying the combined image on a screen. In this way, the result of the method is provided to the user in an intuitive manner, as described above. The displayed combined image can form the basis for a further user selection, also referred to herein as an updated user selection, such that the user can reach the sample region of his / her particular interest in an iterative manner.

[0026] According to another embodiment, the individual images are displayed on the screen in a stepwise manner. In particular, the individual images can be displayed on the screen substantially as soon as they become available after capturing the image data. In this way, the user can already be provided with information for deciding the next step in the analysis of the sample even before the region of interest is fully displayed on the screen. From the user's perspective, the combined image is built on the screen stepwise. The stepwise building of the combined image can start from the middle of the region of interest and can then move to the edges of the region of interest. The combined image can also be built in a row-wise or column-wise manner, possibly starting from a center line or column. The combined image can be built on the screen in a stepwise manner to provide an intuitive reception of the combined image by the user on the one hand and / or to convey a speed concept to the user, thereby making the operation of the digital microscope more convenient.

[0027] According to another embodiment, the method further comprises receiving an updated user selection of an updated region of interest with respect to the sample, the updated user selection indicating a position and an extension of the updated region of interest; interrupting the one or more method steps that are currently being performed; reselecting one of the full resolution mode and the reduced resolution mode depending on the updated user selection; moving the stage relative to the optical system and the image sensor and generating individual images of the updated region of interest according to the reselected one of the full resolution mode and the reduced resolution mode; and combining the individual images into a combined image representing the updated region of interest. In this way, the method of providing a combined image can immediately respond to a command of the user. Due to the previous instances of the method, it can be ensured that the updated region of interest is scanned and converted into a combined image without delay. Thus, the user receives feedback of the high responsiveness of the method and can complete the analysis of the sample in a particularly time-saving manner.

[0028] An exemplary embodiment of the present application also comprises a digital microscope system comprising an optical system; an image sensor having a predefined number of image pixels; a stage for holding a sample; a stage drive assembly for moving the stage relative to the optical system and the image sensor; a control unit for controlling the generation of image data for a combined image. The control unit is configured to: receive a user selection regarding a region of interest of the sample, the user selection indicating a position and an extension of the region of interest; select one of a full resolution mode and a reduced resolution mode depending on the user selection, in the full resolution mode individual images having the predefined number of image pixels are generated, in the reduced resolution mode individual images having a reduced number of image pixels compared to the predefined number of image pixels are generated; control the stage drive assembly to move the stage relative to the optical system and the image sensor; control the generation of individual images of the region of interest in accordance with the selected one of the full resolution mode and the reduced resolution mode. The additional features, modifications and advantages described above with respect to the method of providing a combined image using a digital microscope apply in an analogous manner to the digital microscope system. In particular, it is explicitly disclosed herein that the control unit can be configured to perform the above-mentioned method steps and / or can be configured to cause components of the digital microscope to perform the above-mentioned method steps. The digital microscope system can be a digital microscope. The digital microscope system can also be a distributed system comprising a digital microscope and a data processing device, e.g. a computer, coupled thereto. The control unit can be provided in one of these entities or can be a distributed component between the digital microscope and the data processing device.

[0029] According to another embodiment, the control unit is configured to control the generation of individual images in the reduced resolution mode by causing at least one of sub-sampling the predefined number of image pixels of the image sensor and downscaling the image data generated by the image sensor.

[0030] According to a further embodiment, the control unit is configured to control the stage drive assembly to move the stage in an at least partially continuous manner in the reduced resolution mode and to control the image sensor to capture image data for individual images while the stage is moved.

[0031] The digital microscope can be an optical microscope. It can have an illumination assembly arranged on a side of the stage remote from the optical system and the image sensor.

[0032] An exemplary embodiment of the present invention further includes a procedure for providing a composite image using a digital microscope, the digital microscope having an optical system, an image sensor having a predefined number of image pixels, and a platform for holding a sample, the platform being movable relative to the optical system and the image sensor. The procedure includes receiving a user selection regarding a region of interest (ROI) of the sample, the user selection indicating the location and extent of the ROI; selecting, depending on the user selection, one of a full-resolution mode and a reduced-resolution mode; instructing the platform to move relative to the optical system; instructing the image sensor to generate image data; based on the image data, generating separate images of the ROI with a predefined number of image pixels if the full-resolution mode is selected, and generating separate images of the ROI with a reduced number of image pixels if the reduced-resolution mode is selected; and combining the separate images into a composite image representing the ROI. The additional features, modifications, and beneficial effects described above regarding the method of providing a composite image using a digital microscope are applicable to procedures for providing composite images using a digital microscope in a similar manner. Attached Figure Description

[0033] Further exemplary embodiments of the invention are described with reference to the accompanying drawings, in which:

[0034] Figure 1 A perspective view of a digital microscope according to an exemplary embodiment of the present invention is shown;

[0035] Figure 2 Showing Figure 1 A schematic diagram of selected components of a digital microscope;

[0036] Figure 3 A schematic diagram of a digital microscope system from a user's perspective according to an exemplary embodiment of the present invention is shown;

[0037] Figure 4 Showing Figure 3 A block diagram of selected components of a digital microscope system; and

[0038] Figure 5 A flowchart illustrating a method for providing combined images using a digital microscope according to an exemplary embodiment of the present invention is shown. Detailed Implementation

[0039] Figure 1 A digital microscope 2 according to an exemplary embodiment of the present invention is shown in perspective and three-dimensional view. The digital microscope 2 has a base 4 that supports it. The base 4 can be placed on a table to provide a sturdy support.

[0040] Base 4 includes lighting components and platform drive components, which are... Figure 1The base housing in the way of the view will be described below. The stage 10 is mounted on the base 4. The stage is movable relative to the base 4. In particular, the stage 10 moves in two dimensions, here referred to as the x-direction and the y-direction. In operation, the stage 10 is moved in the x- and y-directions by a stage drive assembly.

[0041] The stage 10 has a transparent or translucent portion. A sample can be placed on the transparent or translucent portion. In Figure 1 In the depicted operational scenario, the sample 12 comprises two slides which are arranged on the transparent or translucent portion of the stage 10 by a clamping mechanism. In operation, the illumination assembly illuminates the sample 12 from below. The plane of the transparent or translucent portion of the stage 10 is referred to as the x-y plane of the digital microscope.

[0042] The digital microscope 2 further comprises a support arm 6 and a tube assembly 8. The support arm 6 is shaped to support the tube assembly 8 such that the tube assembly 8 hovers above the stage 10. The tube assembly 8 houses various optical components. In particular, the tube assembly 8 houses a digital camera and an optical system, in Figure 1 In the exemplary embodiment of the digital microscope 2, the optical system in turn has a barrel lens structure and an objective 24. While the digital camera and the barrel lens structure are hidden from view in Figure 1 by the tube assembly housing, the objective 24 extends somewhat further towards the stage 10 from there.

[0043] The tube assembly 8 is movable relative to the support arm 6 in a direction of movement perpendicular to the x-y plane. In other words, the tube assembly 8 moves in the z-direction of the microscope reference system. While this movement is very limited, it is sufficient to align a focal point of the sample 12 relative to the optical system contained in the tube assembly 8.

[0044] In operation, the stage drive assembly brings the stage 10 to a desired position in the x- and y-directions. The stage drive assembly can have any type of suitable actuator, for example two small electric motors for the two directions of movement. The illumination assembly provides illumination of the sample 12 from below, and the image data of the portion of the sample 12 which is placed in the light path from the illumination assembly to the digital camera can be captured by the digital camera. The captured image data is referred to herein as image data of a single image, as it refers to image data corresponding to a singular position of the sample 12 relative to the stage 10 and thus relative to the optical system and the digital camera. By driving the stage 10 to various positions, a plurality of single images can be generated.

[0045] Figure 2 Selected components of the digital microscope 2 of Figure 1 are shown in a schematic view. In particular, Figure 2The components related to the illumination of the sample 12 and the guiding of the light within the tube assembly 8 are shown. As mentioned above, the illumination assembly 40 is arranged below the stage 10, i.e. below the sample 12, and directs light upwards to the sample 12. In Figure 2 In an exemplary embodiment, the illumination assembly 40 has a light source 42 and a collimating lens 44 for directing a large amount of light to the part of the sample 12 where the image is captured. It should be noted that the illumination assembly 40 can have any suitable setup / design.

[0046] The digital microscope 2 has an optical system 25. In Figure 2 In an exemplary embodiment, the optical system 25 has a barrel lens structure 22 and an objective lens 24. The barrel lens structure 22 has individual barrel lenses 23. The objective lens is provided for achieving a desired magnification of the part of the sample 12 where its image is captured. In other words, the shape and design of the objective lens 24 sets the magnification of the sample 12 for the individual image produced. In Figure 2 In an exemplary embodiment, the magnification of the objective lens 24 is 20.

[0047] The digital microscope 2 also has a digital camera 20. The digital camera 20 has an image sensor and a shutter. It can also have other components common in the field of digital cameras that can contribute to the operation of the digital camera to capture image data. The barrel lens structure 22 directs the light from the objective lens 24 to the image sensor of the digital camera 20. In this way, an optical path 50 is established from the illumination assembly 40 through the sample 12, through the objective lens 24, through the barrel lens structure 22 and to the image sensor of the digital camera 20.

[0048] Figure 3 A schematic view of a digital microscope system 100 according to an exemplary embodiment of the present application is shown. The digital microscope system 100 comprises a digital microscope 2. The digital microscope 2 can be a digital microscope 2 having a mechanical setup as described above with respect to Figure 1 the optical setup as described above with respect to Figure 2 and the control setup as will be described below with respect to Figure 4 and Figure 5 . With respect to Figure 3 , the focus is on the user perspective and the user interaction with the digital microscope system 100.

[0049] The digital microscope system 100 further comprises a computer 80 coupled to the digital microscope 2 and a screen 70 coupled to the computer 80. The computer 80 can be any type of suitable processing device providing interaction between the screen 70 and the digital microscope 2 in a suitable manner. The computer 80 can for example be a standard personal computer, embodied as a desktop computer or a laptop. In Figure 3The processing capabilities exemplarily provided by the computer 80 in the exemplary embodiments of the application can also be embedded into the digital microscope 2 or the screen 70. The screen 70 can for example be part of a tablet or smartphone having screen functionality and processing functionality for interfacing directly with the digital microscope 2. It is also possible to provide the processing capabilities exemplarily provided by the computer 80 in the exemplary embodiments of the application on a remote processing device, e.g. a remote server as part of a cloud-based solution. Figure 3 The processing capabilities exemplarily provided by the computer 80 in the exemplary embodiments of the application.

[0050] The screen 70 is a control interface for a user of the digital microscope system 100. In the exemplary embodiments of the application, the screen 70 is a touch screen, such that the screen 70 provides both user input functionality and image output functionality. The user can control the entire operation of the digital microscope system 100 by means of the touch screen 70. However, it is also possible to provide other input devices in addition to / instead of the touch screen 70. For example, the user can be provided with a keyboard and / or a mouse and / or any other suitable input device to control the digital microscope system 100. It is also possible to provide multiple screens for the output of images. It is also possible to output images to other entities. For example, images can be saved to a hard drive or other data storage medium in a file format. Figure 3 In the exemplary embodiments of the application, the screen 70 provides two different outputs. In the upper left corner of the screen 70, an image preview 72 is displayed. The image preview 72 is an overview image of the sample arranged on the stage of the digital microscope 2. The overview image can be taken using an additional digital camera. The additional digital camera can be arranged next to the objective 24, i.e. it can be arranged with a viewing direction downwards onto the sample and arranged offset from the objective 24. The additional digital camera can be a low-quality, simple digital camera whose only purpose is to provide a quick overview of the sample with enough detail to allow the user to navigate the sample roughly. It is noted that the image preview 72 can also be generated in any other suitable way. It is also possible to completely omit the image preview 72 and for the user to navigate the sample without such an image preview 72.

[0051] Figure 3 In the exemplary use case of the application, the sample is a biological sample. The sample comprises a cell culture 14 arranged on a transparent slide. Thus, from the user's perspective, the sample comprises the cell culture 14 surrounded by empty space.

[0052] In the exemplary use case of the application, the sample is a biological sample. The sample comprises a cell culture 14 arranged on a transparent slide. Thus, from the user's perspective, the sample comprises the cell culture 14 surrounded by empty space. Figure 3 In the exemplary use case of the application, the sample is a biological sample. The sample comprises a cell culture 14 arranged on a transparent slide. Thus, from the user's perspective, the sample comprises the cell culture 14 surrounded by empty space.

[0053] Figure 3 ​​In the exemplary embodiment, the user can select a region of interest 74 in the image preview 72. The selection of the region of interest 74 is a user selection that determines the operation of the digital microscope and determines what will be displayed on the screen 70. This will be explained in detail below. The user selection can be made in any suitable way, for example by placing a user's finger on a touch screen, using a suitable stylus, using an auxiliary input device such as a mouse, using a menu based selection tool, etc. For the described use example, it is assumed that the user selects the region of interest 74 with his / her finger on a touch screen in a part of the screen 70 that depicts the image preview 72. The user selection indicates the location and extension of the region of interest. In other words, the user selection contains an unambiguous determination of which part of the sample is the user's region of interest. The unambiguous determination can be in any form that allows the location and extension of the region of interest to be derived. For example, the user selection can include the coordinates of the lower left corner of the region of interest and a two-dimensional extension in the x-y coordinate system of the image preview 72, as seen from the digital microscope system 100. In another example, the user selection can include the coordinates of the upper left corner and the lower right corner of the region of interest, as seen from the digital microscope system 100. In yet another example, the user selection can include a single coordinate, for example the center coordinate of the region of interest, and a degree of zooming compared to the size of the entire sample, as seen from the digital microscope system 100.

[0054] Based on the user selection, the digital microscope system 100 generates a combined image that corresponds to the region of interest 74 and is displayed in the combined image screen portion 76. Details of said generating a combined image will be described below with reference to Figure 4 and Figure 5 The details of said generating a combined image are described from the perspective of the system. From the user's perspective, the selection of the region of interest 74 results in a highly magnified combined image in the combined image screen portion 76 compared to the image preview 72, wherein the combined image corresponds to the region of interest 74 of the sample. The term "corresponds" does not necessarily mean that the region of interest and the combined image are exactly identical. For example, the displayed combined image can also display a slightly larger part of the sample than the region of interest 74, for example to match the size of the screen 70.

[0055] In Figure 3In an exemplary embodiment, the combined image screen 76 covers the entire screen 70, except for the image preview 72. The user can also make user selections within the combined image screen 76. For example, the user can navigate a virtual representation of a sample via horizontal panning and zoom operations, which can extend beyond the screen 70. In a particular example, the user can use a two-finger zoom command, as known from smartphone applications. The expansion of the region of interest can also be referred to as the zoom level of the region of interest. User selection, including the location and zoom level of the region of interest, thus also indicates the location and expansion of the region of interest. In a particular example, an image preview taken by a low-quality attached camera may initially be displayed on the full screen, with the user navigating entirely to the region of interest via horizontal panning and zoom operations.

[0056] Figure 4 It shows Figure 3 A block diagram of selected components of the digital microscope system 100. Although Figure 3 The focus is on the user front-end and the interaction between the user and the digital microscope system 100, but the system side, including the hardware back-end and the inserted control structures, is also relevant. Figure 4 Describe it. Figure 4 The selected component depicted has two external connections: a user selection input 92 and a combined image output 94. These two external connections can be viewed as... Figure 3 The interface of the touchscreen 70. When the user selects to input via user selection 92, the data is transmitted from the touchscreen 70 to... Figure 4 When the component of the digital microscope system 100 is in use, the component returns the combined image to the touch screen 70 in an appropriately encoded form for display on the combined image screen 76 via the combined image output 94.

[0057] As described above, selected components of the digital microscope system 100 are in Figure 4 As described in the text. Specifically, the digital camera 20, lighting assembly 40, and platform drive assembly 46, as described above, are... Figure 4 The digital camera 20 is described in the figure. It includes an image sensor 120, a shutter 122, and an image sensor driver 124. When the shutter 122 is open, the image sensor 120 captures image data, and the image sensor driver 124 reads the captured image data from the image sensor 120.

[0058] The digital microscope system 100 further comprises a control unit 90, an image data post-processing unit 96 and an image combination unit 98. Each of the control unit 90, the image data post-processing unit 96 and the image combination unit 98 can be provided in the digital microscope 2 or in the computer 80 or on a remote server. In an alternative, each of these components can also be provided in the screen 70. Further, the control unit 90, the image data post-processing unit 96 and the image combination unit 98 can be distributed in the digital microscope system 100. Further, each of the control unit 90, the image data post-processing unit 96 and the image combination unit 98 can have hardware components or software components or mixed hardware and software components. In Figure 4 In the exemplary embodiment, the control unit 90, the image data post-processing unit 96 and the image combination unit 98 are part of a software program running on the computer 80 and configured for controlling the digital microscope 2.

[0059] The control unit 90 is coupled to a user selection input 92 for receiving a user selection regarding a region of interest of a sample. The control unit 90 is further coupled to the illumination assembly 40, the stage drive assembly 46 and the digital camera 20. The control unit is configured to control the illumination assembly 40, the stage drive assembly 46 and the digital camera 20, in particular the shutter 122 of the digital camera 20 and the image sensor driver 124 of the digital camera 20, in accordance with the user selection.

[0060] The control unit 90 is further coupled to the image data post-processing unit 96 and the image combination unit 98. Further, the digital camera 20 is coupled to the image data post-processing unit 96, which is in turn further coupled to the image combination unit 98, which is in turn further coupled to the combined image output 94. In this way, the control unit 90 is able to control the image processing chain downstream of the digital camera 20, as will be described below.

[0061] Based on the user selection, the control unit 90 determines which individual images of the sample to generate in order to generate the combined image corresponding to the region of interest indicated by the user selection. With the image sensor 120 having a predefined number of image pixels and the optical system of the digital microscope 2 having a set magnification, the size of the sample portion that is converted into image data of an individual image is set by the system parameters. In other words, the size of the sample portion that can be captured by one single operation of the shutter 122 of the digital camera 120 is set. Given this set size, the control unit 90 determines, based on the location and extension of the region of interest, at which individual location the sample image data for an individual image is captured. For example, the individual locations of a line-by-line or column-by-column scan of the region of interest can be determined by the control unit 90. The control unit 90 then controls the stage drive assembly 46, the shutter 122 and the image sensor driver 124 in a synchronized manner to produce the image data at the determined individual locations. The control unit 90 can control the illumination assembly 40 to provide continuous illumination or to provide intermittent illumination synchronized with the other components.

[0062] The control unit 90 does not only control the location at which the image data of an individual image is captured. Based on the user selection, the control unit 90 also determines whether to generate the individual image using a full resolution mode or to generate the individual image using a reduced resolution mode. In case a plurality of reduced resolution sub-modes is provided in the digital microscope system 100, the control unit selects one of the full resolution mode and the plurality of reduced resolution sub-modes to generate the individual image. In the exemplary embodiment described with respect to Figure 4 In the exemplary embodiment described above, two reduced resolution sub-modes are provided in the digital microscope system 100 and the control unit 90 selects one of the three different modes to generate the individual image based on the user selection.

[0063] As described above, the image sensor 120 has a predefined number of image pixels. For the full resolution mode, the individual image has the predefined number of image pixels. The image sensor driver 124 reads out the predefined number of image pixels and the image post-processing unit 96 performs any kind of post-processing, such as color filtering or other filtering, on the image data having the predefined number of image pixels. The individual image output by the image post-processing unit 96 has the predefined number of image pixels.

[0064] In Figure 4In the digital microscope system 100, for one or more reduced resolution modes, there are two methods to reduce the number of image pixels. The first method to reduce the number of image pixels is to subsample the predefined number of image pixels. In particular, the image sensor driver 124 can subsample the predefined number of image pixels of the image sensor. In other words, the image sensor driver 124 can read out less image pixels than the predefined number of image pixels. For example, the image sensor driver 124 can be able to read out image data every other or every second or every third image pixel, etc., i.e., every nth image pixel in general. In particular, the image sensor driver 124 can be able to read every nth image pixel in both dimensions of the image sensor 120. For the exemplary case of reading out image data every second pixel by the image sensor driver 124, the number of image pixels is reduced by a factor of 4. For the exemplary case of reading out image data every second pixel by the image sensor driver 124, the number of image pixels is reduced by a factor of 9.

[0065] Subsampling the predefined number of image pixels allows to speed up the generation of the combined image in a twofold way. First, the reading out of image data from the image sensor 120 is accelerated. In other words, the amount of time required to read out the image data of each individual image from the image sensor 120 is shorter. As a consequence, the image sensor 120 is ready to receive new image data more quickly and is able to continuously capture image data of subsequent individual images more quickly. Thus, the capturing of image data can be accelerated. Second, a smaller amount of image data is provided downstream of the digital camera 20. In particular, a smaller amount of image data is provided to the image post-processing unit 96 and the image combining unit 98. Due to the smaller number of image pixels, the image processing operations performed in these downstream units of the digital camera 20 are performed faster.

[0066] The second method to reduce the number of image pixels is to downsize the image data captured by the image sensor 120. In particular, the image post-processing unit 96 can downsize the image data captured by the image sensor 120. The term "downsize" refers to reducing the number of pixels by an image filtering operation. In an exemplary case of downsizing, the 4 pixels of a 2x2 pixel window can be replaced by a single pixel. The single replacement pixel can be the result of a relatively simple operation, e.g., the average of the 4 pixels in the 2x2 pixel window. In this way, the number of image pixels is reduced by a factor of 4. This approach can be extended to larger pixel windows. Also more sophisticated approaches can be employed to calculate the replacement pixel. For example, larger image structures, e.g., contours, can be taken into account when calculating the replacement pixel. Downsampling is known per se and it will be apparent to the skilled person that various downsampling approaches can be used.

[0067] Downsampling of the image data can be performed at different points in the image processing chain. For example, the downsampling can be performed directly at the input of the image post-processing unit 96, i.e. directly on the image data received from the image sensor driver 124. The image post-processing unit 96 can also apply further image processing before performing the downsampling. Furthermore, further image processing can be applied to the image data after the downsampling at the image post-processing unit 96. The downsampling can be embedded into the image processing chain such that further image processing operations are applied to a larger or smaller number of pixels, depending on the image size required by those further image processing operations.

[0068] The downsampling of the image data allows to speed up the generation of the combined image, because all subsequent image processing operations after the downsampling can be performed faster due to the reduced number of image pixels. The subsequent image processing operations can include individual image level image processing in the image post-processing unit 96 and do indeed include the combination of the individual images in the image combination unit 98. The combined image data output can be provided in a faster and / or in a more easily processable size. Furthermore, in case the combined image data output is provided in a stepwise manner, the individual data packets can be provided in a faster succession and / or in a more easily processable size.

[0069] As described above, Figure 4 The digital microscope system 100 has a full resolution mode and two reduced resolution sub-modes. Based on a user selection, the control unit 90 selects one of the full resolution mode and the two reduced resolution sub-modes. The control unit 90 further controls the image sensor driver 124 and the image post-processing unit 96 in accordance with the selected one of the full resolution mode and the two reduced resolution sub-modes.

[0070] According to the exemplary embodiments described with respect to Figure 4 In the full resolution mode, the control unit 90 controls the image sensor driver 124 to read out image data of the full number of predefined image pixels from the image sensor 120. Furthermore, the control unit 90 controls the image post-processing unit 96 to not perform any downsampling of the image data received from the image sensor driver 124. For the exemplary case of an image sensor having 1920 pixels x 1920 pixels, the individual image also has 1920 pixels x 1920 pixels.

[0071] According to the exemplary embodiments described with respect to Figure 4In the exemplary embodiment described, in the first reduced-resolution sub-mode, the control unit 90 controls the image sensor driver 124 to read image data from the image sensor 120 at intervals of two pixels in both dimensions. Further, the control unit 90 controls the image post-processing unit to perform a 2x downsampling on each of the two dimensions. In the exemplary case of an image sensor with 1920 pixels × 1920 pixels, the image sensor driver 124 provides 640 pixels × 640 pixels of image data to the image post-processing unit 96, and the individual image has 320 pixels × 320 pixels after downsampling.

[0072] According to Figure 4 In the exemplary embodiment described, in the second reduced-resolution sub-mode, the control unit 90 controls the image sensor driver 124 to read image data from the image sensor 120, every two pixels in each of the two dimensions. Furthermore, the control unit 90 controls the image post-processing unit to perform a 10x downsampling on each of the two dimensions. In the exemplary case of an image sensor with 1920 pixels × 1920 pixels, the image sensor driver 124 provides 640 pixels × 640 pixels of image data to the image post-processing unit 96, and the individual image has 64 pixels × 64 pixels after downsampling.

[0073] Image combining unit 98 is configured to combine individual images into a combined image. The combined image can be presented to the user on the screen, such as... Figure 3 As shown, the images can be saved to a file for later viewing or provided to the user in any other suitable manner. Given the different number of pixels in the individual images in the full-resolution mode and the first and second reduced-resolution sub-modes, the image combining unit 98 may spend different amounts of time combining the individual images. In particular, the image combining unit 98 may need less time for smaller individual images. The image combining unit 98 can combine the individual images by any suitable procedure, such as by suitable stitching algorithms known per se. For high-quality stitching, individual images with overlap between adjacent individual images can be generated.

[0074] exist Figure 4In an exemplary embodiment, the control unit 90 selects one of a full-resolution mode and a first and a second reduced-resolution sub-mode based on the expansion of the region of interest selected by the user. Specifically, the control unit 90 may have two region thresholds. If the region of interest is smaller than the first region threshold, the full-resolution mode is selected. If the region of interest is larger than the first region threshold but smaller than the second region threshold, the first reduced-resolution sub-mode is selected. If the region of interest is larger than the second region threshold, the second reduced-resolution sub-mode is selected. The basic principle behind these thresholds is as follows: The smaller the region of interest, the greater the likelihood that the user is interested in the details of the sample, thus providing better image quality, i.e., a higher resolution for individual images.

[0075] In addition to controlling the image sensor driver 124 and image post-processing unit 96 according to user selection, the control unit 90 can also be configured to control the platform drive assembly 46 and shutter 122 according to user selection. Specifically, different platform movement modes may exist for the full-resolution mode and the first and second reduced-resolution sub-modes. In particular, the different modes may differ in how the image sensor captures image data when the platform is moving or stationary.

[0076] exist Figure 4 In an exemplary embodiment, in full-resolution mode, image data is captured by image sensor 20 when the platform is in a stopped position. This avoids introducing blurring due to platform movement, thereby optimizing image quality. However, since the platform stops for each individual image, subsequent individual images may be generated at a non-maximum speed. The operating time of the platform-driven components may be a limiting factor in generating individual and combined images.

[0077] exist Figure 4 In an exemplary embodiment, image data is captured by the image sensor 20 as the platform moves, in both the first and second reduced-resolution sub-modes. Specifically, the platform can move at a first platform movement speed in the first reduced-resolution sub-mode and at a second platform movement speed in the second reduced-resolution sub-mode. The second platform movement speed is faster than the first platform movement speed. By capturing image data during platform movement, image data for subsequent individual images can be captured more quickly, enabling faster generation of individual and combined images. Capturing image data during platform movement is particularly effective by subsampling a predefined number of pixels using the image sensor driver 124. As described above, subsampling allows for rapid readout of image data and prepares the image sensor for rapid capture of new image data. This rapid preparation can be utilized by rapidly and continuously capturing image data during platform movement. The platform movement speed and the level of subsampling can be adapted to each other, depending on the characteristics and limitations of the technical components involved.

[0078] The first and second moving speeds can also be set such that no more than an acceptable level of blur is introduced into the combined image. Various factors can be considered in this paper. Based on the characteristics of the optical system and the physical pixel size in the image sensor, it can be determined which region of the sample is associated with a pixel in the image sensor. Furthermore, based on the level of subsampling and / or downsampling, it can be determined which region of the sample is associated with a pixel in the combined image. Additionally, based on the acceptable level of blur in the combined image, it can be determined what the maximum acceptable platform moving speed is to keep the blur below the acceptable level. The acceptable level of blur can be defined relative to the number of neighboring pixels in the combined image affected by a given point in the sample. For example, a quality standard can be set where any given point in the sample affects only two pixels in the combined image. This is also referred to as 1-pixel blur, because any given point in the sample may affect at most one pixel than if image data is captured when the platform stops. Based on the acceptable 1-pixel blur, the maximum platform moving speed can be set to the length of the sample region associated with a pixel in the combined image divided by the shutter opening time of one image capture operation of the image sensor. If the readout operation of captured image data is important compared to the shutter opening time, the maximum platform movement speed can also be set to the length of the sample region associated with the pixels in the combined image divided by the total capture and processing time of a single image capture operation of the digital camera 20. Potential further limitations on transferring image data from the image sensor 120 to a memory that can provide the image data for further processing independent of real-time can also be considered. By setting the first and second platform movement speeds below the corresponding maximum platform movement speeds, as set for specific operating scenarios of the first and second reduced-resolution sub-modes, the blurriness of the combined image can be kept at an acceptable level while allowing the combined image to be delivered very quickly.

[0079] As received by control unit 90, the desired pixel size of the composite image may also be part of the user's selection. In other words, the user's selection may include information about the desired resolution of the composite image. For example, the number of pixels on screen 70 may be part of the information transmitted to control unit 90. Control unit 90 may be configured to select one of a full-resolution mode and one or more reduced-resolution (sub)modes based on the expansion of the region of interest and the desired resolution of the composite image. For example, control unit 90 may be configured to adjust the first and second region thresholds as described above according to the desired resolution of the composite image. In particular, the first and second region thresholds may be increased to obtain a higher desired resolution. In this way, the quality of the composite image can be adapted to the output medium, and a high-quality composite image can be presented to the user on the specific output medium used.

[0080] To allow users to work on already available portions of the combined image, the combined image data can be displayed to the user in a step-by-step manner. For example, when a new individual image becomes available from the image post-processing unit 96, the image combining unit 98 can perform stitching of that individual image with previously received individual images. This process can also be considered as stitching the newly received individual image with the already available portions of the combined image. The stitching operation may change the image data of the individual images and can therefore be viewed as a filtering operation. The filtered individual images can be provided to the screen and displayed together with the previously available portions of the combined image, thereby increasing the available display portion of the combined image. On the screen, the user can experience this process as gradually building the combined image. This can improve the efficiency of the user's sample analysis. For example, displaying the already available portions of the combined image allows the user to be at the location of updating the region of interest, such as by zooming in before the entire combined image is presented. It can also allow the user to find the specific features he / she is looking for in the sample before the entire combined image is presented.

[0081] Figure 5 A flowchart illustrating a method for providing a combined image using a digital microscope according to an exemplary embodiment of the present invention is shown. In step 200, a user selection regarding a region of interest (ROI) of a sample is received. In step 202, based on the user selection received in step 200, one of a full-resolution mode and a reduced-resolution mode is selected, possibly a full-resolution mode and one of multiple reduced-resolution sub-modes. Depending on the mode selected in step 202, separate images of individual portions of the ROI are generated in step 204. Based on the selected mode for generating separate images, all or a subset of the illumination assembly, platform drive assembly, digital camera shutter, digital camera image sensor driver, and image post-processing unit are controlled according to the selected mode. In step 206, the separate images are combined to generate a combined image. In step 208, the combined image is displayed to the user on a screen.

[0082] It should be noted that steps 204, 206, and 208 can be performed in the order depicted. Steps 204, 206, and 208 can also be performed in a partially parallel manner. As discussed above, the combination of individual images and the display of potentially combined images may have already occurred for a portion of the region of interest, while individual images for another portion of the region of interest are still generated.

[0083] exist Figure 5In an exemplary embodiment, the method can be configured to react to updated user selections regarding an updated region of interest at any point throughout the method. Specifically, the method can be configured to interpret the receipt of an updated user selection as an interruption and halt one or more currently executing steps. The method can interpret an updated user selection as an interruption and can return to step 202 based on the updated user selection. This is in Figure 5 The dashed line 210 illustrates the reception of updated user selections, which may occur at any point throughout the method. Individual images generated in the previous instance of step 204 and still relevant to the updated user selections may be reused instead of being regenerated. This allows for very rapid generation of combined images for minor changes in the region of interest. Furthermore, portions of the combined image still relevant to the updated region of interest and already generated in the previous instance of step 206 can continue to be displayed on the screen. It can gradually supplement other portions of the combined image of the updated region of interest. This prevents screen downtime that would disrupt user analysis due to the user having to adapt to a completely new display on the screen.

[0084] Although the invention has been described with reference to exemplary embodiments, those skilled in the art will understand that various changes can be made and elements can be substituted with equivalents without departing from the scope of the invention. Furthermore, many modifications can be made to adapt particular situations or materials to the teachings of the invention without departing from its essential scope. Therefore, the invention is intended to be limited to the specific embodiments disclosed, but rather to include all embodiments falling within the scope of the appended claims.

Claims

1. Method of providing a combined image using a digital microscope (2) having an optical system (25), an image sensor (120) having a predefined number of image pixels, and a stage (10) for holding a sample (12), the stage (10) being movable relative to the optical system (25) and the image sensor (120), the method comprising: receiving a user selection regarding a region of interest (74) of the sample (12), the user selection indicating a position and an extension of the region of interest (74); selecting one of a full resolution mode and a reduced resolution mode depending on the user selection, wherein in the full resolution mode individual images having the predefined number of image pixels are generated, and wherein in the reduced resolution mode individual images having a reduced number of image pixels compared to the predefined number of image pixels are generated; moving the stage (10) relative to the optical system (25) and the image sensor (120) and generating individual images of the region of interest (74) in accordance with the selected one of the full resolution mode and the reduced resolution mode; combining the individual images into a combined image representing the region of interest (74); and displaying the combined image on a screen (70), the individual images being displayed on the screen (70) in a stepwise manner; wherein the method further comprises: receiving an updated user selection regarding an updated region of interest of the sample while the individual images are displayed on the screen in the stepwise manner, the updated user selection indicating a position and an extension of the updated region of interest; interrupting one or more method steps currently being performed; reselecting one of the full resolution mode and the reduced resolution mode depending on the updated user selection; moving the stage (10) relative to the optical system and the image sensor and generating individual images of the updated region of interest in accordance with the reselected one of the full resolution mode and the reduced resolution mode; and combining the individual images into a combined image representing the updated region of interest. the individual images having the reduced number of image pixels are generated by subsampling the predefined number of image pixels of the image sensor (120).

2. The method of claim 1, wherein, the individual images having the reduced number of image pixels are generated by downsizing image data generated by the image sensor (120).

3. The method of claim 1, wherein, the individual images having the reduced number of image pixels are generated by downsizing image data generated by the image sensor (120).

4. The method of claim 2, wherein, the stage (10) is moved in the reduced resolution mode in an at least partially continuous manner, wherein the image sensor (120) captures image data for the individual images while the stage is moved.

5. The method according to any of the preceding claims, characterized in that, a movement speed of the stage (10) is selected such that blurring of the image data is limited to at most 2 pixels.

6. The method of claim 5, wherein, a movement speed of the stage (10) is selected such that blurring of the image data is limited to at most 1 pixel.

7. The method of claim 6, wherein, the stage (10) is moved in the full resolution mode in an intermittent manner, the image sensor (120) capturing image data for the individual images while the stage is in a stopped position.

8. The method according to any of the preceding claims 1 - 4, 6 - 7, characterized in that, ​ 9. The method of claim 5, wherein, The platform (10) is moved in the full resolution mode in an intermittent manner, the image sensor (120) capturing image data of individual images when the platform is in a stop position.

10. The method according to any of the preceding claims 1 - 4, 6 - 7, 9, characterized in that, The reduced resolution mode comprises a plurality of reduced resolution sub-modes, and wherein the step of selecting one of the full resolution mode and the reduced resolution mode comprises selecting one of the full resolution mode and one of the plurality of reduced resolution sub-modes.

11. The method of claim 5, wherein, The reduced resolution mode comprises a plurality of reduced resolution sub-modes, and wherein the step of selecting one of the full resolution mode and the reduced resolution mode comprises selecting one of the full resolution mode and one of the plurality of reduced resolution sub-modes.

12. The method of claim 8, wherein, The reduced resolution mode comprises a plurality of reduced resolution sub-modes, and wherein the step of selecting one of the full resolution mode and the reduced resolution mode comprises selecting one of the full resolution mode and one of the plurality of reduced resolution sub-modes.

13. The method according to claim 11 or 12, characterized in that, Each of the plurality of reduced resolution sub-modes has a respective sub-mode specific reduced number of image pixels for the individual images, wherein the plurality of reduced resolution sub-modes differ from at least one of a level of sub-sampling of the predefined number of image pixels of the image sensor (120) and a level of downscaling of the image data generated by the image sensor (120).

14. The method of claim 10, wherein, Each of the plurality of reduced resolution sub-modes has a respective sub-mode specific reduced number of image pixels for the individual images, wherein the plurality of reduced resolution sub-modes differ from at least one of a level of sub-sampling of the predefined number of image pixels of the image sensor (120) and a level of downscaling of the image data generated by the image sensor (120).

15. The method according to any of the preceding claims 1-4, 6-7, 9, 11-12, 14, characterized in that, The user selection further indicates a representation resolution, the representation resolution indicating a screen resolution of a screen (70) for depicting the combined image.

16. The method of claim 5, wherein, The user selection further indicates a representation resolution, the representation resolution indicating a screen resolution of a screen (70) for depicting the combined image.

17. The method of claim 8, wherein, The user selection further indicates a representation resolution, the representation resolution indicating a screen resolution of a screen (70) for depicting the combined image.

18. The method of claim 10, wherein, The user selection further indicates a representation resolution, the representation resolution indicating a screen resolution of a screen (70) for depicting the combined image.

19. The method of claim 13, wherein, The user selection further indicates a representation resolution, the representation resolution indicating a screen resolution of a screen (70) for depicting the combined image. The user selection further indicates a representation resolution, the representation resolution indicating a screen resolution of a screen (70) for depicting the combined image.

20. Digital microscope system (100) comprising: an optical system (25); an image sensor (120) having a predefined number of image pixels; a platform (10) for holding a sample (12); a platform drive assembly (46) for moving the platform (10) relative to the optical system (25) and the image sensor (120); and a control unit (90) for controlling the generation of image data for a combined image, wherein the control unit is configured to: receive a user selection regarding a region of interest (74) of the sample (12), the user selection indicating a position and an extension of the region of interest (74); - depending on the user selection, one of a full resolution mode and a reduced resolution mode is selected, wherein in the full resolution mode separate images are generated having a predefined number of image pixels, wherein in the reduced resolution mode separate images are generated having a reduced number of image pixels compared to the predefined number of image pixels; - controlling the platform drive assembly (46) to move the platform (10) relative to the optical system (25) and the image sensor (120); - controlling the generation of separate images of the region of interest (74) in accordance with the one of the full resolution mode and the reduced resolution mode selected; - controlling the combining of the separate images into a combined image representing the region of interest (74); and - controlling the display of the combined image on the screen (70), the separate images being displayed on the screen (70) in a stepwise manner; wherein the control unit is further configured to: - receive an updated user selection of an updated region of interest with respect to the sample while the separate images are displayed on the screen in the stepwise manner, the updated user selection indicating a position and an extension of the updated region of interest; - interrupt one or more method steps currently being performed; - depending on the updated user selection, reselecting one of the full resolution mode and the reduced resolution mode; - controlling the platform drive assembly (46) to move the platform (10) relative to the optical system (25) and the image sensor (120); - controlling the generation of separate images of the region of interest in accordance with the one of the full resolution mode and the reduced resolution mode reselected; and - controlling the combining of the separate images into a combined image representing the updated region of interest.

21. The digital microscope system (100) of claim 20, characterized in that The control unit (90) is configured to control the generation of the separate images in the reduced resolution mode by at least one of sub-sampling and downscaling the predefined number of image pixels of the image sensor from the image data generated by the image sensor.

22. The digital microscope system (100) according to claim 20 or 21, characterized in that The control unit (90) is configured to control the platform drive assembly (46) to move the platform in an at least partially continuous manner in the reduced resolution mode and to control the image sensor (120) to capture image data for the separate images while the platform is moved.

23. Program product for providing a combined image using a digital microscope (2) having an optical system (25), an image sensor (120) having a predefined number of image pixels and a platform (10) for holding a sample (12), the platform (10) being movable relative to the optical system (25) and the image sensor (120), the program comprising: receiving a user selection of a region of interest (74) with respect to the sample (12), the user selection indicating a position and an extension of the region of interest (74); depending on the user selection, one of a full resolution mode and a reduced resolution mode is selected; instructing the platform (10) to move relative to the optical system (25); instructing the image sensor (120) to generate image data; On the basis of the image data, if the full resolution mode is selected, a separate image of the region of interest (74) is generated having a predefined number of image pixels, and if the reduced resolution mode is selected, a separate image of the region of interest (74) is generated having a reduced number of image pixels; combining the separate images into a combined image representing the region of interest (74); and indicating the display of the combined image on a screen (70), the separate images being displayed on the screen (70) in a stepwise manner; wherein the program further comprises: receiving an updated user selection of an updated region of interest with respect to the sample while the separate images are displayed on the screen in a stepwise manner, the updated user selection indicating a position and an extension of the updated region of interest; interrupting one or more method steps that are currently being performed; reselecting one of the full resolution mode and the reduced resolution mode depending on the updated user selection; indicating the movement of the platform (10) relative to the optical system and the image sensor; indicating the image sensor (120) to generate image data; generating, on the basis of the image data, a separate image of the updated region of interest in accordance with the reselected one of the full resolution mode and the reduced resolution mode; and combining the separate images into a combined image representing the updated region of interest.

Citation Information

Patent Citations

  • Method and device for microscopically examining a sample, computer program and computer program product

    US20100194873A1