Optical imaging system, apparatus for an optical imaging system, method and computer program

Abstract

Examples relate to an optical imaging system 100 comprising a microscope (120). The microscope (120) comprises at most one optical imaging sensor (122) configured to acquire a first image of a sample (110) based on a first frequency range of visible light and a second image of the sample (110) based on a second frequency range of infrared light. Further, the microscope (120) comprises a first optical element (116) arranged along an optical path (140) of the microscope (120) for collimating a beam of light. The microscope (120) further comprises a second optical element (118) arranged along the optical path (140) of the microscope (120) defining the first aperture of the microscope (120) for the first frequency range of visible light. Further, the microscope (120) comprises an infrared filter (126) arranged along the optical path (140) of the microscope (120) defining a second aperture of the microscope (120) for the second frequency range of infrared light. The first aperture is different from the second aperture.

Description

[0001] Optical Imaging System, Apparatus for an Optical Imaging System, Method and Computer Program

[0002] Technical field

[0003] Examples relate to an optical imaging system, such as surgical optical imaging system, and an apparatus for an optical imaging system, a method, and a computer program.

[0004] Background

[0005] In a multi-sensor microscope, multiple sensors or cameras are used to simultaneously capture images at different focal planes. Each sensor is positioned at a specific focal distance within the system. By capturing images at various focal planes, the microscope can obtain information from different depths in the sample, effectively extending the depth of field (DOF).

[0006] The captured images from each sensor can then be combined using software algorithms to create a final composite image that exhibits an extended DOF. This technique allows for greater clarity and detail throughout the sample, even in regions that would normally appear out of focus in a single image. However, the use of multiple sensors or cameras is costly and resource intensive.

[0007] Alternatively, wave-front coding and color-coding optical element on the aperture are usually used on single sensor camera system by implementing diffraction or color cutting filter on the aperture plane, respectively.

[0008] Wave-front coding requires a complete camera model and calculates the diffraction parameters of the optical element. However, the parameters from the model are not accurate since the error accumulation in the real world is hard to model. This leads to the inaccurate design of the optical element which will cause more artifact in postprocessing. Color coding allows different color of light to pass different size of aperture. However, the total amount of light is reduced when passing color coded optical elements which reduces the image quality. Thus, there may be a desire for an improved concept for improving a DOF of an optical imaging system.

[0009] Summary

[0010] This desire is addressed by the subject-matter of the independent claims.

[0011] The concept proposed in the present disclosure is based on the insight, that a DOF of a first image can be increased based on a DOF of a second image utilizing at most one optical imaging sensor and an infrared filter to provide two different apertures for a microscope. The different apertures may be provided for different wavelengths captured by the at most one optical imaging sensor. In this way, image acquisition with different DOF for different wavelengths can be achieved. By combining images acquired with different DOF, a DOF for a desired wavelength can be increased.

[0012] Examples provide an optical imaging system comprising a microscope. The microscope comprises at most one optical imaging sensor configured to acquire a first image of a sample based on a first frequency range of visible light and a second image of the sample based on a second frequency range of infrared light. Further, the microscope comprises a first optical element arranged along an optical path of the microscope for collimating a beam of light. The microscope further comprises a second optical element arranged along the optical path of the microscope defining the first aperture of the microscope for the first frequency range of visible light. Further, the microscope comprises an infrared filter arranged along the optical path of the microscope defining a second aperture of the microscope for the second frequency range of infrared light. The first aperture is different from the second aperture. The infrared filter may be arranged between the first optical element and the second optical element. For example, the infrared filter may be arranged in an area of the optical path, in which the beam of light is collimated. The infrared filter may be opaque to infrared light and transparent to visible light. Thus, by using the infrared filter the aperture of the microscope for infrared light can be set to a different value than the aperture of the microscope for visible light. In this way, images with different DOF can be acquired. By combining the acquired images with different DOFs, DOF of a desired wavelength, e.g., visible light, can be increased. In an example, the first aperture may be greater than the second aperture. When the second aperture is smaller, a DOF of the image acquired with infrared light may be increased. Thus, the information about the image received from the increased DOF of the infrared light can be used to post process an image acquired using visible light. The post processing may increase a DOF of the image acquired with visible light.

[0013] In an example, the first aperture may be at least 200% greater than the second aperture. In this way, a desired relationship between the DOF of infrared light and visible light can be set.

[0014] In an example, a DOF of the first image may be lower than a DOF of the second image. Thus, the second image can be used to increase the DOF of the first image.

[0015] In an example, the at most one optical imaging sensor may be an RGB infrared optical imaging sensor. An RGB infrared sensor, also known as a multispectral sensor, is an optical imaging sensor that combines the ability to capture visible light (RGB) and infrared radiation. Thus, the first image and the second image can be acquired by at most one optical imaging sensor in a facilitated way.

[0016] In an example, the infrared filter may have an opening arranged along an optical axis of the optical path. The opening may define an area of the infrared filter which is transparent to infrared light. Thus, the opening may define the aperture of the microscope for infrared light.

[0017] In an example, the infrared filter may be transparent to the first frequency range. Thus, the infrared filter may not affect image acquisition using visible light.

[0018] In an example, the optical imaging sensor may be configured to receive visible light from the sample indicative of the first image and infrared light from the sample indicative of the second image at the same time. Thus, the image acquisition of the first image and the second image can be performed simultaneously.

[0019] In an example, the optical imaging system may further comprise an apparatus configured to receive sensor data from the at most one optical imaging sensor. The sensor data is indicative of a first image of the sample based on a first frequency range of visible light and a second image of the sample based on a second frequency range of infrared light. A DOF of the second image is greater than a DOF of the first image. Further, the apparatus is configured to increase a DOF of the first image based on the DOF of the second image. Thus, the apparatus can improve or correct the first image utilizing the second image. For example, a contour in the first image can be sharpened based on the second image. In this way, an improved (first) image (i.e., increased DOF) for displaying to the user can be generated.

[0020] Examples provide an apparatus configured to receive sensor data from at most one optical imaging sensor. The sensor data is indicative of a first image of the sample based on a first frequency range of visible light and a second image of the sample based on a second frequency range of infrared light. A DOF of the second image is greater than a DOF of the first image. Further, the apparatus is configured to increase a DOF of the first image based on the DOF of the second image. Thus, the apparatus can generate an improved (fist) image by combining the high resolution of the first image with the increased DOF of the second image.

[0021] Examples provide a method for an optical imaging system. The method comprises receiving a first image of a sample based on a first frequency range of visible light and receiving a second image of the sample based on a second frequency range of infrared light. A DOF of the second image is greater than a DOF of the first image. Further, the method comprises increasing a depth of field of the first image based on the depth of field of the second image.

[0022] Various examples of the present disclosure relate to a corresponding computer program with a program code for performing the above method when the computer program is executed on a processor.

[0023] Short description of the Figures

[0024] Some examples of apparatuses and / or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which

[0025] Figs, la and lb show schematic diagrams of an example of an apparatus for an optical imaging system and of a corresponding optical imaging system;

[0026] Figs. 2a-c show an example of a top view of an infrared filter and plots of a corresponding optical imaging sensor response curve and a transmission curve; Figs. 3a-c show a traditional first image, an improved (first) image and a post processed improved (first) image;

[0027] Fig. 4 shows a flow chart of an example of a method for an optical imaging system; and

[0028] Fig. 5 shows a schematic diagram of a system comprising a microscope and a computer system.

[0029] Detailed Description

[0030] Various examples will now be described more fully with reference to the accompanying drawings in which some examples are illustrated. In the figures, the thicknesses of lines, layers and / or regions may be exaggerated for clarity.

[0031] Figs, la and lb show schematic diagrams of an example of an apparatus 130 for an optical imaging system 100 and of a corresponding optical imaging system 100, e.g., comprising the apparatus 130. The apparatus 130, as shown in Fig. lb, is tasked with controlling various aspects of a microscope 120 of the optical imaging system 100, which may be a surgical optical imaging system, and of the entire optical imaging system 100 and / or with processing various types of sensor data of the optical imaging system 100. Consequently, the apparatus 130 may be implemented as a computer system, which interfaces with the various components of the optical imaging system, e.g., the sensor 122.

[0032] The apparatus 130 comprises, as shown in Fig. lb, one or more processors 134 and one or more storage devices 136. Optionally, the apparatus 130 further comprises one or more interfaces 132. The one or more processors 134 are coupled to the one or more storage devices 136 and to the optional one or more interfaces 132. In general, the functionality of the apparatus 130 may be provided by the one or more processors 134 (for increasing the DOF of a first image), in conjunction with the one or more interfaces 132 (for exchanging information, e.g., with at most one optical imaging sensor 122) and / or with the one or more storage devices 136 (for storing and / or retrieving information). The apparatus 130 is configured to receive sensor data from at most one optical imaging sensor 122. The sensor data is indicative of a first image of the sample 110 based on a first frequency range of visible light and a second image of the sample 110 based on a second frequency range of infrared light. A DOF of the second image is greater than a DOF of the first image. Further, the apparatus 130 is configured to increase a DOF of the first image based on the DOF of the second image.

[0033] The proposed concept may be built around two main components - the microscope 120, which comprises the optical components, and the apparatus 130, which may be used to control the optical imaging system 100, process sensor data of the microscope 120, e.g., the optical imaging sensor 122, and / or to increase a DOF of the first image.

[0034] In general, a microscope, such as the microscope 120, is an optical instrument that is suitable for examining objects that are too small to be examined by the human eye (alone). For example, a microscope 120 may provide an optical magnification of a sample, such as a sample 110 shown in Fig. la. In modern microscopes, the optical magnification is often provided for a camera or an imaging sensor, such as the optical imaging sensors 122 of the microscope 120.

[0035] There are a variety of different types of optical imaging systems. If the optical imaging system 100 is used in the medical or biological fields, the sample 110 may be a sample of organic tissue, e.g., arranged within a petri dish or present in a part of a body of a patient. In some examples of the present disclosure the optical imaging system 100 may be a surgical optical imaging system, e.g., an optical imaging system that is to be used during a surgical procedure, such as an oncological surgical procedure or during tumor surgery. However, the proposed concept may also be applied to other types of microscopy, e.g., microscopy in a laboratory or microscopy for the purpose of material inspection.

[0036] As can be seen in Fig. la, the optical imaging system 100 comprises a microscope 120. The microscope 120 comprises at most one optical imaging sensor 122 configured to acquire a first image of a sample 110 based on a first frequency range of visible light and a second image of the sample 110 based on a second frequency range of infrared light. For example, the at most one optical imaging sensor 122 may be a multispectral optical imaging sensor or a hyperspectral optical imaging sensor. A multi spectral / hyperspectral optical imaging sensor may be a sensor that captures data across multiple spectral bands or wavelengths of electromagnetic radiation. Unlike traditional RGB sensors that capture only red, green, and blue light, multi spectral / hyperspectral sensors can detect a broader range of wavelengths, including ultraviolet, visible, and infrared radiation. Thus, the first image of the sample 110 and the second image of the sample 110 can be acquired by only one optical imaging sensor 122.

[0037] The first frequency range and / or the second frequency range may consist of a single wavelength or may be wavelength range. For example, the first wavelength may comprise the wavelength ranges corresponding to red, green, and blue regions of the visible spectrum.

[0038] The at most one optical imaging sensor may acquire the first image and / or second image by generating sensor data. The sensor data may be indicative of the first image and the second image of the sample 110. The sensor data can be utilized by the apparatus 130 to increase a DOF of the first image. For example, the apparatus 130 may receive the sensor data as raw data, i.e., the optical imaging sensor 122 may have performed no post process of the sensor data. Alternatively, the apparatus 130 may receive post processed sensor data.

[0039] The first frequency range may comprise visible light and the second frequency range may comprise infrared light, e.g., especially near infrared light.

[0040] Further, the microscope 120 comprises a first optical element 116 arranged along an optical path 140 of the microscope 120 for collimating a beam of light. The first optical element 116 may be a collimating optical element positioned along the optical path 140 of the microscope 120. For example, the first optical element 116 may be arranged between the sample 110 and the optical imaging sensor 122. For example, the first optical element 116 may be arranged between the infrared filter 126 and the sample 110.

[0041] In the microscope 120 the first optical element 116 may be used to collimate the light that emanates from the sample 110. The collimated beam of light is formed by the first optical element 116 when the rays of light exiting the sample 110 are aligned parallel or nearly parallel to each other.

[0042] The first optical element 116 may take the diverging or non-parallel rays of light emitted by the sample 110 and may convert them into a collimated beam of light. It may accomplish this by adjusting the curvature and refractive properties of the lens surface to redirect the light rays in such a way that they become parallel. The first optical element 116 may be a collimator lens or a reflective diffraction grating.

[0043] The collimated beam of light is then directed to subsequent optical components, such like the second optical element 118, the infrared filter 126 or a detector, such like the optical imaging sensor 122. The first optical element 116 may ensure that the infrared filter 126 stays within the collimated beam of light, i.e., within a parallel light path. In this way, the second aperture defined by the infrared filter 126 can be defined in an improved way.

[0044] The microscope 120 further comprises a second optical element 118 arranged along the optical path 140 of the microscope 120 defining the first aperture of the microscope 120 for the first frequency range of visible light. The second optical element 118 may be a converging optical element positioned along the optical path 140 of the microscope 120. For example, the second optical element 118 may be arranged between the sample 110 and the optical imaging sensor 122. For example, the second optical element 118 may be arranged between the infrared filter 126 and the optical imaging sensor 122.

[0045] The second optical element 118 is for converging the collimated beam of light. For example, the second optical element 118 may define the image formed by the sample 110 onto the imaging sensor 122. The second optical element 118 may help that the first image and / or the second image is properly magnified, corrected for aberrations, and / or positioned for viewing or acquisition.

[0046] In the microscope 120, for example, the second optical element 118 may be placed at a specific distance from the sample 110 to achieve the desired magnification and image quality. The second optical element 118 may be a tube lens or a telecentric lens.

[0047] Further, the microscope 120 comprises an infrared filter 126 arranged along the optical path 140 of the microscope 120 defining a second aperture of the microscope 120 for the second frequency range of infrared light. The first aperture is different from the second aperture. The infrared filter 126 may be an optical filter that selectively blocks or attenuates infrared light while allowing visible light to pass through. It is designed to filter out infrared wavelengths, allowing only the desired range of visible light to reach the optical imaging sensor 122. The infrared filter 126 may be arranged within the collimated beam of light, for example. Thus, the infrared filter 126 may be arranged between the first optical element 116 and the second optical element 118.

[0048] Using the infrared filter 126 may allow to set to different DOFs for the microscope 120 using only one optical imaging sensor 122. A typical microscope usually has a limited DOF, which is the range of distances within the sample 110 that appear sharp and in focus. This limitation is due to the characteristics of the numerical aperture of the microscope. The infrared filter 126 may allow to set different DOFs. Thus, information of simultaneously acquired images can be combined to post process an image based on another image. For example, the first image can be sharpened by using information of the second image, or vice versa. Depending on the first aperture and the second aperture either the first image or the second image may be sharper. For example, the image acquired with the wavelength of light which has the smaller aperture along the optical path 140 may be sharper.

[0049] With the infrared filter 126 the limitation of the DOF can be overcome. Further, multi -focus imaging of focus stacking can be avoided. An improved image of the sample 110 for a desired wavelength can be achieved in a facilitated way, e.g., using the DOF of the second image to increase the DOF of the first image.

[0050] For example, a user of the optical imaging system 100 may need at larger DOF to get a better sense of a distance between a tool and a surgical spot. The wrong sense of the distance could lead to serious surgical accident. Decreasing the aperture of the visible light to increase the DOF may decrease the definition of the image of the sample 110. This may cause a poor user, e.g., surgery, experience. Utilizing the infrared filter 126 may allow to increase the DOF without decreasing the definition of the image of the sample 110.

[0051] The infrared filter 126 allows to involve infrared light, e.g., the near infrared light, in the optical imaging system 100. Thus, different sizes of apertures can be set. For example, the infrared filter 126 may be a near infrared cutting filter to allow the visible light to pass, i.e., the aperture of the visible light is defined by the normal size aperture of the optical imaging system 100, while the near infrared light passes the infrared filter 126, i.e., the aperture of the near infrared light is defined by the infrared filter 126 and it is therefore different. In this way, the image acquired using visible light is not affected, e.g., a visible light intensity is not affected. Only the intensity of the near infrared light is affected by the infrared filter 126. In this way, two different apertures can be set for the optical imaging system 100. Based on the two different apertures the first image may have a higher resolution and the second image may have an increased DOF. Thus, by combining both, a resulting improved image can be generated, providing increased DOF while maintaining high resolution. Moreover, a design of the optical imaging system 100 can be facilitated, e.g., a complex design or a use of a diffraction optical element such like a Bragg cell can be avoided.

[0052] In an example, the first aperture may be greater than the second aperture. Alternatively, the first aperture may be smaller than the second aperture.

[0053] Using the microscope 120 different aperture sizes can be used to generate an improved image, e.g., by the apparatus 130. For example, if the first aperture is greater than the second aperture the visible light path may have a larger aperture size, which may guarantee a high definition of the first image acquired using visible light. Further, the second image acquired using infrared light may provide a larger DOF due to the smaller aperture. By postprocessing the first image acquired using visible light and the second image acquired using infrared light a high- definition image with increased DOF can be generated. For example, the DOF of the first image can be increased based on the second image. Further, the infrared filter 126 may replace traditional infrared filters used in optical imaging systems. In this way, a complexity of the optical imaging system 100 can be reduced.

[0054] In an example, the first aperture may be at least 100%, at least 150%, at least 200%, at least 250% or at least 300% greater than the second aperture. By defining a relationship between the first aperture and the second aperture a desired DOF can be adjusted. For example, the infrared filter 126 may be adjustable, such that the second aperture can be adjusted. Thus, the DOF can be increased to an actual need of user.

[0055] For example, the infrared filter 126 may be an iris diaphragm. The iris diaphragm may be a circular adjustable aperture, e.g., located near the objective lens of a microscope 120. It may consist of overlapping metal blades that can be opened or closed to control the amount of light entering the objective. While its primary purpose is to adjust the illumination and DOF, the iris diaphragm can also act as an adaptive filter for infrared light. Alternatively, the infrared filter 126 may be a liquid crystal tunable filter or an acousto-optic tunable filter. In an example, a DOF of the first image may be lower than a DOF of the second image. Thus, the second image can be used to increase the DOF of the first image. For example, the apparatus 130 may post process the sensor data to increase the DOF of the first image based on the increased DOF of the second image.

[0056] In an example, the at most one optical imaging sensor 122 may be an RGB infrared optical imaging sensor. A multispectral sensor may allow to acquire the first image and the second image simultaneously. Thus, a reliability of the generated first image can be improved. For example, it can be ensured that the second image used to increase the DOF of the first image actually shows the same (area of the) sample 110 at the same condition.

[0057] In an example, the infrared filter 126 has an opening arranged along an optical axis 142 of the optical path 140. The opening 128 may define the second aperture. For example, an outer dimension of the opening 128 may define the aperture. The opening 128 may have any desired shape such like circular or rectangular. For example, the infrared filter 126 may have a two- dimensional torus shape, such like a ring or a donut. The opening 126 of the infrared filter, i.e., the center of the infrared filter 126, may be transparent to both visible light and infrared light. The opening 126 may be a recess / cutout.

[0058] In an example, the infrared filter 126 is substantially transparent to the first frequency range. Thus, the infrared filter may not affect image acquisition using visible light. Alternatively, the infrared data 126 may be semi-transparent to visible light.

[0059] In an example, the optical imaging sensor 122 may be configured to receive visible light from the sample 110 indicative of the first image and infrared light from the sample 110 indicative of the second image at the same time. For example, the optical imaging sensor 122 may measure the first image using visible light and the second image using infrared light simultaneously. An image acquisition of the first image and the second image can be performed at the same time by the optical imaging sensor 122. Therefore, the sensor data may be indicative of the first image and of the second image.

[0060] In an example, the optical imaging system 100 may further comprise an apparatus 130 configured to receive sensor data from the at most one optical imaging sensor 122. The sensor data may be indicative of a first image of the sample 110 based on a first frequency range of visible light and a second image of the sample 110 based on a second frequency range of infrared light. A DOF of the second image may be greater than the DOF of the first image. Further, the apparatus 130 may be configured to increase a DOF of the first image based on the DOF of the second image.

[0061] As shown in Fig. lb the optional one or more interfaces 132 is coupled to the respective one or more processors 134 at the apparatus 130. In examples the one or more processors 134 may be implemented using one or more processing units, one or more processing devices, any means for processing, such as a processor, a computer or a programmable hardware component being operable with accordingly adapted software. Similar, the described functions of the one or more processors 134 may as well be implemented in software, which is then executed on one or more programmable hardware components. Such hardware components may comprise a general-purpose processor, a Digital Signal Processor (DSP), a micro-controller, etc. The one or more processors 134 is capable of controlling the one or more interfaces 132, so that any data transfer that occurs over the one or more interfaces 132 and / or any interaction in which the one or more interfaces 132 may be involved may be controlled by the one or more processors 134.

[0062] In an embodiment the apparatus 130 may comprise a memory, e.g., the one or more storage devices 136 and at least one or more processors 134 operably coupled to the memory and configured to perform the method described below.

[0063] In examples the one or more interfaces 132 may correspond to any means for obtaining, receiving, transmitting or providing analog or digital signals or information, e.g., any connector, contact, pin, register, input port, output port, conductor, lane, etc. which allows providing or obtaining a signal or information. The one or more interfaces 132 may be wireless or wireline and it may be configured to communicate, e.g., transmit or receive signals, information with further internal or external components.

[0064] The apparatus 130 may be a computer, processor, control unit, (field) programmable logic array ((F)PLA), (field) programmable gate array ((F)PGA), graphics processor unit (GPU), application-specific integrated circuit (ASICs), integrated circuits (IC) or system-on-a-chip (SoCs) system. The apparatus 130 may be part of the microscope 120. Alternatively, the apparatus 130 may be external to the microscope 120 and may communicate with the microscope 120, e.g., with the optical imaging sensor 122.

[0065] More details and aspects are mentioned in connection with the examples described below. The example shown in Fig. 1 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described below (e.g., Fig. 2 - 5).

[0066] Figs. 2a-c show an example of a top view of an infrared filter 226 and plots of a corresponding optical imaging sensor response curve and a transmission curve. As can be seen in Fig. 2a the infrared filter 226 may have a two-dimensional torus shape. The infrared filter 226 may comprise a first region 250 and the second region 252. The first region 250 may be formed by an opening 250. The opening 250 may be (fully) transparent to both visible light and infrared light. The second region 252 may be opaque to infrared light. Thus, the aperture for infrared light may be defined by the opening 250. The aperture of visible light may be not affected by the infrared filter 226. Thus, Fig. 2a demonstrates an example of a near infrared filter design with a smaller aperture for the (near) infrared light and a larger aperture for the visible light.

[0067] Fig. 2b shows a response curve of an optical imaging sensor, e.g., an optical imaging sensor as described with reference to Fig. 1. The response curve comprises information about the response of the optical imaging sensor for blue 260, green 262, red 264 and near infrared light 266. The relative sensitivity is plotted over the wavelength of light. As can be seen, the response curve suggests that the optical imaging sensor can receive the RGB and near infrared light at the same time.

[0068] Fig. 2c shows a transmission curve of the infrared filter 226. The intensity is plotted over the wavelength. The graph 270 corresponds to the transmission curve of the first region 250. It can be seen that neither the visible light nor the infrared light is blocked by the infrared filter 226. The graph 272 corresponds to the transmission curve of the second region 252. For the visible light can nearly no loss of intensity can be observed. In contrast, the infrared light is blocked by region 252. This leads to a decreased intensity for wavelengths above roughly 630 nm. The spectrum shown in Fig. 2c is an example for an infrared filter for illustration purposes. Other infrared filters may have a different cutoff frequency. For example, a cutoff frequency may be in the range from 600 nm - 800 nm. The cutoff frequency can be selected according to an optical imaging sensor, for example.

[0069] More details and aspects are mentioned in connection with the examples described above and / or below. The example shown in Fig. 2 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1) and / or below (e.g., Fig. 3 - 5).

[0070] Figs. 3a-c show a traditional first image, an improved (first) image and a post processed improved (first) image. Fig. 3a shows a traditional image acquired using an RGB optical imaging sensor. The large aperture may result in a smaller DOF and a high resolution. Fig. 3b shows an improved (first) image. The improved (first) image is generated based on sensor data indicative of a first image acquired with visible light and a second image acquired with infrared light. The improved (first) image may combine the benefit of a large DOF of the second image acquired using infrared light and the high resolution of the first image acquired using visible light. However, the color may be pinkish because of the existing data of the infrared light. The improved (first) image may be generated by an apparatus as described with reference to Fig. 1.

[0071] Fig. 3c shows a post processed improved (first) image. The original image color can be restored. Further, the post processed improved (first) image may have a larger DOF without a loss of resolution.

[0072] More details and aspects are mentioned in connection with the examples described above and / or below. The example shown in Fig. 3 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1 - 2) and / or below (e.g., Fig. 4 - 5).

[0073] Fig. 4 shows a flow chart of an example of a method for an optical imaging system. The method 400 may be performed by an apparatus as described with reference to Fig. 1. The method 400 comprises receiving 410 a first image of a sample based on a first frequency range of visible light and receiving 420 a second image of the sample based on a second frequency range of infrared light. A DOF of the second image is greater than a DOF of the first image. Further, the method 400 comprises increasing 430 a depth of field of the first image based on the depth of field of the second image.

[0074] More details and aspects are mentioned in connection with the examples described above and / or below. The example shown in Fig. 4 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1 - 3) and / or below (e.g., Fig. 5).

[0075] Some embodiments relate to a microscope comprising an apparatus as described in connection with one or more of the Fig. 1. Alternatively, a microscope may comprise of or can be communicatively connected to an apparatus as described in connection with one or more of the Fig. 1. Fig. 5 shows a schematic illustration of a system 500, e.g., an optical imaging system, configured to perform a method described herein, e.g., with reference to Fig. 4. The system 500 comprises a microscope 510 and a computer system 520. The microscope may comprise the apparatus as described above, e.g., with reference to Fig. 1 and / or the infrared filter as described above, .e.g., with reference to Fig. 2. The microscope 510 is configured to take images and is connected to the computer system 520. The computer system 520 is configured to execute at least a part of a method described herein. The computer system 520 may be configured to execute a machine learning algorithm. The computer system 520 and microscope 510 may be separate entities but can also be integrated together in one common housing. The computer system 520 may be part of a central processing system of the microscope 510 and / or the computer system 520 may be part of a subcomponent of the microscope 510, such as a sensor, an actor, a camera or an illumination unit, etc. of the microscope 510.

[0076] The computer system 520 may be a local computer device (e.g., personal computer, laptop, tablet computer or mobile phone) with one or more processors and one or more storage devices or may be a distributed computer system (e.g., a cloud computing system with one or more processors and one or more storage devices distributed at various locations, for example, at a local client and / or one or more remote server farms and / or data centers). The computer system 520 may comprise any circuit or combination of circuits. In one embodiment, the computer system 520 may include one or more processors which can be of any type. As used herein, processor may mean any type of computational circuit, such as but not limited to a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), multiple core processor, a field programmable gate array (FPGA), for example, of a microscope or a microscope component (e.g., camera) or any other type of processor or processing circuit. Other types of circuits that may be included in the computer system 520 may be a custom circuit, an application-specific integrated circuit (ASIC), or the like, such as, for example, one or more circuits (such as a communication circuit) for use in wireless devices like mobile telephones, tablet computers, laptop computers, two-way radios, and similar electronic systems. The computer system 520 may include one or more storage devices, which may include one or more memory elements suitable to the particular application, such as a main memory in the form of random access memory (RAM), one or more hard drives, and / or one or more drives that handle removable media such as compact disks (CD), flash memory cards, digital video disk (DVD), and the like. The computer system 520 may also include a display device, one or more speakers, and a keyboard and / or controller, which can include a mouse, trackball, touch screen, voice-recognition device, or any other device that permits a system user to input information into and receive information from the computer system 520.

[0077] More details and aspects are mentioned in connection with the examples described above. The example shown in Fig. 5 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1 - 4).

[0078] Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a processor, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.

[0079] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a non- transitory storage medium such as a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

[0080] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0081] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.

[0082] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0083] In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0084] A further embodiment of the present invention is, therefore, a storage medium (or a data carrier, or a computer-readable medium) comprising, stored thereon, the computer program for performing one of the methods described herein when it is performed by a processor. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitionary. A further embodiment of the present invention is an apparatus as described herein comprising a processor and the storage medium.

[0085] A further embodiment of the invention is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.

[0086] A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein. A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0087] A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0088] In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0089] If some aspects have been described in relation to a device or system, these aspects should also be understood as a description of the corresponding method and vice versa. For example, a block, device or functional aspect of the device or system may correspond to a feature, such as a method step, of the corresponding method. Accordingly, aspects described in relation to a method shall also be understood as a description of a corresponding block, a corresponding element, a property or a functional feature of a corresponding device or a corresponding system.

[0090] The following claims are hereby incorporated in the detailed description, wherein each claim may stand on its own as a separate example. It should also be noted that although in the claims a dependent claim refers to a particular combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly proposed, unless it is stated in the individual case that a particular combination is not intended. Furthermore, features of a claim should also be included for any other independent claim, even if that claim is not directly defined as dependent on that other independent claim.

[0091] The aspects and features described in relation to a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the features into the further example.

[0092] List of reference Signs

[0093] 100 optical imaging system

[0094] 116 first optical component

[0095] 118 second optical component

[0096] 120 microscope

[0097] 122 optical imaging sensor

[0098] 126 infrared filter

[0099] 128 opening

[0100] 130 apparatus

[0101] 132 interface

[0102] 134 processor

[0103] 136 storage device

[0104] 140 optical path

[0105] 142 optical axis

[0106] 226 infrared filter

[0107] 250 first region

[0108] 252 second region

[0109] 260 sensor response blue light

[0110] 262 sensor response green light

[0111] 264 sensor response red light

[0112] 266 sensor response infrared light

[0113] 270 transmission intensity first region

[0114] 272 transmission intensity second region

[0115] 400 method for an optical imaging system

[0116] 410 receiving a first image of a sample

[0117] 420 receiving a second image of the sample

[0118] 430 increasing a depth of field of the first image

[0119] 500 system

[0120] 510 microscope

[0121] 520 computer system

Claims

Claims1. An optical imaging system (100), comprising: a microscope (104) comprising: at most one optical imaging sensor (122) configured to acquire a first image of a sample (110) based on a first frequency range of visible light and a second image of the sample (110) based on a second frequency range of infrared light; a first optical element (116) arranged along an optical path (120) of the microscope (104) for collimating a beam of light; a second optical element (118) arranged along the optical path (120) of the microscope (104) defining a first aperture of the microscope (104) for the first frequency range of visible light; and an infrared filter (126) arranged along the optical path (120) of the microscope (104) defining a second aperture of the microscope (104) for the second frequency range of infrared light, wherein the first aperture is different from the second aperture.

2. The optical imaging system (100) according to claim 1, wherein the first aperture is greater than the second aperture.

3. The optical imaging system (100) according to claim 2, wherein the first aperture is at least 200% greater than the second aperture.

4. The optical imaging system (100) according any one of the preceding claims, wherein a depth of field of the first image is lower than a depth of field of the second image.

5. The optical imaging system (100) according any one of the preceding claims, wherein the at most one optical imaging sensor (122) is an RGB-infrared optical imaging sensor.

6. The optical imaging system (100) according to any one of the preceding claims, wherein the infrared filter (126) has an opening (128) arranged along an optical axis (140) of the optical path (120).

7. The optical imaging system (100) according to claim 6, wherein the infrared filter (126) is transparent to the first frequency range.

8. The optical imaging system (100) according to any one of the preceding claims, wherein the optical imaging sensor (122) is configured to receive visible light from the sample (110) indicative of the first image and infrared light from the sample (110) indicative of the second image at the same time.

9. The optical imaging system (100) according to any one of the preceding claims, further comprising an apparatus (130) configured to: receive sensor data from at most one optical imaging sensor (122), the sensor data indicative of a first image of a sample (110) based on a first frequency range of visible light; and a second image of the sample (110) based on a second frequency range of infrared light, wherein a depth of field of the second image is greater than a depth of field of the first image; and increase a depth of field of the first image based on the depth of field of the second image.

10. An apparatus (130) for an optical imaging system (100), comprising one or more processors (134) and one or more storage devices (136), wherein the apparatus (130) is configured to: receive sensor data from at most one optical imaging sensor (122), the sensor data indicative of: a first image of a sample (110) based on a first frequency range of visible light; anda second image of the sample (110) based on a second frequency range of infrared light, wherein a depth of field of the second image is greater than a depth of field of the first image; and increase a depth of field of the first image based on the depth of field of the second image.

11. A method (400) for an optical imaging system, comprising: receiving (410) a first image of a sample based on a first frequency range of visible light; receiving (420) a second image of the sample based on a second frequency range of infrared light, wherein a depth of field of the second image is greater than the depth of field of the first image; and increasing (430) a depth of field of the first image based on the depth of field of the second image.

12. A computer program with a program code for performing the method according to claim 11 when the computer program is executed on a processor.

Images