Microscope system

Abstract

A microscope system avoids a focus shift of an imaging device in an AR microscope. The microscope system includes: an image sensor; an eyepiece-side observation optical system that forms an image of a sample on an object side of an eyepiece using light from the sample; a camera-side observation optical system that forms an image of the sample on the image sensor using light from the sample; an aperture that limits at least an exit side numerical aperture of the camera-side observation optical system; an analysis section that analyzes the image of the sample acquired by the image sensor; and an overlay optical system that overlays the analysis result of the image with the image formed on the object side of the eyepiece. When a numerical aperture on the object side of the eyepiece is NA, when an exit side numerical aperture of the camera-side observation optical system determined by a light beam emitted from the camera-side observation optical system toward the image sensor at the time of image acquisition is NA', and when a total magnification of the camera-side observation optical system is M1, the following conditional expression is satisfied: M1 x NA' < NA … (1).

Description

Technical Field

[0001] The disclosure of this specification relates to microscope systems. Background Technology

[0002] Augmented Reality (AR) microscopes are known to display auxiliary information superimposed on the optical image of a specimen being observed through an eyepiece. The auxiliary information displayed by an AR microscope is typically generated by analyzing images of the specimen taken by a camera mounted on the AR microscope.

[0003] During visual observation, the depth of focus is greater than that of a camera, corresponding to the eye's adjustment ability. Therefore, even if the user of an AR microscope focuses on the sample visually, the focus may not be correct in a camera, resulting in blurry images. Techniques related to this problem are described, for example, in Non-Patent Literature 1 and 2.

[0004] Existing technical documents

[0005] Non-patent literature 1: Po-Hsuan Cameron Chen, Krishna Gadepalli, Robert MacDonald, Yun Liu, Shiro Kadowaki, Kunal Nagpal, Timo Kohlberger, Jeffrey Dean, Greg S. Corrado, Jason D. Hipp, Craig H. Mermel & Martin C. Stumpe, An augmented realitymicroscope with real-time artificial intelligence integration for cancerdiagnosis, Nature Medicine, 25, p.1453-1457(2019)

[0006] Non-patent document 2: Augmented Reality Microscope, [online], [searched June 16, 2022], Internet<URL:https: / / www.augmentiqs.com / augmented-reality-microscope / > Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] In the AR microscopes described in Non-Patent Documents 1 and 2, the captured images obtained by taking pictures with a camera are displayed on a monitor, so that the user can confirm whether the focus on the camera side is in place through the monitor.

[0009] However, AR microscopes were originally designed to be highly efficient by reducing the need for the user's gaze to move back and forth between the eyepiece and the monitor. Therefore, as described in Non-Patent Documents 1 and 2, AR microscopes do not require the user to constantly check the captured image on the monitor.

[0010] In view of the above-mentioned circumstances, one aspect of the present invention is to provide a technique for avoiding focus shift of the camera device in an AR microscope.

[0011] Methods for solving problems

[0012] A microscope system according to one aspect of the present invention includes: an image sensor; an eyepiece-side observation optical system including an objective lens and an eyepiece, wherein, for a sample observed through the eyepiece, an image of the sample is formed on the object side of the eyepiece using light incident from the sample via the objective lens; a camera-side observation optical system sharing the objective lens with the eyepiece-side observation optical system, wherein an image of the sample is formed on the image sensor using light incident from the sample via the objective lens; an aperture that at least limits the numerical aperture of the exit side of the camera-side observation optical system; a resolving unit that resolves the image of the sample acquired by the image sensor; and an overlay optical system that overlays the result obtained by the resolving unit of the image with the image formed on the object side of the eyepiece. Let NA be the numerical aperture on the object side of the objective lens, defined as the specification of the objective lens; let NA′ be the numerical aperture on the exit side of the camera-side observation optics system, determined by the light beam emitted from the camera-side observation optics system toward the image sensor when the image is acquired; and let M1 be the total magnification of the camera-side observation optics system. The microscope system satisfies the following conditional equation.

[0013] M1×NA′ <NA…(1)

[0014] Invention Effects

[0015] By following the above method, focus shift of the camera device in the AR microscope can be avoided. Attached Figure Description

[0016] Figure 1 This is a diagram showing the structure of a microscope system according to one embodiment.

[0017] Figure 2It is a diagram used to illustrate the relationship between the light beams used in visual observation and the light beams used in camera photography.

[0018] Figure 3 This is a diagram showing the structure of the microscope system according to the first embodiment.

[0019] Figure 4 This is a diagram showing the structure of the microscope system according to the second embodiment.

[0020] Figure 5 This is a diagram showing the structure of the microscope system according to the third embodiment.

[0021] Figure 6 This is a diagram showing the structure related to aperture control in the microscope system of the fourth embodiment.

[0022] Figure 7 This is a diagram showing the structure of the microscope system according to the fifth embodiment.

[0023] Figure 8 This is a diagram showing the structure of the microscope system according to the sixth embodiment.

[0024] Figure 9 This is a diagram illustrating the hardware structure of a computer used to implement the control device.

[0025] Label Explanation

[0026] 1, 100, 200, 300, 400, 500, 600: Microscope systems;

[0027] 13. 501: Light source;

[0028] 14: Converter;

[0029] 15, 101, 201, 301, 505, 601: Objective lenses;

[0030] 30: Eyepiece;

[0031] 40, 110, 214, 312, 510, 607: Camera devices;

[0032] 41, 110s, 214s, 312s, 510s, 607s: Image sensor;

[0033] 50, 111, 215, 313, 511, 608: Projector;

[0034] 60, 150, 250, 350, 450, 550, 650: Control device;

[0035] 102, 203, 206, 306, 506, 603: Beam splitters;

[0036] 103, 202, 302, 507, 604: Imaging lenses;

[0037] 104, 208, 308, 508, 605: Prisms;

[0038] 106, 109, 204, 207, 210, 213, 304, 307, 311: relay lenses; 108, 212, 310, 410, 503, 602: apertures;

[0039] 151, 251, 351, 451, 551, 651: Analysis section;

[0040] 452, 552, 652: Aperture control unit;

[0041] 460, 560, 660: Drive unit;

[0042] 502: Collimating lens;

[0043] 504: Condensing lens;

[0044] 1000: Computer;

[0045] 1001: Processor;

[0046] 1002: Memory;

[0047] 1003: Storage device;

[0048] S: Sample;

[0049] L1, L2: Beam;

[0050] P1, P2: Image positions. Detailed Implementation

[0051] Figure 1 This is a diagram illustrating the structure of a microscope system 1 according to one embodiment. Microscope system 1 is a microscope system with AR display functionality. Microscope system 1 can be, for example, a biological microscope system or an industrial microscope system.

[0052] like Figure 1 As shown, the microscope system 1 includes: a microscope corresponding to visual observation and photographic photography, including an objective lens 15, an eyepiece 30, and an imaging device 40; and a control device 60 that controls the microscope. Regarding the microscope system 1, the control device 60 controls a projector 50 disposed on the microscope, thereby realizing an AR display function, which is the function of displaying auxiliary information superimposed on the optical image of the sample S observed through the eyepiece 30.

[0053] The microscope is equipped with an eyepiece-side observation optical system, which uses light from the sample S incident via the objective lens 15 on the object side of the eyepiece 30 to observe the sample S through the eyepiece 30. Figure 1 The image of the sample S is formed at image position P2; and the camera-side observation optical system utilizes the light from the sample S incident via the objective lens 15 onto the image sensor 41 of the imaging device 40 ( Figure 1 The image of the sample S is formed at position P1. In addition, the eyepiece-side observation optical system includes at least an objective lens 15 and an eyepiece 30, and the camera-side observation optical system shares the objective lens 15 with the eyepiece-side observation optical system.

[0054] The control device 60 is connected to the imaging device 40 and operates as a resolution unit that resolves the image of the sample S acquired by the image sensor 41. The resolution process performed by the control device 60 is not particularly limited; for example, it may include processing to detect regions of interest (e.g., cell nuclei) in the sample. The result obtained by the control device 60 in resolving the image of the sample S (resolution result) is output to a projector 50 installed on the microscope for AR display. Furthermore, the resolution result may include, for example, information identifying the region of interest.

[0055] The projector 50 is not particularly limited; it can be a liquid crystal projector that uses liquid crystal devices, an organic EL projector that uses organic EL devices, or a DMD (registered trademark) projector.

[0056] The microscope also includes an overlay optical system that overlays the result resolved by the control device 60 onto the image of the sample S formed on the object side of the eyepiece 30 (image position P2). AR display is achieved by projecting the resolved result onto the image position P2 through the overlay optical system, thereby allowing the user of the microscope system 1 to simultaneously observe the image of the sample S and the resolved result through the eyepiece 30.

[0057] In addition, Figure 1 In this context, control device 60 is represented as a single device, but it can also be a collection of two or more devices. For example, control device 60 can consist of a microscope control box and a general-purpose computer.

[0058] The structure of the microscope will be explained in more detail below. For example... Figure 1 As shown, the microscope includes: a microscope body 10, which includes an objective lens 15; a microscope tube assembly 20, which includes a projector 50; an eyepiece 30; and an imaging device 40, which includes an image sensor 41. Figure 1 An upright microscope is shown in the example, but the microscope included in microscope system 1 may also be an inverted microscope.

[0059] The microscope body 10 includes: a stage 11 for holding the sample S, a lamp cover 12 for housing the light source 13, and a converter 14 for mounting the objective lens 15. A microscope tube assembly 20 is mounted on the microscope body 10. Additionally, an eyepiece 30 and a camera assembly 40 are mounted on the microscope tube assembly 20.

[0060] The stage 11 is connected to the focusing handle via a rack and pinion mechanism. The rotational motion of the focusing handle is converted into linear motion of the stage 11 via the rack and pinion mechanism. Thus, the stage 11 moves up and down by rotating the focusing handle. The stage 11 is an example of a focusing mechanism that adjusts the position of the focal plane of the objective lens 15 relative to the sample S along the optical axis of the objective lens 15.

[0061] Additionally, the stage 11 includes an XY stage that moves in a direction orthogonal to the optical axis of the objective lens 15. The stage 11 moves in a direction orthogonal to the optical axis of the objective lens 15 by operating a handle (not shown). Furthermore, an opening of a size sufficient to allow illumination light from the capacitor to pass through, and a size that prevents the sample S from falling off, is formed in the stage 11.

[0062] The lampshade 12 includes a light source 13 that emits illumination light to irradiate the sample S. The light source 13 is lit by turning on a power switch (not shown) and turned off by turning off the power switch.

[0063] The light source 13 is not particularly limited, and can be, for example, a halogen lamp. The illumination light emitted from the light source 13 is directed onto the sample S disposed on the stage 11 via a capacitor provided in the microscope body 10. The amount of illumination light emitted from the stage 11 can be adjusted by a dial provided in the microscope body 10.

[0064] The converter 14 can accommodate multiple objectives, preferably multiple objectives with different magnifications. By rotating the converter 14, the objectives positioned in the optical path of the microscope are switched. The converter 14 is an example of an objective switching mechanism.

[0065] The lens barrel assembly 20 is a three-lens lens barrel capable of assembling the eyepiece 30 and the imaging device 40. Light from the sample S, which has been illuminated, enters the lens barrel assembly 20 via the objective lens 15. The light entering the lens barrel assembly 20 is split within the lens barrel assembly 20 into light reaching the eyepiece 30 and light reaching the imaging device 40.

[0066] The telescope tube assembly 20 includes an eyepiece sleeve 21 for mounting an eyepiece 30. The eyepiece sleeve 21 is disposed in the rotating part of the telescope tube assembly 20, which is capable of rotating about an axial direction in the horizontal direction. The eyepiece 30, mounted on the eyepiece sleeve 21, rotates together with the rotating part, thereby adjusting the height of the eyepoint.

[0067] Eyepiece 30 is mounted on eyepiece sleeve 21 provided in the lens barrel assembly 20. Imaging device 40 is, for example, a digital camera equipped with image sensor 41. Imaging device 40 photographs the sample S. Image sensor 41 is CCD, CMOS, etc. Imaging device 40, by being mounted in lens barrel assembly 20, is positioned on the imaging optical path branching from the visual observation optical path from objective lens 15 to eyepiece 30.

[0068] A projector 50 is disposed within the lens tube assembly 20. The projector 50 is an example of an overlay device for overlaying and resolving the image plane (image position P2) of the image of the sample S. The projector 50 may be a liquid crystal projector using a liquid crystal device, an organic EL projector using an organic EL device, or a DMD (registered trademark) projector.

[0069] Figure 2 This diagram illustrates the relationship between the light beam used in visual observation via eyepiece 30 and the light beam used in photographic capture by the camera in imaging device 40. As described above, generally, when focusing an AR microscope while looking through the eyepiece, the captured image obtained by the imaging device is sometimes blurry. This blurring in the captured image can hinder image resolution in the control device, which is undesirable. To solve this technical problem, in microscope system 1, an aperture is provided inside the microscope that at least limits the numerical aperture of the exit side of the camera-side observation optics system, such as... Figure 2 As shown, the light beam is limited to use in camera photography.

[0070] also, Figure 2 The beam L shown corresponds to the numerical aperture NA (=n·sinθ) on the object side, defined as the specification of objective lens 15, and is the beam used for observation of the sample via eyepiece 30. On the other hand, beam L1 represents the beam used for photographic imaging. Beam L1 is a portion of beam L and corresponds to a numerical aperture NA1 (=n·sinθ1) smaller than the aforementioned numerical aperture NA (=n·sinθ). n is the refractive index of the medium (e.g., air) between objective lens 15 and sample S.

[0071] like Figure 2 As shown, compared to the case without an aperture, by using an aperture to limit the numerical aperture NA′ of the camera-side observation optical system when the image sensor 41 acquires an image, the depth of focus of the camera-side observation optical system is increased. That is, the microscope system 1 satisfies the following conditional expression.

[0072] M1×NA′<NA…(1)

[0073] Here, NA is the numerical aperture on the object side of the objective lens 15, defined by the specifications of the objective lens 15. NA′ is the numerical aperture on the exit side of the camera-side observation optical system, determined by the beam emitted from the camera-side observation optical system toward the image sensor 41 when the image is acquired. M1 is the total magnification of the camera-side observation optical system, that is, the magnification of the image at image position P1. Furthermore, NA1 is calculated from the product of NA′ and M1.

[0074] By satisfying condition (1), it is possible to ensure the same depth of focus as the eyepiece-side observation optical system, which corresponds to the depth of focus magnification and the adjustment capability of the human eye, even in the camera-side observation optical system. Therefore, in the microscope system 1, blurry images can be avoided by focusing while looking at the eyepiece 30.

[0075] The microscope system 1 preferably satisfies the following condition.

[0076]

[0077] Here, λ is the center wavelength of the light from the sample S. Furthermore, the unit of λ is μm. M2 is the total magnification of the eyepiece-side observation optical system, that is, the magnification obtained by multiplying the image magnification at image position P2 by the magnification of the eyepiece 30.

[0078] Furthermore, the lower limit of condition (2) is equivalent to the value obtained by converting the depth of focus in visual observation derived from BEREK's formula to the object-side depth (i.e., depth of field) using the total magnification. The upper limit of condition (2) is equivalent to four times the value obtained by converting the depth of focus in visual observation derived from BEREK's formula to the object-side depth (i.e., depth of field) using the total magnification. That is, condition (2) means that by using the aperture, the depth of focus of the camera-side observation optics should be adjusted from the same level as the depth of focus in visual observation derived from BEREK's formula to within four times its original value. Furthermore, the comparison on the object side is to eliminate the influence of the difference in total magnification between the camera-side and eyepiece-side observation optics.

[0079] By setting the focal depth of the camera-side observation optical system to at least the same level as the focal depth in visual observation, blurring of the captured image can be avoided by focusing while looking through the eyepiece 30. Furthermore, by suppressing the focal depth of the camera-side observation optical system to within four times the focal depth in visual observation, extreme degradation of the captured image resolution can be suppressed. Therefore, by satisfying condition (2), blurring and resolution degradation of the captured image can be avoided in the microscope system 1, and a captured image suitable for resolution can be obtained.

[0080] Hereinafter, in each embodiment, Figure 1 A specific example of the microscope system 1 shown will be described. Furthermore, the first to fourth embodiments are examples where the aperture is configured only in the optical path of the camera-side observation optical system. The fifth embodiment is an example where the aperture is configured in the optical path of the illumination optical system. The sixth embodiment is an example where the aperture is configured in the optical path shared by the eyepiece-side observation optical system and the camera-side observation optical system.

[0081] (First Implementation)

[0082] Figure 3 This diagram illustrates the structure of the microscope system 100 according to this embodiment. The microscope system 100, like the microscope system 1 described above, includes: an image sensor 110s disposed on the imaging device 110; an eyepiece-side observation optical system including an objective lens 101 and an eyepiece (not shown); a camera-side observation optical system sharing the objective lens 101 with the eyepiece-side observation optical system; an aperture 108; a control device 150 including a resolution unit 151; an overlay optical system; and a projector 111.

[0083] The eyepiece-side observation optical system consists of objective lens 101 to prism 104 and eyepiece, forming an image of the sample at image position P2. The camera-side observation optical system consists of objective lens 101 to relay lens 109, forming an image of sample S at image position P1. The overlay optical system consists of beam splitter 102 to prism 104, lens 112 and polarizer 113, overlaying the result obtained by resolving the image of sample S by resolving unit 151 onto image position P2. More specifically, the resolution result displayed on projector 111 is projected onto image position P2.

[0084] like Figure 3 As shown, the microscope system 100 includes a prism 104 as an optical path branching element, which branches the light from the sample S into a path towards the eyepiece (not shown) and a path towards the image sensor 110s. An aperture 108 is disposed on the optical path between this optical path branching element (prism 104) and the image sensor 110s. Thus, the aperture 108 can limit only the numerical aperture of the emission side of the camera-side observation optical system. Therefore, the sample S can be brightly observed through visual observation via the eyepiece without limiting the numerical aperture of the emission side of the eyepiece-side observation optical system.

[0085] Furthermore, the camera-side optical system of the microscope system 100 includes a relay optical system that relays the primary image of the sample S formed by light from the sample S. The relay optical system, composed of relay lenses 106 and 109, relays the primary image formed by the objective lens 101 and the imaging lens 103 to image position P1, thereby forming a secondary image of the sample S at image position P1. The pupil of the objective lens 101 is projected to a position optically conjugate to the pupil of the objective lens 101 located in the optical path of the relay optical system; therefore, the aperture 108 is positioned at or near the position optically conjugate to the pupil of the objective lens 101. Thus, the aperture 108 functions as an aperture diaphragm, thereby efficiently limiting the numerical aperture of the exit side of the camera-side observation optical system. Furthermore, in the microscope system 100, the optical path is appropriately bent using a mirror 107, thereby enabling a compact optical path that extends the optical path length through the relay optical system, thus also suppressing the enlargement of the device.

[0086] Furthermore, the aperture diameter (aperture diameter) of aperture 108 is designed to satisfy conditions (1) and (2) in the microscope system 100. Specifically, the aperture diameter of aperture 108 is determined by pre-assuming the range of magnification of the objective lenses used in the microscope system 100 and the range of object-side numerical aperture, in a manner that satisfies conditions (1) and (2) within these ranges. This avoids blurring of the captured image and degradation of resolution. Aperture 108 is, for example, an aperture with a fixed aperture diameter determined by the above method.

[0087] Furthermore, in the microscope system 100, light from the projector 111 converges with the optical path of the observation optical system at the object-side position of the prism 104 via the beam splitter 102. However, only linearly polarized light in a specific direction converges with the optical path of the observation optical system via the polarizer 113. Therefore, the light from the projector 111 can be blocked by the analyzer 105 disposed on the optical path of the observation optical system on the camera side. As a result, light emitted from the projector 111 can be prevented from incident on the image sensor 110s, thus preventing the resolution result from being reflected in the captured image. Moreover, the reflection of the resolution result in the captured image may adversely affect the resolution accuracy in the resolution unit 151, and such reflection is undesirable.

[0088] The microscope system 100 configured as described above, like the microscope system 1, can avoid focus deviation of the imaging device 110 by focusing while looking at the eyepiece.

[0089] (Second Implementation)

[0090] Figure 4This is a diagram showing the structure of the microscope system 200 of this embodiment. The difference between the microscope system 200 and the microscope system 100 of the first embodiment is that, on the image side of the beam splitter 203, which serves as an optical path branching element, the light from the projector 215 merges with the optical path of the eyepiece-side observation optical system.

[0091] In addition, like the microscope system 1 and microscope system 100 described above, microscope system 200 includes: an image sensor 214s disposed on the imaging device 214; an eyepiece-side observation optical system including an objective lens 201 and an eyepiece (not shown); a camera-side observation optical system sharing the objective lens 201 with the eyepiece-side observation optical system; an aperture 212; a control device 250 including a resolution unit 251; an overlay optical system; and a projector 215.

[0092] The eyepiece-side observation optical system, consisting of objective lens 201, mirror 209, and eyepiece, forms an image of the sample at image position P2. The camera-side observation optical system, consisting of objective lens 201, beam splitter 203, and relay lenses 210 and 213, forms an image of sample S at image position P1. The overlay optical system, consisting of lens 216, beam splitter 206, and mirror 209, overlays the result obtained by resolving the image of sample S by the resolving unit 251 onto image position P2. More specifically, the resolving result displayed on projector 215 is projected onto image position P2.

[0093] In microscope system 200, such as Figure 4 As shown, aperture 212 is positioned in the optical path between beam splitter 203 (which serves as an optical path branching element) and image sensor 214s, just like in microscope system 100. Furthermore, the camera-side observation optical system of microscope system 200 includes a relay optical system (relay lens 210, relay lens 213) that relays the primary image formed by objective lens 201 and imaging lens 202 to image position P1 to form a secondary image. Aperture 212 is positioned at or near the position optically conjugate to the pupil of objective lens 201, again just like in microscope system 100. Additionally, the device is compactly constructed by bending the optical path using mirror 211, again just like in microscope system 100. Moreover, the aperture diameter (aperture diameter) of aperture 212 is designed to satisfy conditions (1) and (2), again just like in microscope system 100.

[0094] In the microscope system 200, the eyepiece-side observation optical system, like the camera-side observation optical system, also has a relay optical system. Specifically, the eyepiece-side observation optical system has a relay optical system consisting of a relay lens 204 and a relay lens 207, which relays the primary image formed by the objective lens 201 and the imaging lens 202 to the image position P2 to form a secondary image. A beam splitter 206, which merges the light from the projector 215 with the optical path of the eyepiece-side observation optical system, is positioned on the optical path of the parallel beam traveled between the relay lens 204 and the relay lens 207.

[0095] Furthermore, in the microscope system 200, the light path can also be appropriately bent using a mirror 205 and a prism 208, thereby compactly constructing an optical path that extends the length of the light path through a relay optical system and lowers the eyepoint.

[0096] In this way, by setting up a relay optical system in the observation optical system on the eyepiece side, the light from the projector 215 is converged in the optical path of the relay optical system, thus preventing the light emitted from the projector 215 from incident on the image sensor 214s. Therefore, similar to the microscope system 100, the microscope system 200 can also prevent the resolution result from being reflected in the captured image.

[0097] As described above, the microscope system 200, like microscope system 1 and microscope system 200, can avoid focus shift of the imaging device 214 by focusing while looking at the eyepiece.

[0098] (Third Implementation)

[0099] Figure 5 This is a diagram showing the structure of the microscope system 300 of this embodiment. The microscope system 300 differs from the microscope system 200 of the second embodiment in that the beam splitter 306, which is a beam splitting element, also serves as a beam merging element that merges the light from the projector 313 with the light path toward the eyepiece, wherein the beam splitting element branches the light path from the sample S into a light path toward the eyepiece and a light path toward the image sensor 312s.

[0100] Furthermore, in microscope system 300, the eyepiece-side observation optical system and the camera-side observation optical system each have relay optical systems (relay lenses 304 and 307, and relay lenses 304 and 311) that relay the single image formed by objective lens 301 and imaging lens 302, just as in microscope system 200. However, unlike microscope system 200, the camera-side observation optical system and the eyepiece-side observation optical system share a portion of the lens (relay lens 304) of the relay optical system.

[0101] In addition, like the microscope system 200, the microscope system 300 includes: an image sensor 312s disposed on the camera device 312; an eyepiece-side observation optical system including an objective lens 301 and an eyepiece (not shown); a camera-side observation optical system sharing the objective lens 301 with the eyepiece-side observation optical system; an aperture 310; a control device 350 including a resolution unit 351; an overlay optical system; and a projector 313.

[0102] The eyepiece-side observation optical system consists of an objective lens 301, a reflecting mirror 309, and an eyepiece, forming an image of the sample at image position P2. The camera-side observation optical system consists of an objective lens 301, a beam splitter 306, an aperture 310, and a relay lens 311, forming an image of the sample S at image position P1. The overlay optical system consists of a lens 314, a beam splitter 306, and a reflecting mirror 309, overlaying the result obtained by resolving the image of the sample S by the resolving unit 351 onto image position P2. More specifically, the resolving result displayed on the projector 315 is projected onto image position P2.

[0103] like Figure 5 As shown, aperture 310 is positioned on the optical path between beam splitter 306 (which serves as an optical path branching element) and image sensor 312s, which is the same for microscope system 300 as for microscope system 200. Furthermore, aperture 310 is positioned at or near the position optically conjugate to the pupil of objective lens 301, which is also the same for microscope system 300 as for microscope system 200. Additionally, by using mirrors and prisms (mirror 303, mirror 305, beam splitter 306, prism 308, mirror 309) to bend the optical path, the device is compactly constructed as a whole, and the eyepoint is lowered, which is also the same for microscope system 300 as for microscope system 200. Moreover, the aperture diameter (aperture diameter) of aperture 310 is designed to satisfy conditions (1) and (2), which is also the same for microscope system 200.

[0104] The microscope system 300 configured as described above, like the microscope systems of the other embodiments described above, can avoid focus shift of the imaging device 312 by focusing while looking through the eyepiece.

[0105] (Fourth Implementation)

[0106] Figure 6 This diagram illustrates the structure associated with aperture control in the microscope system 400 of this embodiment. Furthermore, the microscope system 400 differs from the microscope system 300 in that it includes a control device 450 instead of a control device 350; it includes an aperture 410 instead of an aperture 310; and it includes a drive unit 460 for driving the aperture 410.

[0107] The control device 450 includes a resolution unit 451 and an aperture control unit 452. The resolution unit 451 is the same as the resolution unit 351 of the microscope system 300. The aperture control unit 452 controls the aperture 410. Specifically, the aperture control unit 452 changes the opening diameter of the aperture 410 via the drive unit 460. As shown in conditional expression (2), the depth of focus that the camera-side observation optical system should achieve using the aperture 410 depends on the magnification and numerical aperture of the objective lens. Therefore, it is preferable to adjust the opening diameter of the aperture 410 when changing the objective lens. Therefore, it is preferable that the aperture control unit 452 changes the opening diameter of the aperture 410 synchronously with the objective lens switching operation. For example, the control device 450 can also detect the change of the objective lens arranged in the optical path by detecting the position of the hole of the converter arranged in the optical path. By pre-registering the relationship between the hole position of the converter and the objective lens, the aperture control unit 452 can change the opening diameter of the aperture 410 to an appropriate size corresponding to the objective lens.

[0108] Furthermore, the aperture 410 can also be a variable aperture, like a leaf aperture, where the opening diameter is mechanically changed. Alternatively, the aperture 410 can be, for example, a liquid crystal device composed of multiple pixels, and the opening diameter can be changed by electrically controlling the transmittance of the aperture 410. Moreover, multiple apertures 410 with different opening diameters can be provided in the microscope system 400, and multiple apertures 410 can be switched in conjunction with the objective lens.

[0109] The microscope system 400 configured as described above, like the microscope systems of the other embodiments described above, can avoid focus shift of the imaging device 312 by focusing while looking through the eyepiece. Furthermore, according to the microscope system 400, the depth of focus of the camera-side observation optical system can be optimized in relation to the objective lens, thus achieving a high level of balance between focusing and image resolution.

[0110] (Fifth Implementation)

[0111] Figure 7 This diagram illustrates the structure of the microscope system 500 according to this embodiment. Similar to the microscope system 1 described above, the microscope system 500 includes: an image sensor 510s disposed on the imaging device 510; an eyepiece-side observation optical system including an objective lens 505 and an eyepiece (not shown); a camera-side observation optical system sharing the objective lens 505 with the eyepiece-side observation optical system; an aperture 503; a control device 550 including a resolution unit 551; an overlay optical system; and a projector 511.

[0112] The eyepiece-side observation optical system consists of objective lens 505, prism 508, and eyepiece, forming an image of the sample at image position P2. The camera-side observation optical system consists of objective lens 505 and analyzer 509, forming an image of sample S at image position P1. The overlay optical system consists of beam splitter 506, prism 508, lens 512, and polarizer 513, overlaying the result obtained by resolving the image of sample S by resolving unit 551 onto image position P2. More specifically, the resolution result displayed on projector 511 is projected onto image position P2.

[0113] like Figure 7 As shown, the microscope system 500 includes a prism 508 as an optical path branching element, which branches the light from the sample S into a path towards the eyepiece (not shown) and a path towards the image sensor 510s. An aperture 503 is disposed on the common optical path closer to the object than the optical path branching element (prism 508). Specifically, the microscope system 500 includes an illumination optical system for illuminating the sample S, and the aperture 503 is disposed on the optical path of the illumination optical system. More specifically, the aperture 503 is disposed on or near a position on the optical path of the illumination optical system that is optically conjugate to the pupil of the objective lens 505. Furthermore, the illumination optical system includes a collimating lens 502 that collimates the illumination light emitted from the light source 501; and a condenser lens 504 that focuses the illumination light onto the sample S.

[0114] In the microscope system 500, the aperture 503 is positioned on the object side of the prism 508. Therefore, when the light beam is limited by the aperture 503, not only the numerical aperture of the exit side of the camera-side observation optical system is limited, but also the numerical aperture of the exit side of the eyepiece-side observation optical system is limited simultaneously. Thus, the microscope system 500 substantially eliminates the adverse effects on visual observation caused by limiting the numerical aperture by providing an aperture control unit 552 in the control device 550 to control the aperture 503, and by changing the aperture control unit 552 synchronously with the image sensor 510s's shooting timing.

[0115] Specifically, the aperture control unit 552 matches the exposure period of the image sensor 510s, and the aperture diameter of the aperture 503 is changed via the drive unit 560 in a manner that at least satisfies condition (1). By limiting the exposure period of each exposure to a sufficiently short time that the microscope user can hardly perceive the change in the image during visual observation, it is possible to eliminate adverse effects on visual observation and increase the depth of focus of the camera-side observation optical system.

[0116] Furthermore, the aperture control unit 552 preferably changes the aperture 503's opening diameter via the drive unit 560 in a manner that matches the exposure period of the image sensor 510s and satisfies conditional expressions (1) and (2). Additionally, the aperture control unit 552 preferably changes the aperture 503's opening diameter to a diameter corresponding to the objective lens via the drive unit 560 in a manner that matches the exposure period of the image sensor 510s.

[0117] In the microscope system 500, it is necessary to change the aperture diameter of the aperture 503 at high speed. Therefore, the aperture 503 is preferably, for example, a liquid crystal device composed of multiple pixels, and the aperture diameter is preferably changed by electrically controlling the transmittance of the aperture 503.

[0118] The microscope system 500 configured as described above, like the microscope systems of other embodiments described above, can avoid focus shift of the imaging device 510 by focusing while looking through the eyepiece. Furthermore, according to the microscope system 500, the depth of focus of the camera-side observation optical system can be optimized based on the objective lens, thus achieving a high level of balance between focusing and image resolution. Moreover, in the microscope system 500, no relay optical system is provided in the observation optical system, and a single image formed by the objective lens 505 and the imaging lens 507 is projected at image positions P1 and P2. Therefore, compared to other embodiments, the device can be configured more compactly.

[0119] (Sixth Implementation Method)

[0120] Figure 8 This diagram illustrates the structure of the microscope system 600 according to this embodiment. The microscope system 600 differs from the microscope system 500 of the fifth embodiment in that the aperture 602 is disposed within the objective lens 601. That is, the aperture 602 is positioned at the pupil position of the objective lens 601. Furthermore, the structure of the observation optical system after the aperture 602 is the same as that of the microscope system 500.

[0121] In the microscope system 600, when the aperture 602 is used to limit the light beam, not only the numerical aperture of the exit side of the camera-side observation optical system is limited, but also the numerical aperture of the exit side of the eyepiece-side observation optical system is limited. Therefore, in the microscope system 600, as in the microscope system 500, the aperture control unit 652 provided in the control device 550 changes the opening diameter of the aperture 602 synchronously with the shooting timing of the image sensor 607s, thereby substantially eliminating the adverse effects on visual observation caused by limiting the numerical aperture.

[0122] Specifically, the aperture control unit 652, in accordance with the exposure period of the image sensor 607s, changes the opening diameter of the aperture 602 via the drive unit 660 in a manner that at least satisfies condition (1). Preferably, the aperture opening diameter of the aperture 602 is changed in a manner that satisfies both condition (1) and condition (2). More preferably, the aperture control unit 552, in accordance with the exposure period of the image sensor 607s, changes the opening diameter of the aperture 602 to an opening diameter corresponding to the objective lens. This control of the aperture 602 is the same as that in the microscope system 500.

[0123] The microscope system 600 configured as described above, like the microscope systems of the other embodiments described above, can avoid focus shift of the imaging device 607 by focusing while looking through the eyepiece. Furthermore, it is similar to the microscope system 500 in the following two aspects: the depth of focus of the camera-side observation optical system can be optimized in accordance with the objective lens to achieve a high level of balance between focusing and image resolution; and the device can be compactly configured without the need for a relay optical system.

[0124] The above-described embodiments are specific examples provided for ease of understanding of the invention, and the present invention is not limited to these embodiments. It is possible to include variations derived from the above-described embodiments, as well as alternatives to the above-described embodiments. That is, each embodiment can be modified in terms of its constituent elements without departing from its spirit and scope. Furthermore, new embodiments can be implemented by appropriately combining multiple constituent elements disclosed in one or more embodiments. Additionally, several constituent elements may be deleted from the constituent elements shown in each embodiment, or several constituent elements may be added to the constituent elements shown in the embodiments. Moreover, the processing procedures shown in each embodiment can be performed in different orders as long as they are not contradictory. In other words, the microscope system of the present invention can be modified and altered in various ways without departing from the scope of the claims.

[0125] Figure 9 This is a diagram illustrating the hardware structure of the computer 1000 used to implement the control device described above. Figure 9 The hardware structure shown includes, for example, a processor 1001, a memory 1002, a storage device 1003, a reading device 1004, a communication interface 1006, and an input / output interface 1007. Furthermore, the processor 1001, memory 1002, storage device 1003, reading device 1004, communication interface 1006, and input / output interface 1007 are interconnected, for example, via a bus 1008.

[0126] The processor 1001 can be any circuit, such as a single processor, a multi-processor, or a multi-core processor. The processor 1001 can also function as the aforementioned parsing unit and aperture control unit by reading and executing programs stored in the storage device 1003.

[0127] The memory 1002 is, for example, a semiconductor memory, which may include a RAM area and a ROM area. The storage device 1003 is, for example, a hard disk, flash memory, or other semiconductor memory or external storage device.

[0128] The reading device 1004 accesses the storage medium 1005, for example, according to the instructions of the processor 1001. The storage medium 1005 may be implemented using, for example, a semiconductor device, a medium for inputting and outputting information via magnetic action, or a medium for inputting and outputting information via optical action. Furthermore, the semiconductor device may be, for example, a USB (Universal Serial Bus) memory. Additionally, the medium for inputting and outputting information via magnetic action may be, for example, a magnetic disk. The medium for inputting and outputting information via optical action may be, for example, a CD (CompactDisc)-ROM (Read-Only Optical Disc), a DVD (Digital Versatile Disk), or a Blu-ray Disc (Blu-ray is a registered trademark).

[0129] The communication interface 1006 communicates with other devices, for example, according to the instructions of the processor 1001. The input / output interface 1007 is, for example, an interface between input and output devices. Input devices may be, for example, devices that receive instructions from the user, such as a keyboard, mouse, or touch panel. Output devices may be, for example, display devices such as a monitor, and sound devices such as a speaker.

[0130] The program executed by the processor 1001 is provided to the computer 1000, for example, in the following manner.

[0131] (1) Pre-installed in storage device 1003.

[0132] (2) Provided by storage medium 1005.

[0133] (3) Provided by servers such as program servers.

[0134] In addition, refer to Figure 9The hardware structure of the computer 1000 used to implement the control device described above is exemplary, and the implementation is not limited thereto. For example, a part of the above structure may be deleted, and new structures may be added. In other embodiments, for example, part or all of the functions of the above circuit may be installed as hardware based on FPGA (Field Programmable Gate Array), SoC (System-on-a-Chip), ASIC (Application Specific Integrated Circuit), and PLD (Programmable Logic Device).

Claims

1. A microscope system, characterized in that, The microscope system includes: Image sensor; An eyepiece-side observation optical system includes an objective lens and an eyepiece, wherein for a sample observed through the eyepiece, an image of the sample is formed on the object side of the eyepiece using light from the sample incident through the objective lens. A camera-side observation optical system shares the objective lens with the eyepiece-side observation optical system, and uses light from the sample incident through the objective lens to form an image of the sample on the image sensor; The aperture, which at least limits the numerical aperture of the exit side of the camera-side observation optical system; The analysis unit analyzes the image of the sample acquired by the image sensor; and An overlay optical system that overlays the result obtained by the resolving unit from the image with the image formed on the object side of the eyepiece. Let NA be the numerical aperture on the object side of the objective lens, defined as the specification of the objective lens; let NA′ be the numerical aperture on the exit side of the camera-side observation optics system, determined by the light beam emitted from the camera-side observation optics system toward the image sensor when the image is acquired; let M1 be the total magnification of the camera-side observation optics system; let λ be the center wavelength of the light from the sample; and let M2 be the total magnification of the eyepiece-side observation optics system. The following conditional expression must be satisfied. ； 。 2. The microscope system according to claim 1, characterized in that, The microscope system also includes an optical path branching element that branches the light from the sample into a path toward the eyepiece and a path toward the image sensor. The aperture is positioned on the optical path between the optical path branching element and the image sensor.

3. The microscope system according to claim 2, characterized in that, The camera-side observation optical system also includes a relay optical system that relays the primary image of the sample formed by light from the sample. The aperture is positioned on or near the optical path of the relay optical system at a position optically conjugate to the pupil position of the objective lens.

4. The microscope system according to claim 2 or 3, characterized in that, The microscope system also includes an aperture control unit for controlling the aperture. The aperture control unit changes the aperture opening diameter synchronously with the switching action of the objective lens.

5. The microscope system according to claim 1, characterized in that, The microscope system also features: An optical path branching element that branches the light from the sample into a path toward the eyepiece and a path toward the image sensor; and An aperture control unit that controls the aperture. The aperture is positioned on the common optical path closer to the object than the optical path branching element. The aperture control unit changes the aperture opening diameter synchronously with the image sensor's shooting timing.

6. The microscope system according to claim 5, characterized in that, The microscope system also includes an illumination optical system for illuminating the sample. The aperture is positioned in the optical path of the illumination optics system.

7. The microscope system according to claim 5, characterized in that, The aperture is located inside the objective lens.

8. The microscope system according to any one of claims 5 to 7, characterized in that, The aperture control unit electrically controls the transmittance of the aperture.

9. The microscope system according to claim 1, characterized in that, The analytical unit detects the region of interest in the sample. The result obtained by analyzing the image includes information for identifying the region of interest.

Images