Microscope system, imaging method, and program

Abstract

To enable a clear image of a sample free of partial luminance reduction in a visual field to be captured, without imposing limits to mechanism design and incurring a significant reduction in imaging speed, even when the sample surface is tilted.SOLUTION: A microscope system for acquiring an image of the sample to be observed, comprises: a light emitting device for irradiating illumination light; a microscope which has a confocal scanner which has a scanning unit to scan illumination light on the sample and an objective lens for projecting to the sample an image having been created by illumination light scanned by the confocal scanner; an imaging device for detecting light to be measured that comes from the sample irradiated with illumination light, with the imaging plane arranged to the scanning unit at a confocal position; and a processing device for controlling the operation of the microscope and the imaging device. The processing device transmits a control signal for capturing images by the imaging device to the microscope and the imaging device, while displacing the relative position of the focus of the objective lens to the sample in a direction parallel to the optical axis of illumination light.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This disclosure relates to a microscope system, an imaging method, and a program. [Background technology]

[0002] Patent Document 1 describes an optical image separation device connected to the confocal image extraction port of a Nipow disk type confocal scanner.

[0003] Patent Document 2 describes a confocal scanner that performs optical scanning by rotating a pinhole disk, which has multiple types of pinholes of different diameters arranged therein, and which is capable of selectively passing light only through pinholes of any particular diameter.

[0004] Patent Document 3 describes a drug discovery screening apparatus comprising means for acquiring a fluorescence signal from a sample as a fluorescence image and means for acquiring a fluorescence signal from a sample as a confocal image, and which performs optical path switching to obtain either a fluorescence image or a confocal image.

[0005] Patent Document 4 describes a microscope system that includes a confocal scanner for acquiring slice images of a sample as confocal images, and an actuator for moving the focal position of the microscope's objective lens in the optical axis direction, and performs imaging while moving the focal position of the objective lens in the optical axis direction. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2002-62480 [Patent Document 2] Japanese Patent Publication No. 2011-90145 [Patent Document 3] Japanese Patent Publication No. 2008-64671 [Patent Document 4] Japanese Patent Publication No. 2005-70689 [Overview of the project] [Problems that the invention aims to solve]

[0007] Well plates are commonly used as observation containers when imaging cells under a microscope. The frame and bottom surface of the well plate may bend due to distortion during molding (the distortion during molding of the frame is about several hundred μm, and the distortion during molding of the bottom surface is about several tens of μm), and as a result, the sample surface may be tilted relative to the imaging plane of the microscope.

[0008] Therefore, when confocal imaging is performed using the configuration of Patent Document 1 with the sample surface tilted relative to the imaging plane, the shallow depth of field causes a significant decrease in brightness in a part of the field of view. While imaging using the configurations of Patent Documents 2 to 4 can prevent this decrease in brightness in a part of the field of view, there were issues such as limitations in mechanical design, an increase in the required image storage capacity, and an increase in imaging time.

[0009] The purpose of this disclosure is to provide a microscope system, imaging method, and program that can capture clear images of a sample without partial brightness reduction within the field of view, even when the sample surface is tilted, without mechanical design limitations or significant reductions in imaging speed. [Means for solving the problem]

[0010] Some embodiments of the microscope system are microscope systems for acquiring an image of a sample to be observed, comprising: a microscope having a light-emitting device for irradiating illumination light; a confocal scanner having a scanning unit for scanning the illumination light on the sample; an objective lens for projecting an image of the illumination light scanned by the confocal scanner onto the sample; and a position sensor for detecting the position of a container on which the sample is placed relative to the focal position of the objective lens; an imaging device having an imaging surface positioned conjugate to the scanning unit for detecting the light to be measured from the sample irradiated with the illumination light; and a processing device for controlling the operation of the microscope and the imaging device, wherein the processing device transmits control signals to the microscope and the imaging device for performing imaging by the imaging device, while displacing the relative position of the focal point of the objective lens with respect to the sample in a direction parallel to the optical axis of the illumination light, based on the position of the container relative to the focal position of the objective lens. In this way, by performing confocal imaging while displacing the focal position of the objective lens on the sample in a direction parallel to the optical axis of the illumination light, it is possible to acquire a clear image without partial brightness reduction within the field of view, even when the sample surface is tilted. The microscope system does not require a complex configuration and is therefore free from mechanical design limitations. The microscope system acquires the image in a single scan, so the imaging speed is not significantly reduced. Therefore, even when the sample surface is tilted, it is possible to acquire a clear image without partial brightness reduction within the field of view, without mechanical design limitations or a significant reduction in imaging speed.

[0011] In one embodiment, the confocal scanner rotates a pinhole disk on which a plurality of pinholes are arranged as the scanning unit to scan the illumination light emitted from the light-emitting device onto the sample, the objective lens projects the images of the plurality of pinholes, which are scanned by the illumination light from the confocal scanner, onto the sample, and the imaging device may have its imaging surface positioned conjugate to the plurality of pinholes. In this way, because the pinhole disk is rotated at high speed to scan the illumination light onto the sample, a confocal image of the sample can be formed on the imaging surface of the imaging device in a short time. Therefore, imaging can be performed at high speed.

[0012] In one embodiment, the microscope further comprises a drive device capable of displacing the objective lens in a direction parallel to the optical axis of the illumination light, and the processing device may control the microscope and the imaging device so that the imaging device performs imaging while the drive device displaces the objective lens.

[0013] In one embodiment, the processing apparatus may control the microscope and the imaging device so that imaging is performed by the imaging device while the relative position of the focal point of the objective lens with respect to the sample is displaced at a constant velocity. In this way, since the relative position of the focal point of the objective lens with respect to the sample is displaced at a constant velocity during imaging, it is possible to prevent the relative position of the sample with respect to the objective lens from moving in a direction perpendicular to the optical axis of the illumination light during imaging, thereby preventing distortion of the captured image.

[0014] In one embodiment, the processing apparatus may detect the local tilt of the container based on the position of the container relative to the focal position of the objective lens, determine the displacement range of the relative position of the focal point of the objective lens with respect to the sample during imaging by the imaging device based on the detected local tilt of the container, and control the microscope and the imaging device to perform imaging while displacing the relative position of the focal point of the objective lens with respect to the sample by the determined displacement range. In this way, by determining the displacement range of the relative position of the focal point of the objective lens with respect to the sample without excess or deficiency based on the local tilt of the container during imaging, it is possible to perform imaging by appropriately displacing the relative position of the focal point of the objective lens with respect to the sample and obtain a clear image.

[0015] In one embodiment, the local tilt of the container may be detected, and based on the local tilt, a process of determining the displacement range of the relative position of the focal point of the objective lens with respect to the sample may be performed, and the process of controlling the microscope and the imaging device to perform imaging by the imaging device while displacing the relative position of the focal point of the objective lens by the determined displacement range, thereby imaging each of the multiple samples placed in the container. In this way, by alternating between determining the displacement range and imaging the samples, it is possible to perform imaging of each of the multiple samples with the correct displacement corresponding to the local tilt of the container at high speed by performing only one scan.

[0016] In one embodiment, the processing apparatus may control the microscope and the imaging device so that imaging is performed by the imaging device while displacing the relative position of the focal point of the objective lens with respect to the sample by a displacement range specified by the user. In this way, imaging is performed while displacing the relative position of the focal point of the objective lens with respect to the sample by a displacement range specified by the user, so that imaging is performed with an appropriate displacement of the relative position of the focal point of the objective lens with respect to the sample, and a clear image can be obtained.

[0017] An imaging method for a microscope system according to several embodiments is an imaging method for a microscope system comprising a light-emitting device, a confocal scanner, a microscope, an imaging device, and a processing device for acquiring an image of a sample to be observed, the method comprising the steps of: the light-emitting device irradiating illumination light, the scanning unit of the confocal scanner scanning the illumination light onto the sample; the objective lens of the microscope projecting the image of the illumination light scanned by the confocal scanner onto the sample; the position sensor of the microscope detecting the position of the container on which the sample is placed with respect to the focal position of the objective lens; and the imaging device, whose imaging surface is positioned conjugate to the scanning unit, detecting the light to be measured from the sample irradiated with the illumination light, wherein the processing device transmits control signals to the microscope and the imaging device for imaging by the imaging device, while displacing the relative position of the focal point of the objective lens with respect to the sample in a direction parallel to the optical axis of the illumination light, based on the position of the container with respect to the focal position of the objective lens. In this way, by performing confocal imaging while displacing the focal position of the objective lens on the sample in a direction parallel to the optical axis of the illumination light, it is possible to acquire a clear image without partial brightness reduction within the field of view, even when the sample surface is tilted. The imaging method of the microscope system does not require a complex configuration and therefore does not impose limitations on mechanical design. The imaging method of the microscope system acquires the image in a single scan, so the imaging speed is not significantly reduced. Therefore, even when the sample surface is tilted, it is possible to acquire a clear image without partial brightness reduction within the field of view, without mechanical design limitations or a significant reduction in imaging speed.

[0018] Some embodiments of the program cause the computer to operate as a processing unit for the microscope system. In this way, confocal imaging is performed while displacing the focal position of the objective lens on the sample in a direction parallel to the optical axis of the illumination light. Therefore, even when the sample surface is tilted, confocal imaging makes it possible to capture a clear image without partial brightness reduction within the field of view. The program does not require a complex configuration and is therefore free from mechanical design limitations. The program acquires the image in a single scan, so the imaging speed is not significantly reduced. Consequently, even when the sample surface is tilted, it is possible to capture a clear image without partial brightness reduction within the field of view without mechanical design limitations or significant reductions in imaging speed. [Effects of the Invention]

[0019] According to one embodiment of the present disclosure, even when the sample surface is tilted, it is possible to capture a clear image of the sample without partial brightness reduction within the field of view, without mechanical design limitations or a significant decrease in imaging speed. [Brief explanation of the drawing]

[0020] [Figure 1] This figure shows the configuration of a microscope system according to one embodiment. [Figure 2A] Figure 1 is a top view of the well plate. [Figure 2B] Figure 2A is a cross-sectional view of the well plate. [Figure 3] This diagram schematically shows a side view of the imaged sample. [Figure 4] This diagram schematically shows a side view of the imaged sample. [Figure 5] This diagram schematically shows a side view of the imaged sample. [Figure 6] This diagram schematically illustrates the control of the objective lens's focal position according to the depth of the imaged sample. [Figure 7] This flowchart shows the operation of the imaging process performed by the microscope system. [Figure 8]This diagram schematically shows the deflection of the bottom surface of a well plate. [Figure 9A] This flowchart shows the operation of the imaging process performed by the microscope system. [Figure 9B] This flowchart shows the operation of the imaging process performed by the microscope system. [Figure 10A] This flowchart shows the operation of the imaging process performed by the microscope system. [Figure 10B] This flowchart shows the operation of the imaging process performed by the microscope system. [Modes for carrying out the invention]

[0021] <Comparative Example> Patent Document 1 describes an optical image separation device connected to the confocal image extraction port of a Nipow disk type confocal scanner. The configuration of Patent Document 1 separates the reflected light from the sample emitted from the confocal image extraction port of the confocal scanner into light in multiple wavelength regions using a dichroic mirror. As a result, the configuration of Patent Document 1 can acquire confocal images at high speed for each arbitrarily separated wavelength region or for multiple identical wavelength regions.

[0022] When confocal imaging is performed using the configuration described in Patent Document 1, if the sample surface is aligned with the imaging plane of the microscope, an image with uniform brightness across the entire field of view can be obtained. However, if confocal imaging is performed with the sample surface tilted relative to the imaging plane of the microscope, the brightness of the image will be significantly reduced in a part of the field of view due to the shallow depth of field.

[0023] To prevent a significant decrease in brightness in a portion of the field of view even when the sample surface is tilted relative to the imaging plane, it is conceivable to enlarge the pinhole diameter of the confocal scanner using the method described in Patent Document 2, thereby increasing the depth of focus on the sample surface and preventing a partial decrease in brightness of the sample image. However, using this method would reduce the scanning speed due to the decrease in the number of pinhole rows of the Nipow disk. Furthermore, the addition of a pinhole switching mechanism would limit the mechanical design and increase the equipment cost. In addition, enlarging the pinhole diameter would reduce the resolution in the XYZ directions. The XY directions are two mutually orthogonal directions in a plane perpendicular to the optical axis of the excitation light beam 11, and the Z direction is parallel to the optical axis of the excitation light beam 11.

[0024] Furthermore, as described in Patent Document 3, in addition to the means for acquiring the fluorescence signal from the sample as a confocal image, it is conceivable to provide a switchable means for acquiring the fluorescence signal from the sample as a fluorescence image, so that when the tilt of the sample surface with respect to the imaging plane is significant, a fluorescence image can be acquired. Because epifluorescence observation has a deep depth of field, even if the sample surface is tilted, the sample can be captured within that deep depth of field, preventing partial brightness reduction of the sample image. However, adopting such a configuration would require a mechanism for switching the optical path, which would limit the design of the mechanism and increase the equipment cost. In addition, there would be a mismatch in the field of view due to the axial misalignment of the confocal optical path and the epifluorescence optical path, and adjustments would be necessary to correct this. Moreover, in the configuration of Patent Document 3, there would be a mismatch in the magnification of the relay lens in the confocal optical path and the relay lens in the epifluorescence optical path, and image processing would be necessary to correct this.

[0025] Furthermore, as described in Patent Document 4, it is conceivable to configure a system that includes a confocal scanner that acquires slice images of a sample as confocal images, and an actuator that moves the focal position of the microscope's objective lens in the optical axis direction, thereby enabling the acquisition of slice images in the depth direction of the sample. However, in the configuration of Patent Document 4, the objective lens drive waveform is generated based on the video signal output from the video rate camera, so a separate camera capable of outputting a video signal is required. Also, the configuration of Patent Document 4 generates the objective lens drive waveform based on an analog signal, which is the video signal. Therefore, if the objective lens is driven by a stepping motor or the like, it is necessary to perform AD (Analog-to-Digital) conversion of the video signal, calculation of the objective lens drive pattern, and conversion of the objective lens drive pattern to a pulse waveform, which has the disadvantage of causing a time lag.

[0026] <Embodiment> The embodiments of this disclosure enable the acquisition of clear images without partial brightness reduction within the field of view, even when the sample surface is tilted, without mechanical design limitations or a significant decrease in imaging speed.

[0027] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In each drawing, parts having the same configuration or function are denoted by the same reference numerals. In the description of this embodiment, redundant descriptions of the same parts may be omitted or simplified as appropriate.

[0028] Figure 1 is a schematic diagram showing the configuration of a microscope system 1 according to one embodiment of the present disclosure. The microscope system 1 comprises a light-emitting device 10, a microscope 20, a confocal scanner 30, a camera 40, and a processing device 50.

[0029] In Figure 1, the light-emitting device 10 irradiates the sample 6 placed on the well plate 60 with an excitation luminous beam 11 as illumination light. The sample 6 has a fluorescent reagent attached to it, and when irradiated with the excitation luminous beam 11, the sample 6 emits a fluorescence signal 12 as the light to be measured.

[0030] The microscope 20 comprises an objective lens 21, a drive unit 22, a relay lens 23, and a position sensor 24. The objective lens 21 and the relay lens 23 constitute an infinity-correcting optical system and act optically on the excitation light beam 11 emitted from the light-emitting device 10 and the fluorescence signal 12 emitted from the sample 6. The drive unit 22 can displace the objective lens 21 in a direction parallel to the optical axis of the excitation light beam 11 based on a control signal transmitted from the processing unit 50.

[0031] The position sensor 24 detects the Z-direction position of the well plate (container) 60 on which the sample 6 is placed, for example, by a focus detection method, using a method such as focusing on the Z-direction positions of a predetermined number of reference points on the well plate 60. Such reference points may include the position of the bottom surface of the well plate 60. The position sensor 24 includes a light source 241, spectral mirrors 242, 243, and an optical sensor 244. When the position sensor 24 emits a focusing light 25, the focusing light 25 is reflected by the spectral mirrors 242, 243 and guided to the objective lens 21. The spectral mirror 242 may be, for example, a half mirror or a polarizing beam splitter. The spectral mirror 243 may be, for example, a dichroic mirror. The focusing light 25 is focused by the objective lens 21 and irradiated onto the bottom surface of the well plate 60. The focusing light 25 reflected from the bottom surface of the well plate 60 passes through the objective lens 21 and returns to the position sensor 24. The reflected focusing light 25 is reflected by the spectral mirror 243 in the position sensor 24, passes through the spectral mirror 242, and is detected by the light sensor 244. The focusing light 25 detected by the light sensor 244 is output to the processing unit 50 as a focus error signal that reflects the error from the focal position of the objective lens 21. Based on the focus error signal, the processing unit 50 detects the position of the reference point of the well plate 60 in the Z direction. In this way, the position sensor 24 may detect the position of the objective lens 21 in the Z direction where the focal plane of the objective lens 21 is located at the reference point of the well plate 60, and then detect the position of the reference point in the Z direction based on the Z position of the objective lens 21. In addition to the astigmatism method, the position sensor 24 may also detect the position of the well plate 60 in the Z direction using any focus detection method such as the knife-edge method, confocal method, and triangulation method.

[0032] The confocal scanner 30 is optically connected to the microscope 20 and the light-emitting device 10. For example, the confocal scanner 30 is attached to both the microscope 20 and the light-emitting device 10. The confocal scanner 30 comprises a pinhole array disk (hereinafter referred to as "Nipow disk") 31, a component 32, a microlens array disk (hereinafter referred to as "ML disk") 33, a dichroic mirror (hereinafter referred to as "DM") 34, a relay lens 35, a bandpass filter 36, and a relay lens 37.

[0033] DM34 is designed to transmit the excitation light beam 11 and reflect the desired fluorescence signal 12. Multiple condensing optical elements (microlenses) are arranged on the ML disk 33, for example, in a helical pattern. The Nipou disk 31 has multiple pinholes, each positioned at the focal point of the excitation light beam 11 by the multiple condensing optical elements of the ML disk 33. Based on a control signal transmitted from the processing unit 50, the confocal scanner 30 can rotate the Nipou disk 31 and the ML disk 33 at high speed around the rotation axis 39 using a motor or the like, while they are mechanically connected to each other by a member 32. Here, the individual microlenses and pinholes are arranged on the Nipou disk 31 so that each pinhole sweeps the surface of the sample 6. Therefore, the Nipou disk 31 functions as a scanning unit that scans the illumination light (excitation light beam 11) over the sample 6. The Nipou disk 31 rotates the multiple pinholes in a plane substantially perpendicular to the optical axis of the excitation light beam 11. During the imaging operation, the ML disk 33 and the Nipow disk 31 are constantly rotating.

[0034] Most of the excitation light beam 11 that passes through the ML disk 33 passes through the Nipou disk 31. Therefore, when the confocal scanner 30 uses the ML disk 33 and the Nipou disk 31 together, the irradiation intensity of the excitation light beam 11 onto the sample 6 is increased compared to when the Nipou disk 31 is used alone. Furthermore, reflection of the excitation light beam 11 in parts of the Nipou disk 31 other than the pinhole is suppressed. Consequently, the signal-to-noise ratio (SNR) of the image of the sample 6 is increased.

[0035] The excitation beam 11 is focused into individual beams by the ML disk 33, passes through the DM 34, and then through the individual pinholes of the Nipow disk 31. After passing through the relay lens 23 of the microscope 20, the excitation beam 11 is focused by the objective lens 21 onto the sample 6 placed in the wells (holes) of the well plate 60.

[0036] As described above, a fluorescent reagent is added to each of the samples 6 in the well plate 60. The fluorescent signals 12 emitted by each fluorescent reagent in the sample 6 pass through the objective lens 21 and relay lens 23 again and are focused onto the individual pinholes of the Nipow disk 31.

[0037] The fluorescence signal 12 that passes through the pinhole of the Nipow disk 31 is reflected by the DM 34. The confocal scanner 30 has relay lenses 35 and 37 of an infinity correction optical system that images the fluorescence signal 12 reflected by the DM 34 onto the two-dimensional sensor (image sensor) 41 of the camera 40, which is an imaging device. The confocal scanner 30 also has a bandpass filter 36 between the relay lenses 35 and 37. The bandpass filter 36 may be implemented by a barrier filter (absorption filter) that selectively transmits the fluorescence contained in the fluorescence signal 12.

[0038] The plane on which the pinholes of the Nipow disk 31 are arranged, the surface of the sample 6, and the light-receiving surface of the camera 40's two-dimensional sensor 41 are arranged in an optically conjugate relationship with each other. Therefore, an optical cross-sectional image of the sample 6, i.e., a confocal image, is formed on the camera 40's two-dimensional sensor 41. Furthermore, as described above, the objective lens 21 drive device 22 is driven based on a control signal from the processing device 50 to displace the objective lens 21 in a direction parallel to the optical axis of the excitation light beam 11. Since the objective lens 21 and the relay lens 23 constitute an infinity correction optical system, the displacement of the objective lens 21 can displace the focal plane of the objective lens 21 on the sample 6 while maintaining an optically conjugate relationship with the Nipow disk 31 and the light-receiving surface of the two-dimensional sensor 41. The focal plane of the objective lens 21 constitutes the imaging plane of the microscope 20. Based on the exposure signal from the processing device 50, the camera 40 exposes the image projected onto the light-receiving surface of its two-dimensional sensor 41 for a specified time and converts it into digital image data. In other words, the camera 40 performs exposure while receiving an ON exposure signal from the processing unit 50, forming an image on the two-dimensional sensor 41. The digital image data is transferred to the processing unit 50 and stored in the storage unit 52 of the processing unit 50.

[0039] The processing unit 50 is connected to each component included in the microscope system 1 and controls the operation of the entire microscope system 1 by transmitting control signals to each component. However, the processing unit 50 transmits an exposure signal to the camera 40 to control exposure during imaging. The processing unit 50 also provides a user interface for the user to operate the microscope system 1 and perform observations. The processing unit 50 is, for example, a computer device and includes any device such as a PC (Personal Computer), tablet PC, mobile phone such as a smartphone and feature phone, and a personal digital assistant (PDA).

[0040] The processing unit 50 comprises a control unit 51, a storage unit 52, and an input / output unit 53. The control unit 51 includes one or more processors. In one embodiment, the "processor" is a general-purpose processor or a dedicated processor specialized for a specific process, but is not limited to these. The control unit 51 is communicatively connected to each component constituting the processing unit 50 and controls the operation of the entire processing unit 50.

[0041] The storage unit 52 includes any storage module, including an HDD (Hard Disk Drive), SSD (Solid State Drive), ROM (Read-Only Memory), and RAM (Random Access Memory). The storage unit 52 may function, for example, as main memory, auxiliary memory, and cache memory. The storage unit 52 stores any information used in the operation of the processing unit 50 or obtained as a result of the operation of the processing unit 50. For example, the storage unit 52 stores programs such as system programs and application programs for operating the processing unit 50, as well as image data of images captured by the camera 40.

[0042] The functions of the processing unit 50 can be realized by executing a program (computer program) that can be used to operate the microscope system 1 according to this embodiment on the processor included in the control unit 51. In other words, the functions of the processing unit 50 can be realized by software. The program causes the computer to execute the processing steps included in the operation of the processing unit 50, thereby realizing the functions corresponding to the processing of each step. In other words, the program is a program that causes the computer to function as the processing unit 50 according to this embodiment.

[0043] The program can be recorded on a computer-readable recording medium. Examples of computer-readable recording media include magnetic recording devices, optical discs, magneto-optical recording media, or semiconductor memory. The program can be distributed, for example, by selling, transferring, or leasing portable recording media such as DVDs (Digital Versatile Discs) or CD-ROMs (Compact Disc ROMs) on which the program is recorded. The program may also be distributed by storing it in server storage and transferring it from the server to other computers via a network. The program may also be provided as a program product.

[0044] A computer, for example, stores a program recorded on a portable storage medium or a program transferred from a server in its main memory. Then, the computer reads the program stored in the main memory with its processor and executes the processing according to the read program. The computer may also read the program directly from the portable storage medium and execute the processing according to the program. The computer may also execute the processing according to the received program sequentially each time a program is transferred to it from a server. Such processing may also be performed by a so-called ASP (Application Service Provider) type service, which realizes its function only through execution instructions and result retrieval, without transferring the program from the server to the computer. A program includes information used for processing by an electronic computer that is equivalent to a program. For example, data that is not a direct instruction to the computer but has the nature of defining the computer's processing falls under the category of "information equivalent to a program."

[0045] Some or all of the functions of the processing unit 50 may be implemented by a dedicated circuit included in the control unit 51. In other words, some or all of the functions of the processing unit 50 may be implemented by hardware. Furthermore, the processing unit 50 may be implemented by a single information processing unit, or by the cooperation of multiple information processing units.

[0046] The input / output unit 53 includes an input unit that receives user operations and inputs information based on those operations, and an output unit that outputs the calculation results of the processing unit 50 and captured images taken by the camera 40. The input unit is, for example, a physical key, a capacitive key, a pointing device, a touchscreen integrated with the display of the output unit, or a microphone that accepts voice input. The output unit is, for example, a display that outputs information as an image, or a speaker that outputs information as sound.

[0047] The microscope system 1 having the above configuration performs imaging while displacing the focal plane of the objective lens 21 in a direction parallel to the optical axis of the excitation light beam 11 relative to the sample 6, thereby enabling the acquisition of a focused image across the entire field of view even when the sample 6 is tilted.

[0048] Figure 2A is a top view of the well plate 60, which serves as a container on which the sample 6 is placed during microscopic cell imaging. As shown in Figure 2A, the well plate 60 is provided with numerous wells 68 for placing the sample 6.

[0049] Figure 2B is a cross-sectional view of the well plate 60 shown in Figure 2A, obtained by cutting along surface AA. As shown in Figure 2B, the well plate 60 comprises a frame 61 and a bottom surface 62.

[0050] The frame 61 is made of resin, and the bottom surface 62, which serves as the sample surface, is generally made of a thin sheet of resin or glass. Therefore, due to distortion (approximately several hundred μm) during the molding of the frame 61, the frame 61 may bend, preventing the well plate 60 from being placed horizontally on the sample stage of the microscope 20, resulting in the sample surface being tilted relative to the focal plane of the microscope 20. In addition, due to distortion (approximately several tens of μm) during the molding of the bottom surface 62, the bottom surface 62 may bend, causing the sample surface to be tilted relative to the focal plane of the microscope 20.

[0051] Figures 3 to 5 schematically show a side view of the sample 63 to be imaged. In Figure 3, the cell culture surface 64 is positioned horizontally, and the culture surface 64 of the sample 63 to be imaged aligns with the focal plane 65 of the objective lens 21 of the microscope 20. Therefore, when imaging is performed in this state, the camera 40 acquires an image that is in focus across the entire field of view.

[0052] In Figure 4, the cell culture surface 64 is tilted relative to the focal plane 65 of the objective lens 21. Therefore, if imaging is performed in this state, an image will be obtained in which the brightness of a part of the field of view is significantly reduced. In Figure 5, numerous samples to be imaged 63 have gathered together to form a thick mass 66. Therefore, if imaging is performed using a confocal microscope with the focal plane 65 of the microscope 20 passing through the center of the mass 66, only a portion of the thick mass 66 will be reflected in the image.

[0053] Figure 6 is a schematic diagram illustrating the control of the focal position of the objective lens 21 in accordance with the depth of the sample 63 to be imaged, as performed by the microscope system 1 according to this embodiment. In Figure 6, the culture surface 64 of the sample 63 to be imaged is inclined with respect to the focal plane. Therefore, the processing device 50 controls the camera 40 to perform imaging while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 in a direction parallel to the optical axis of the excitation light beam 11. Specifically, in this embodiment, the processing device 50 controls the camera 40 to perform imaging while the drive device 22 displaces the objective lens 21 without moving the sample 6. The displacement width 71 of the objective lens 21 may be the width of the range in which the sample 6 is distributed in the well plate 60 in a direction parallel to the optical axis of the excitation light beam 11. As shown on the right of Figure 6, the processing device 50 may transmit an exposure signal to the camera 40 and perform exposure while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 at a constant velocity as time t progresses. This allows for the acquisition of an image that covers the depth range in which sample 6 is distributed, making it possible to obtain a clear image.

[0054] (Example 1) Next, with reference to Figures 7 and 8, an embodiment 1 of the operation of the microscope system 1 will be described. Figure 7 is a flowchart showing the operation of the imaging process performed by the microscope system 1. Figure 8 is a schematic diagram showing the deflection of the bottom surface 62 of the well plate 60. The operation of the microscope system 1 described with reference to Figures 7 and 8 corresponds to one of the imaging methods according to Embodiment 1. The operation of each step in Figure 7 is performed based on the control of the control unit 51 of the processing device 50. The program for causing a computer to execute the imaging method according to this embodiment includes each step shown in Figure 7. As a prerequisite for the following process, the well plate 60 on which the sample 6 is placed is set to a measurement position for measurement.

[0055] In step S001, the control unit 51 receives from the user the setting of measurement conditions related to imaging by the microscope system 1. Examples of such measurement conditions include the following: • Displacement range Δz of the relative position of the focal point of the objective lens 21 with respect to the sample 6a (Moving range of the focal position of the objective lens 21) · Exposure time T exp · List of XY positions on the well plate 60 where imaging is performed Here, the displacement width Δz a is, for example, 10 μm to several hundred μm, and is a value indicating the range in the Z direction (direction parallel to the optical axis of the excitation light beam 11) where the sample 6 exists, taking into account the thickness of the sample 6 and the inclination and deflection of the well plate 60. The exposure time T exp may be a value of about 50 ms to 1 s. Also, the control unit 51 may receive an input of the dimension information of the well plate 60.

[0056] The XY position is the position on the well plate 60. As shown in FIG. 8, the XY position can be represented as (x i , y j ), for example, where i is an integer from 1 to m and j is an integer from 1 to n. Each of x i , y j can be associated with, for example, A, B, C, ··· and 1, 2, 3, ··· that identify the wells 68 as shown in FIG. 2A. When there is no inclination and distortion in the well plate 60, the Z-direction positions of the bottom surface 62 of the well plate 60 for each of the XY positions (x i , y j ) are the same, but when inclination or distortion occurs, the Z-direction positions of the bottom surface 62 of the well plate 60 for each of the XY positions (x i , y j ) are not the same. As shown in FIG. 8, the Z-direction position of the bottom surface 62 of the well plate 60 at the XY position (x i , y j ) is represented as z i,j i,j .

[0057] In step S002, the control unit 51 controls each component of the microscope system 1 to align each component and moves the well plate 60 in the XY direction so that the sample 6 at the XY position where the first image is to be taken is within the imaging range of the excitation light beam 11. Subsequently, the control unit 51 performs the processing in steps S003 to S009 for each of the XY positions included in the list that was set in step S001. i ,y j ) is expressed as.

[0058] In step S003, the control unit 51 controls the drive unit 22 to move the objective lens 21 in the Z direction, while reading the signal from the position sensor 24, and the position of the objective lens 21 when the focal point of the objective lens 21 coincides with the bottom surface 62 of the well plate 60. 0(i,j) Search.

[0059] In step S004, the control unit 51 controls the drive unit 22 to find the position z in step S003. 0(i,j) Move the objective lens 21.

[0060] In step S005, the control unit 51 controls the confocal scanner 30 and the light-emitting device 10 to rotate the ML disk 33 and the Nipou disk 31 while emitting an excitation light beam 11 from the light-emitting device 10. As a result, the excitation light beam 11 is focused into individual beams by the ML disk 33, passes through the DM 34, passes through individual pinholes in the Nipou disk 31, and is focused by the objective lens 21 of the microscope 20 onto the sample 6 placed in the well 68 of the well plate 60. The fluorescence signal 12 emitted by the fluorescent reagent in the sample 6 due to the excitation light beam 11 passes through the objective lens 21 again and is focused onto individual pinholes in the Nipou disk 31. The fluorescence signal 12 that has passed through the individual pinholes is reflected by the DM 34, passes through relay lenses 35, 37, and is emitted from the aperture (confocal image extraction port) of the confocal scanner 30 so that it is imaged to the camera 40 via the bandpass filter 36. Meanwhile, the control unit 51 rotates the objective lens 21 z 0(i,j) from z 0(i,j) +Δz aFor example, while moving at a constant speed, exposure time T is recorded with camera 40. exp Only the area is exposed, and an image is acquired. Specifically, the control unit 51 sends a control signal to the drive unit 22, causing the drive unit 22 to displace the objective lens 21, while simultaneously sending an exposure signal to the camera 40 to expose the 2D sensor 41 and perform imaging.

[0061] Once the objective lens 21 has been driven and the camera 40 has finished exposing, in step S006, the control unit 51 stops emitting the excitation light beam 11 from the light-emitting device 10.

[0062] In step S007, the control unit 51 transfers the image data acquired by the camera 40 to the storage unit 52 of the processing unit 50.

[0063] In step S008, the control unit 51 determines whether imaging has been completed for all XY positions specified by the user in step S001. If imaging has not been completed for all XY positions, that is, if there are XY positions specified by the user that have not yet been imaged (NO in step S008), the control unit 51 proceeds to step S009. If imaging has been completed for all XY positions (YES in step S008), the control unit 51 terminates the series of operations.

[0064] In step S009, the control unit 51 moves the well plate 60 in the XY direction so that the sample 6 at the XY position for the next imaging is within the imaging range of the excitation light beam 11, and then returns to step S003.

[0065] As described above, in Embodiment 1, the control unit 51 determines the XY position (x) at which to perform the shooting. i ,y j The position z of the objective lens 21 corresponding to the bottom surface 62 of the well plate 60 in ) 0(i,j) The control unit 51 then detects the position z 0(i,j) From the user-defined displacement range Δz in the Z direction aThe sample 6 is imaged while moving the objective lens 21 in the z direction. In this way, in Example 1, confocal imaging is performed while displacing the focal position of the objective lens 21 on the sample 6 in a direction parallel to the optical axis of the excitation light beam 11. Therefore, even if the sample surface is tilted, confocal imaging makes it possible to capture a clear image without partial brightness reduction in the field of view. Furthermore, since the microscope system 1 does not require a complex configuration, there are no limitations in terms of mechanical design. Since the microscope system 1 acquires the image in a single image, the imaging speed is not significantly reduced. Therefore, even if the sample surface is tilted, it is possible to capture a clear image without partial brightness reduction in the field of view without limitations in mechanical design or a significant reduction in imaging speed.

[0066] (Example 2) Next, with reference to Figures 8, 9A, and 9B, Example 2 of the operation of the microscope system 1 will be described. In Example 1, for each XY position on the well plate 60, the position of the bottom surface 62 of the well plate 60 is detected, and a displacement width Δz set by the user is set from the position of the bottom surface 62. a An example of imaging while displacing by only Δz was described. In this embodiment, the displacement width Δz of the range in which the sample 6 exists corresponds to the local inclination of the well plate 60 at each XY position. c(i,j) An example of detecting and imaging within the imaging range that reflects this detection will be described. Figures 9A and 9B are flowcharts showing the operation of the imaging process performed by the microscope system 1. The operation of the microscope system 1 described with reference to Figures 9A and 9B corresponds to one of the imaging methods according to Embodiment 2. The operation of each step in Figures 9A and 9B is performed based on the control of the control unit 51 of the processing device 50. The program for causing a computer to execute the imaging method according to this embodiment includes the steps shown in Figures 9A and 9B. As a prerequisite for the following process, the well plate 60 on which the sample 6 is placed is set to a measurement position for measurement.

[0067] In step S101, the control unit 51 receives from the user the setting of measurement conditions related to imaging by the microscope system 1. Examples of such measurement conditions include the following: • Displacement range Δz of the relative position of the focal point of the objective lens 21 with respect to the sample 6 b (Movement range of the focal position of the objective lens 21) • Exposure time T exp • List of XY positions on the well plate 60 on which imaging will be performed. Here, the displacement width Δz b This value, for example, ranges from 10 μm to several hundred μm and indicates the width in the Z direction, which is entered by the user based on the thickness of sample 6. Exposure time T exp This value may be approximately 50ms to 1s. The control unit 51 may also accept input of dimensional information for the well plate 60.

[0068] In step S102, the control unit 51 performs a prescan based on the list of XY positions of the well plate 60 on which imaging is to be performed, at the XY positions (x1, y1) ~ (x m ,y n Set ).

[0069] In step S103, the control unit 51 controls each component of the microscope system 1 to align each component and moves the well plate 60 in the XY direction so that the sample 6 at the XY position where the first pre-scan is performed is located within the imaging range of the excitation light beam 11. Pre-scan refers to each XY position (x i ,y j In this case, the focal point of the objective lens 21 coincides with the bottom surface 62 of the well plate 60 at a position z in the Z direction. 0(i,j) This is the operation to search for. Below, the control unit 51 executes the processes in steps S104 to S106 for each of the XY positions set in step S102. Below, the XY position to be processed is (x i ,y j ) is expressed as.

[0070] In step S104, the control unit 51 controls the drive unit 22 to move the objective lens 21 in the Z direction, while reading the signal from the position sensor 24, and the position of the objective lens 21 in the Z direction when the focal point of the objective lens 21 coincides with the bottom surface 62 of the well plate 60. 0(i,j) Search for position z. 0(i,j) is the XY position (x i ,y j This is the position of the objective lens 21 in the Z direction relative to ).

[0071] In step S105, the control unit 51 performs a prescan at all XY positions (x i ,y j ) in z 0(i,j) The control unit 51 determines whether the search is complete. If the pre-scan is not complete for all XY positions on which the pre-scan is to be performed, that is, if there are XY positions specified by the user on which the pre-scan is not complete (NO in step S105), the control unit 51 proceeds to step S106. If the pre-scan is complete for all XY positions on which the pre-scan is to be performed (YES in step S105), the control unit 51 proceeds to step 107.

[0072] In step S106, the control unit 51 moves the well plate 60 in the XY direction so that the sample 6 at the XY position for the next pre-scan is within the imaging range of the excitation light beam 11, and then returns to step S104.

[0073] In step S107, the control unit 51 receives the prescan result (x1, y1, z 0(1,1) )~(x m ,y n ,z 0(m,n) Based on this, each XY position (x) in which imaging is performed is used. i ,y j The inclination of the bottom surface 62 of the well plate 60 in ) grad(z i,j Calculate the slope grad(z). i,j ) indicates the slope in the X direction grad(z i,j ) x And the gradient (z) indicates the slope in the Y direction. i,j )y This becomes a vector quantity having the following properties: grad(z i,j ) x and grad(z i,j ) y This can be calculated, for example, by the following formula 1. [Formula 1] grad(z i,j ) x =( z i+1,j -z i-1,j ) / (x i+1 -x i-1 ) grad(z i,j ) y =( z i,j+1 -z i,j-1 ) / (y j+1 -y j-1 )

[0074] In step S108, the control unit 51 determines the inclination grad(z) of the bottom surface 62 of the well plate 60 at the XY position where imaging is performed. i,j ) and the size of the 2D sensor 41 of the camera 40 are used to perform imaging at each XY position, which is the imaging Z range Δz c(i,j) We calculate Δz. c(i,j) This is the range in the Z direction where the sample 6 exists due to the tilt of the well plate 60. For example, suppose the 2D sensor 41 has a rectangle, with length L1 corresponding to the imaging range in the X direction and length L2 corresponding to the imaging range in the Y direction. In this case, the amount of displacement in the Z direction due to the tilt in the X direction is L1 × grad(z i,j ) x Therefore, the amount of displacement in the Z direction due to the tilt in the Y direction is L² × grad(z i,j ) y Therefore, the control unit 51 determines the imaging Z range Δz by, for example, the following equation 2. c(i,j) You may calculate this. [Formula 2] Δz c(i,j) =L1×grad(z i,j ) x +L2×grad(z i,j ) y

[0075] In step S109, the control unit 51 controls each component of the microscope system 1 to align the components, and moves the well plate 60 in the XY direction so that the sample 6 at the XY position for the first imaging is located within the imaging range by the excitation light beam 11. Hereinafter, the control unit 51 executes the processes of steps S110 to S116 for each of the XY positions included in the list received in step S001. Hereinafter, the XY position to be processed is represented as (x i , y j ).

[0076] In step S110, the control unit 51 controls the drive device 22, reads the signal of the position sensor 24 while moving the objective lens 21 in the Z direction, and searches for the position z 0(i,j) where the focus of the objective lens 21 coincides with the bottom surface 62 of the well plate 60.

[0077] In step S111, the control unit 51 controls the drive device 22 to move the objective lens 21 to the position z 0(i,j) searched in step S110.

[0078] In step S112, the control unit 51 controls the confocal scanner 30 and the light emitting device 10 so as to rotate the ML disk 33 and the Nipow disk 31 while emitting the excitation light beam 11 from the light emitting device 10. As a result, the excitation light beam 11 is condensed into individual light beams by the ML disk 33, passes through the DM34, passes through the individual pinholes of the Nipow disk 31, and is condensed onto the sample 6 placed in the well 68 of the well plate 60 by the objective lens 21 of the microscope 20. The fluorescence signal 12 emitted from the fluorescent reagent of the sample 6 by the excitation light beam 11 passes through the objective lens 21 again and is condensed onto the individual pinholes of the Nipow disk 31. The fluorescence signal 12 that has passed through the individual pinholes is reflected by the DM34, passes through the relay lenses 35 and 37, and is emitted from the aperture (confocal image extraction port) of the confocal scanner 30 so as to form an image on the camera 40 via the bandpass filter 36. On the other hand, the control unit 51 moves the objective lens 21 from z 0(i,j) to z 0(i,j) +Δz b +Δzc(i,j) while moving, for example, at a constant speed, the camera 40 exposes for an exposure time T exp and acquires an image. Specifically, the control unit 51 transmits a control signal to the driving device 22 to displace the objective lens 21 while transmitting an exposure signal to the camera 40 to expose the two-dimensional sensor 41 to perform imaging.

[0079] When the driving of the objective lens 21 and the exposure of the camera 40 are completed, in step S113, the control unit 51 terminates the emission of the excitation light beam 11 from the light emitting device 10.

[0080] In step S114, the control unit 51 transfers the image data acquired by the camera 40 to the storage unit 52 of the processing device 50.

[0081] In step S115, the control unit 51 determines whether imaging for all XY positions where imaging is to be performed as specified by the user in step S101 has ended. If imaging for all XY positions where imaging is to be performed has not ended, that is, if there are un-imaged XY positions specified by the user (NO in step S115), the control unit 51 proceeds to step S116. If imaging has ended at all XY positions where imaging is to be performed (YES in step S115), the control unit 51 ends the series of operations.

[0082] In step S116, the control unit 51 moves the well plate 60 in the XY direction so that the sample 6 at the XY position for the next imaging is located within the imaging range by the excitation light beam 11, and returns to step S113.

[0083] As described above, in addition to the processing of the first embodiment, the microscope system 1 according to the second embodiment performs a pre-scan to detect the local inclination of the well plate 60 at each XY position. Then, the microscope system 1 determines the displacement width Δz of the range where the sample 6 corresponding to the local inclination of the well plate 60 exists c(i,j)The system detects the tilt of the well plate 60 at the XY position where imaging is performed and predicts the range in the Z direction where the sample 6 is located. Therefore, it is possible to set an appropriate imaging Z range and perform imaging.

[0084] (Example 3) Next, with reference to Figures 8, 10A, and 10B, Example 3 of the operation of the microscope system 1 will be described. In Example 2, a pre-scan is performed to determine the displacement width Δz for each XY position on the well plate 60, corresponding to the local inclination of the well plate 60. c(i,j) An example was described in which the detection of the first XY position is performed, and then the scan is performed again to image the sample 6 at each XY position. In this embodiment, the displacement width Δz corresponding to the local tilt of the well plate 60 at each XY position is described within a single scan. c(i,j) An example of performing detection and imaging of sample 6 will be described. Figures 10A and 10B are flowcharts showing the operation of the imaging process performed by the microscope system 1. The operation of the microscope system 1 described with reference to Figures 10A and 10B corresponds to one of the imaging methods according to Embodiment 3. The operation of each step in Figures 10A and 10B is performed based on the control of the control unit 51 of the processing device 50. The program for causing a computer to perform the imaging method according to this embodiment includes the steps shown in Figures 10A and 10B. As a prerequisite for the following process, the well plate 60 on which the sample 6 is placed is set to a measurement position for measurement.

[0085] In step S201, the control unit 51 receives from the user the setting of measurement conditions related to imaging by the microscope system 1. Examples of such measurement conditions include the following: • Displacement range Δz of the relative position of the focal point of the objective lens 21 with respect to the sample 6 b , Δz d (Movement range of the focal position of the objective lens 21) • Exposure time T exp • List of XY positions on the well plate 60 on which imaging is performed (x1, y1) ~ (xm ,y n ) Here, the displacement width Δz b This value, for example, ranges from 10 μm to several hundred μm and represents the width in the Z direction entered by the user based on the thickness of sample 6. Displacement width Δz d This value is, for example, a few μm to a few tens of μm, and is a value that the user predicts and inputs regarding the range in the Z direction where the sample 6 exists, based on the tilt of the well plate 60 at the XY position where imaging is performed. Exposure time T exp This value may be approximately 50ms to 1s. The control unit 51 may also accept input of dimensional information for the well plate 60.

[0086] In step S202, the control unit 51 moves the well plate 60 in the XY direction so that the sample 6 at the first XY position to be imaged is within the imaging range of the excitation light beam 11. Subsequently, the control unit 51 performs the processing in steps S203 to S213 for each of the XY positions set in step S201. Subsequently, the XY positions to be processed are (x i ,y j ) is expressed as follows. Hereafter, the control unit 51 is (x1, y1), (x2, y1), ..., (x m ,y1),(x1,y2),(x2,y2),...,(x m ,y2),...,(x1,y n ), (x2,y n ), , , (x m ,y n The following example explains how to execute the processes in the order shown, but you may execute them in any other order.

[0087] In step S203, the control unit 51 controls the drive unit 22 to move the objective lens 21 in the Z direction, while reading the signal from the position sensor 24, and the position of the objective lens 21 in the Z direction when the focal point of the objective lens 21 coincides with the bottom surface 62 of the well plate 60. 0(i,j) Search for position z. 0(i,j) is the XY position (x i ,y j This is the position of the objective lens 21 in the Z direction relative to ).

[0088] In step S204, the control unit 51 determines the current XY position (x i ,y j For a given XY position (x i-1 ,y j ), (x i ,y j-1 The position z of the objective lens 21 in ) where the focal point of the objective lens 21 coincides with the bottom surface 62 of the well 68. 0(i-1,j) and z 0(i,j-1) Determine if it is known. For example, (x1, y1), (x2, y1), ..., (x m ,y1),(x1,y2),(x2,y2),...,(x m ,y2),...,(x1,y n ), (x2,y n ), , , (x m ,y n If the processing is executed in the order of ), then if i>1 and j>1, (x i-1 ,y j ) and (x i ,y j-1 Since the processing in ) is finished, z 0(i-1,j) and z 0(i,j-1) It is known. If it is known (YES in step S204), the control unit 51 proceeds to step S206. If it is not known (NO in step S204), the control unit 51 proceeds to step S205.

[0089] In step S205, the control unit 51 determines the XY position (x) of the elements of the Z scan range during imaging. i ,y j The inclination of the well plate 60 in ) and the resulting range in the Z direction where the sample 6 exists, Δz c(i,j) For example, the Δz specified by the user in step S201 d The control unit 51 then proceeds to step S208.

[0090] In step S206, the control unit 51 determines the current XY position (x i ,y j The inclination grad(z) of the bottom surface 62 of the well plate 60 i,jThe control unit 51 calculates grad(z) using, for example, the following equation 3. i,j ) x and grad(z i,j ) y You may calculate this. [Formula 3] grad(z i,j ) x =( z i,j -z i-1,j ) / (x i -x i-1 ) grad(z i,j ) y =( z i,j -z i,j-1 ) / (y j -y j-1 )

[0091] In step S207, the control unit 51 calculates grad(z i,j Based on the size of the 2D sensor 41 of the camera 40, the XY position (x i ,y j ) Imaging Z range Δz c(i,j) The following equation 4 calculates the imaging Z range Δz. For example, if the 2D sensor 41 is a rectangle having a length L1 corresponding to the imaging range in the X direction and a length L2 corresponding to the imaging range in the Y direction, the control unit 51 calculates the imaging Z range Δz using the following equation 4, similar to S108 in Figure 9A. c(i,j) You may calculate this. [Formula 4] L1 × grad(z i,j ) x +L2×grad(z i,j ) y Then, the control unit 51 proceeds to step S208.

[0092] In step S208, the control unit 51 controls the drive unit 22 to find the position z in step S203. 0(i,j) Move the objective lens 21.

[0093] In step S209, the control unit 51 controls the confocal scanner 30 and the light-emitting device 10 to rotate the ML disk 33 and the Nipou disk 31 while emitting an excitation light beam 11 from the light-emitting device 10. As a result, the excitation light beam 11 is focused into individual beams by the ML disk 33, passes through the DM 34, passes through individual pinholes in the Nipou disk 31, and is focused by the objective lens 21 of the microscope 20 onto the sample 6 placed in the well 68 of the well plate 60. The fluorescence signal 12 emitted by the fluorescent reagent in the sample 6 due to the excitation light beam 11 passes through the objective lens 21 again and is focused onto individual pinholes in the Nipou disk 31. The fluorescence signal 12 that has passed through the individual pinholes is reflected by the DM 34, passes through relay lenses 35, 37, and is emitted from the aperture (confocal image extraction port) of the confocal scanner 30 so that it is imaged to the camera 40 via the bandpass filter 36. Meanwhile, the control unit 51 rotates the objective lens 21 z 0(i,j) from z 0(i,j) +Δz b +Δz c(i,j) For example, while moving at a constant speed, exposure time T is recorded with camera 40. exp Only the area is exposed, and an image is acquired. Specifically, the control unit 51 sends a control signal to the drive unit 22, causing the drive unit 22 to displace the objective lens 21, while simultaneously sending an exposure signal to the camera 40 to expose the 2D sensor 41 and perform imaging.

[0094] Once the objective lens 21 has been driven and the camera 40 has finished exposing, in step S210 the control unit 51 stops emitting the excitation light beam 11 from the light-emitting device 10.

[0095] In step S211, the control unit 51 transfers the image data acquired by the camera 40 to the storage unit 52 of the processing unit 50.

[0096] In step S212, the control unit 51 determines whether imaging has been completed for all XY positions specified by the user in step S201. If imaging has not been completed for all XY positions, that is, if there are XY positions specified by the user for which imaging has not been completed (NO in step S212), the control unit 51 proceeds to step S213. If imaging has been completed for all XY positions (YES in step S212), the control unit 51 terminates the series of operations.

[0097] In step S213, the control unit 51 moves the well plate 60 so that the sample 6 at the XY position for the next imaging is positioned within the imaging range in the XY direction by the excitation light beam 11, and returns to step S203.

[0098] As described above, the microscope system 1 according to Example 3 performs imaging without pre-scanning, and while performing the imaging operation, it captures the displacement width Δz in the Z direction where the sample 6 is located, which reflects the local inclination of the well plate 60 at the XY position where imaging is performed. c(i,j) This is calculated. Therefore, according to Example 3, it is possible to set an imaging Z range that is neither excessive nor insufficient, and to perform imaging in that imaging Z range at high speed.

[0099] Furthermore, the processing unit 50 may control the microscope 20 and camera 40 so that imaging is performed by the camera 40 while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 at a constant velocity specified by the user. In this way, since the relative position of the focal point of the objective lens 21 with respect to the sample 6 is displaced at a constant velocity during imaging, it is possible to prevent the relative position of the sample 6 with respect to the objective lens 21 from moving in a direction perpendicular to the optical axis of the excitation light beam 11 (XY direction) during imaging, thereby preventing distortion of the captured image.

[0100] Furthermore, the processing device 50 changes the speed at which it displaces the relative position of the focal point of the objective lens 21 with respect to the sample 6, according to the exposure time T specified by the user. exp and relative position displacement range Δz a You may make a decision based on this.

[0101] As described above, in this embodiment, the microscope system 1 for acquiring an image of the sample 6 to be observed comprises a light-emitting device 10, a microscope 20, a confocal scanner 30, a camera 40, and a processing device 50. The light-emitting device 10 irradiates with an excitation light beam 11. The confocal scanner 30 has a scanning unit that scans the excitation light beam 11 over the sample 6. The microscope 20 has an objective lens 21 that projects the image of the excitation light beam 11 scanned by the confocal scanner 30 onto the sample 6, and a position sensor 24 that detects the position of the well plate 60 on which the sample 6 is placed relative to the focal position of the objective lens 21. The camera 40 has an imaging surface positioned conjugate to the scanning unit and detects a fluorescence signal 12 from the sample 6 irradiated with the excitation light beam 11. The processing device 50 controls the operation of the microscope 20 and the camera 40. Here, the processing unit 50 transmits a control signal to the microscope 20 and camera 40 for imaging by the camera 40, while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 in a direction parallel to the optical axis of the excitation light beam 11, based on the position of the well plate 60 with respect to the focal position of the objective lens 21. In this way, the microscope system 1 according to this embodiment performs confocal imaging while displacing the focal position of the objective lens 21 in the sample 6 in a direction parallel to the optical axis of the excitation light beam 11. Therefore, even when the sample surface is tilted, the microscope system 1 can capture a clear image without partial brightness reduction in the field of view by confocal imaging. Furthermore, since the microscope system 1 does not require a complex configuration, it is not subject to limitations in mechanical design. Moreover, since the microscope system 1 acquires the image in a single imaging, it does not require a large load in terms of calculation and storage, nor does it significantly reduce the imaging speed. Therefore, even when the sample surface is tilted, it is possible to acquire a clear image without partial brightness reduction within the field of view, without limitations in the mechanical design or a significant decrease in imaging speed.

[0102] Furthermore, the confocal scanner 30 rotates a pinhole disk, which has multiple pinholes arranged on it, as its scanning unit, to scan the excitation light beam 11 irradiated from the light-emitting device 10 onto the sample 6. The objective lens 21 projects images of the multiple pinholes, created by the excitation light beam 11 scanned by the confocal scanner 30, onto the sample 6. The camera 40 is positioned so that its imaging surface is conjugate to the multiple pinholes. In this way, the microscope system 1 can quickly form a confocal image of the sample 6 on the light-receiving surface of the two-dimensional sensor 41 of the camera 40 by rapidly rotating the ML disk 33 and the Nipow disk 31. Therefore, it is possible to quickly acquire confocal images of all samples in a well plate 60, in which a large number of samples to be examined are arranged in a matrix, while relatively displacing it in a direction perpendicular to the optical axis of the excitation light beam 11 with respect to the microscope 20 and the confocal scanner 30.

[0103] Furthermore, the microscope 20 is equipped with a drive device 22 that can displace the objective lens 21 in a direction parallel to the optical axis of the excitation light beam 11. The processing device 50 controls the microscope 20 and the camera 40 so that the camera 40 takes images while the drive device 22 displaces the objective lens 21.

[0104] Furthermore, the processing device 50 may control the microscope 20 and camera 40 so as to perform imaging with the camera 40 while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 at a constant velocity. In this way, since the relative position of the focal point of the objective lens 21 with respect to the sample 6 is displaced at a constant velocity during imaging, it is possible to prevent the relative position of the sample 6 with respect to the objective lens 21 from moving in a direction perpendicular to the optical axis of the excitation light beam 11 during imaging, thereby preventing distortion of the captured image.

[0105] Furthermore, the processing unit 50 determines the displacement range of the relative position of the focal point of the objective lens 21 with respect to the sample 6 during imaging by the camera 40, based on the position of the container relative to the focal point of the objective lens 21. The processing unit 50 controls the microscope 20 and camera 40 to perform imaging by the camera 40 while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 by the determined displacement range. In this way, the processing unit 50 determines the displacement range of the relative position of the focal point of the objective lens 21 with respect to the sample 6 during imaging, based on the position of the container relative to the objective lens 21 of the microscope 20. Therefore, imaging can be performed by appropriately displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6, and a clear image can be obtained.

[0106] Here, as in Examples 2 and 3, the processing device 50 may detect a local tilt of the container, for example, the bottom surface 62, based on the position of the container relative to the focal position of the objective lens 21. Furthermore, the processing device 50 may determine the displacement range of the relative position of the focal point of the objective lens 21 with respect to the sample 6 during imaging by the camera 40, based on the detected local tilt of the container. By utilizing the local tilt of the container in this way, it is possible to determine the displacement range without excess or deficiency, and to perform imaging by displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6.

[0107] Furthermore, as in Example 3, the processing apparatus 50 may alternately perform the following processes: detecting a local tilt of, for example, the bottom surface 62 of the container and determining the corresponding displacement width, and photographing the sample 6 at the position where the local tilt was detected, thereby imaging each of the multiple samples placed on the container. In this way, by alternately determining the displacement width and photographing the sample 6 to image each of the multiple samples, it is possible to perform imaging with the correct displacement corresponding to the local tilt of the container at high speed by performing only one scan, without performing a pre-scan to determine the displacement width at each XY position to be photographed.

[0108] Furthermore, the processing unit 50 may control the microscope 20 and camera 40 so that imaging is performed by the camera 40 while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 by a displacement range specified by the user. In this way, the processing unit 50 performs imaging while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 by a displacement range specified by the user. Therefore, imaging can be performed by appropriately displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6, and a clear image can be obtained.

[0109] Furthermore, the processing unit 50 may determine a first displacement range, which is the displacement range of the relative position of the focal point of the objective lens 21 with respect to the sample 6, based on the position of the container relative to the focal point of the objective lens 21, while imaging is being performed by the camera 40. In addition, the processing unit 50 may obtain a second displacement range, which is the displacement range of the relative position of the focal point of the objective lens 21 with respect to the sample 6, as specified by the user. Then, the processing unit 50 may control the microscope 20 and camera 40 to perform imaging with the camera 40 while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 by the smaller of the first and second displacement ranges. In this way, the processing unit 50 may perform imaging while displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 by the smaller of the displacement range determined based on the position of the container relative to the objective lens 21 of the microscope 20, and the displacement range specified by the user. Therefore, it is possible to perform imaging by appropriately displacing the relative position of the focal point of the objective lens 21 with respect to the sample 6 and obtain a clear image.

[0110] The processing device 50 may determine the start and end positions, which are the relative positions of the focal point of the objective lens 21 relative to the sample 6, based on the dimensional information of the well plate 60 on which the sample 6 is placed. Furthermore, the processing device 50 may control the microscope 20 and camera 40 so that imaging is performed by the camera 40 while displacing the relative position of the focal point of the objective lens 21 relative to the sample 6 from the start position to the end position. In this way, the processing device 50 determines the start and end positions of the focal point of the objective lens 21 relative to the sample 6 when imaging is performed, based on the dimensional information of the container on which the sample 6 is placed. Therefore, imaging can be performed by appropriately displacing the relative position of the focal point of the objective lens 21 relative to the sample 6, and a clear image can be obtained.

[0111] Furthermore, Figure 7 illustrates an example in which, in step S5, the control unit 51 compares the length of the movement range calculated in step S4 with the length of the movement range set by the user in step S1 and selects the shorter of the two movement ranges. However, the configuration is not limited to this. For example, the control unit 51 may select a range that is included in both the movement range calculated in step S4 and the movement range set by the user in step S1, or a range that is included in at least one of them.

[0112] In the above embodiment, an example was described in which the ML disk 33 is mechanically connected to the Nipow disk 31 by a member 32, but the ML disk 33 and member 32 do not necessarily have to be provided. Also, in the above embodiment, a configuration was described in which the microscope system 1 has a confocal imaging system as the imaging system and one camera 40 for imaging, but the system is not limited to this configuration. For example, the microscope system 1 may have an epi-illumination optical system, or multiple cameras 40 may be used to simultaneously image multiple fluorescence wavelengths.

[0113] Furthermore, in the above embodiment, an example was described in which the objective lens 21 is displaced by the drive device 22 without the sample 6 being displaced in order to displace the relative position of the focal point of the objective lens 21 with respect to the sample 6, but the configuration is not limited to this. For example, the well plate 60, or both the objective lens 21 and the well plate 60, may be moved in a direction parallel to the optical axis of the excitation light beam 11.

[0114] Furthermore, although the above embodiment describes an example in which a well plate 60 is used as the container on which the sample 6 is placed, other sample containers such as a Petri dish, cell culture flask, slide glass, or coverslip chamber may be used instead. Also, while imaging of the sample 6 is being performed, it is preferable to set the drive pattern so that the drive state of the drive device 22 for the focal plane of the objective lens 21 is constant speed, but this is not required. Also, while imaging of the sample 6 is being performed, the direction of displacement of the focal plane of the objective lens 21 may be either in the direction that decreases the distance between the sample 6 and the objective lens 21 or in the direction that increases it.

[0115] Furthermore, the explanation with reference to Figure 6 describes an example where the displacement of the focal plane of the objective lens 21 occurs only once in one direction while imaging of the sample 6 is being performed, but it is not limited to this. For example, the focal plane may be displaced back and forth or displaced multiple times. Also, Figure 1 shows an example where the microscope 20 is an inverted microscope, but the microscope 20 may be in other configurations such as an upright microscope.

[0116] Furthermore, the processing unit 50 may determine the position of the lower end of the displacement of the objective lens 21 based on the design information of the well plate 60. Alternatively, the processing unit 50 may determine the position of the bottom surface 62 of the well plate 60 by detecting it with the position sensor 24.

[0117] This disclosure is not limited to the embodiments described above. For example, multiple blocks shown in the block diagram may be combined, or one block may be divided. Multiple steps shown in the flowchart may be performed in parallel or in a different order, depending on the processing capacity of the device performing each step, or as necessary, instead of being performed in chronological order as described. Other modifications are possible without departing from the spirit of this disclosure. [Explanation of symbols]

[0118] 1. Microscope System 6 samples 10 Light-emitting device 11 Excitation beam 12 Fluorescence Signals 20 Microscopes 21 Objective lens 22 Drive unit 23 Relay Lens 24 Position Sensors 241 Light source 242 Spectroscopic Mirror 243 Spectroscopic Mirror 244 Light Sensor 30 Confocal Scanners 31. Pinhole array disk (Nipow disk) 32 components 33 Microlens array disk 34 Dichroic Mirror 35 Relay Lens 36 Bandpass Filter 37 Relay Lens 39. Rotational center axis 40 Cameras 41 2D sensors 50 Processing Units 51 Control Unit 52 Storage section 53 Input / output section 60-well plate 61 frames 62 Bottom 63 Samples to be imaged 64 Cell culture surface 65 Imaging Plane 66 samples to be imaged 68 wells 71 imaging zone width

Claims

1. A microscope system for acquiring an image of a sample to be observed, A light-emitting device that emits illumination light, A confocal scanner having a scanning unit that scans the illumination light over the sample, A microscope comprising: an objective lens that projects an image of the illumination light scanned by the confocal scanner onto the sample; and a position sensor that detects the position of the container on which the sample is placed relative to the focal position of the objective lens; An imaging device, wherein the imaging surface is positioned conjugate to the scanning unit, detects the light to be measured from the sample irradiated with the aforementioned illumination light. A processing unit that controls the operation of the microscope and the imaging device, Equipped with, The aforementioned processing apparatus is Based on the local tilt of the container, the displacement range of the relative position of the focal point of the objective lens with respect to the sample is determined. Based on the position of the container relative to the focal position of the objective lens, a control signal is transmitted to the microscope and the imaging device to continue exposure and imaging while changing the relative position of the focal point of the objective lens with respect to the sample by the determined displacement amount in a direction parallel to the optical axis of the illumination light. Microscope system.

2. The confocal scanner, as the scanning unit, rotates a pinhole disk on which a plurality of pinholes are arranged, and scans the illumination light irradiated from the light-emitting device onto the sample. The objective lens projects images of the multiple pinholes, scanned by the illumination light from the confocal scanner, onto the sample. The imaging device is configured such that the imaging surface is positioned conjugate to the plurality of pinholes. The microscope system according to claim 1.

3. The microscope further comprises a drive device capable of displacing the objective lens in a direction parallel to the optical axis of the illumination light, The processing apparatus controls the microscope and the imaging device so that the drive device displaces the objective lens while the imaging device continues exposure and imaging. The microscope system according to claim 1 or 2.

4. The microscope system according to any one of claims 1 to 3, wherein the processing apparatus controls the microscope and the imaging device so as to continuously perform exposure and imaging in the imaging device while changing the relative position of the focal point of the objective lens with respect to the sample at a constant speed.

5. The aforementioned processing apparatus is Based on the position of the container relative to the focal position of the objective lens, the local tilt of the container is detected. Based on the local tilt of the container detected, the displacement range is determined. The microscope system according to any one of claims 1 to 4.

6. The microscope system according to claim 5, wherein the processing apparatus alternately performs the following steps: detecting a local tilt of the container and determining the displacement range of the relative position of the focal point of the objective lens with respect to the sample based on the local tilt; and controlling the microscope and the imaging device so as to continue exposure and imaging in the imaging device while changing the relative position of the focal point of the objective lens by the determined displacement range, thereby imaging each of the plurality of samples placed in the container.

7. The microscope system according to any one of claims 1 to 4, wherein the processing apparatus controls the microscope and the imaging device so as to continue exposure and imaging in the imaging device while further changing the relative position of the focal point of the objective lens with respect to the sample by a displacement range specified by the user.

8. An imaging method for a microscope system comprising a light-emitting device, a confocal scanner, a microscope, an imaging device, and a processing device for acquiring an image of a sample to be observed, The light-emitting device emits illumination light, The scanning unit of the confocal scanner performs the step of scanning the illumination light with the sample, The objective lens of the microscope projects the image of the illumination light scanned by the confocal scanner onto the sample. The microscope's position sensor includes the step of detecting the position of the container on which the sample is placed with respect to the focal position of the objective lens, The imaging device, whose imaging surface is positioned conjugate to the scanning unit, performs the step of detecting the light to be measured from the sample irradiated with the illumination light, It has, The aforementioned processing apparatus is Based on the local tilt of the container, the displacement range of the relative position of the focal point of the objective lens with respect to the sample is determined. Based on the position of the container relative to the focal position of the objective lens, a control signal is transmitted to the microscope and the imaging device to continue exposure and imaging while changing the relative position of the focal point of the objective lens with respect to the sample by the determined displacement amount in a direction parallel to the optical axis of the illumination light. Imaging methods for microscope systems.

9. A program that causes a computer to operate as a processing unit included in the microscope system described in any one of claims 1 to 7.

Images