Microscope, method, and program

Abstract

To provide a new method for aligning image positions.SOLUTION: The present invention comprises a first observation optical system for acquiring a macro-image, a second observation optical system for acquiring a micro-image different from the first observation optical system, and a control unit, thus the present invention acquiring position information indicating the position of a region of interest in the macro-image, controlling the positional relationship between a stage having a first sample placed thereupon and the second observation optical system on the basis of the positional information, and acquiring a micro-image of the region of interest by using the second observation optical system. By using the first observation optical system, the present invention acquires a second macro-image of a second sample different from the first sample, calculates a macro deviation amount representing the amount of positional deviation between the macro-image and the second macro-image, acquires second position information indicating the position of a second region of interest corresponding to the first position information in the second macro-image on the basis of the macro deviation amount, and controls a stage having the second sample placed thereupon, thereby executing acquisition of a second micro-image of the second region of interest by using the second observation optical system.SELECTED DRAWING: Figure 7A

Description

[Technical Field]

[0001] The present invention relates to a microscope, a method, and a program. [Background technology]

[0002] BACKGROUND ART Conventionally, a method for performing alignment of images acquired by a microscope is known (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6643072 Summary of the Invention [Problem to be solved by the invention]

[0004] The techniques of this disclosure provide a novel method for image registration. [Means for solving the problem]

[0005] One embodiment of the present invention is a microscope comprising a first observation optical system that acquires a macro image, a second observation optical system that acquires a micro image different from the first observation optical system, and a control unit, wherein the control unit executes the following processes: acquiring a first macro image of a first specimen using the first observation optical system; acquiring first position information that indicates the position of a first region of interest within the first macro image; controlling the positional relationship between a stage on which the first specimen is placed and the second observation optical system based on the first position information, and acquiring a first micro image of the first region of interest using the second observation optical system; acquiring a second macro image of a second specimen different from the first specimen using the first observation optical system; calculating a macro shift amount that is the amount of positional shift between the first macro image and the second macro image based on the macro shift amount; acquiring second position information that indicates the position of a second region of interest in the second macro image that corresponds to the first position information, and controlling the positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the second position information, and acquiring a second micro image of the second region of interest using the second observation optical system.

[0006] Furthermore, one embodiment of the present invention is a method for a microscope including a first observation optical system for acquiring macro images, a second observation optical system for acquiring micro images different from the first observation optical system, and a control unit, the method being executed by the control unit, the method including: a process of acquiring a first macro image of a first specimen using the first observation optical system; an acquisition process of acquiring first position information indicating the position of a first region of interest within the first macro image; a process of controlling the positional relationship between a stage on which the first specimen is placed and the second observation optical system based on the first position information, and acquiring a first micro image of the first region of interest using the second observation optical system; a process of acquiring a second macro image of a second specimen different from the first specimen using the first observation optical system; a process of calculating a macro shift amount, which is the amount of positional shift between the first macro image and the second macro image, based on the macro shift amount; a process of acquiring second position information indicating the position of a second region of interest in the second macro image that corresponds to the first position information, and a process of controlling the positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the second position information, and acquiring a second micro image of the second region of interest using the second observation optical system.

[0007] a control unit; and a program for causing the control unit to execute the following operations: acquiring a first macro-image of a first specimen using the first observation optical system; acquiring first position information indicating the position of a first region of interest within the first macro-image; controlling the positional relationship between a stage on which the first specimen is placed and the second observation optical system based on the first position information, and acquiring a first micro-image of the first region of interest using the second observation optical system; acquiring a second macro-image of a second specimen, different from the first specimen, using the first observation optical system; calculating a macro-shift amount, which is the amount of positional shift between the first macro-image and the second macro-image, based on the macro-shift amount; acquiring second position information indicating the position of a second region of interest in the second macro-image that corresponds to the first position information, based on the macro-shift amount; and controlling the positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the second position information, and acquiring a second micro-image of the second region of interest using the second observation optical system. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram showing the overall configuration of a system according to an embodiment, illustrating a situation in which an optical system P1 is used. [Figure 2] 1 is a diagram showing the overall configuration of a system according to an embodiment, illustrating a situation in which an optical system P2 is used. FIG. 2 is a diagram showing the hardware configuration of an information processing device. [Figure 3] 1A is a block diagram showing the overall configuration of a system, FIG. 1B is a block diagram showing the hardware configuration of an information processing device, and FIG. 1C is a block diagram showing the functional configuration (software configuration) of the information processing device. [Figure 4] 1A and 1B are diagrams illustrating the configuration of biological tissues and specimens. [Figure 5] FIG. 1 is a diagram showing the configuration of a specimen and a region of interest. [Figure 6] FIG. 1 is a diagram showing the configuration of biological tissue and micro-images. [Figure 7A]1 is a flowchart of a process executed in an embodiment. [Figure 7B] 1 is a flowchart of a process executed in an embodiment. [Figure 7C] 1 is a flowchart of a process executed in an embodiment. [Figure 8] FIG. 1 is a diagram illustrating an outline of alignment performed on a macro image and a micro image. [Figure 9] 1 is an example of a screen displayed on an output device, showing a situation in which one micro image is displayed. [Figure 10] 1 is an example of a screen displayed on an output device, showing two micro images superimposed on each other. [Figure 11] This is an example of a screen displayed on an output device, showing two micro images displayed side by side. [Figure 12] 1 is an example of a screen displayed on an output device, showing a situation in which a plurality of micro images are displayed side by side. [Figure 13] 1 is an example of a screen displayed on an output device, showing a situation in which a plurality of micro images are displayed side by side. [Figure 14] 1 is an example of a screen displayed on an output device, showing a situation in which a plurality of micro images are displayed side by side. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, the present invention will be described based on one embodiment thereof with reference to the drawings.

[0010] 〔composition〕 1 and 2 show the configuration of a system 1 according to one embodiment of the present invention. The system 1 includes a microscope device 30 incorporating an information processing device 10, an output device 40, and an input device 50. The output device 40, the input device 50, and the microscope device 30 are connected via communication means such as a bus (not shown) so as to be able to send and receive data to and from each other.

[0011] The output device 40 acquires images captured by the microscope device 30 and processes the images for display. The output device 40 is an interface that outputs various types of information, and is, for example, a screen display device (liquid crystal display (LCD) monitor), a printer, etc.

[0012] The input device 50 is an interface that accepts input of information, and is, for example, a keyboard, a mouse, a touch panel, a card reader, a voice input device (such as a microphone), a voice recognition device, or the like.

[0013] The microscope device 30 is a device that uses a microscope to image (also referred to as photographing) biological tissue BT (FIG. 4) as a subject. One example of the subject is biological tissue such as a cell.

[0014] As shown in FIG. 3, the microscope device 30 includes an information processing device 10 and an optical device 20.

[0015] The optical device 20 includes an objective lens 31 (which includes multiple objective lenses with different magnifications and is switchable), a second objective lens 34, a stage 32, a light source 35 that illuminates the specimen, reflecting mirrors 36 and 37, and an imaging unit 39 (including a CCD sensor, a CMOS sensor, etc.), and is capable of photographing the specimen placed on the stage 32.

[0016] The stage 32 is a member with a flat upper surface, and can move in the directions of the X and Y axes in Figures 1 and 2. The X and Y axes are parallel to the upper surface of the stage 32 and are perpendicular to each other.

[0017] The optical device 20 includes two optical systems P1 and P2. As shown in Fig. 1, the optical system P1 has a reduction lens (not shown) and is used to form a macro image of the entire specimen on the imaging unit 39. As shown by the dashed dotted line in Fig. 1, the optical system P1 guides light from the specimen placed on the stage 32 to the imaging unit 39 via the reduction lens (not shown) and a reflecting mirror 36.

[0018] The optical system P2 is used to magnify a portion of the specimen and form a micro-image on the imaging unit 39. As shown by the dashed line in Figure 2, the optical system P2 guides light from the specimen placed on the stage 32 to the imaging unit 39 via the objective lens 31, the reflecting mirror 37, and the second objective lens 34 in this order.

[0019] The magnification of the optical system P2 varies depending on the magnification of the objective lens 31 used, and is, for example, 4x, 10x, 20x, or 40x.

[0020] The optical device 20 is equipped with two optical systems P1 and P2 to acquire macro and micro images, but the observable range may be changed by switching the lenses that make up a single optical system.

[0021] The macro image is acquired by capturing an optical image of the entire specimen formed by the optical system P1 with the imaging unit 39, and by performing a single image capture. In other words, the macro image is not formed by combining multiple images of the specimen captured for each predetermined region (each divided region), but is acquired as a whole specimen at once.

[0022] Therefore, the capacity of the macro image is approximately 1 / the number of images synthesized (for example, 40 to 100) compared to when multiple images are synthesized, and since image synthesis is not required, processing in the information processing device 10 (including image display on the output device 40), which will be described later, can be performed at high speed.

[0023] The micro image is captured as an image optically enlarged more than the macro image by the objective lens 31 and the second objective lens 34. Furthermore, the micro image is acquired with a higher resolution than the macro image.

[0024] The imaging unit 39 is switchable between a low-resolution mode and a high-resolution mode, and the resolution of the macro image is 10 to 80 μm / px, depending on which mode is selected. The resolution of the micro image is 0.1 to 4 μm / px, depending on the combination of the selected mode of the imaging unit 39 and the magnification (observable range) of the objective lens used.

[0025] The time required from step S7 to step S15 (from acquiring macro images of m target books to acquiring micro images of m target books) shown in Figures 7A-C described below is at least 20 seconds if there is one area of ​​interest (one micro image).

[0026] The information processing device 10 has a function of controlling the optical device 20 and acquiring an image.

[0027] 3(b) shows an example of hardware used to realize the information processing device 10. As shown in the figure, the information processing device 10 includes a processor 101, a main memory device 102, an auxiliary memory device 103, and a communication device 106. These are connected to each other so as to be able to communicate with each other via communication means such as a bus (not shown).

[0028] It should be noted that the information processing device 10 does not necessarily need to have all of its configuration realized by hardware, and all or part of its configuration may be realized by virtual resources such as a cloud server of a cloud system.

[0029] The processor 101 is configured using a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc. The processor 101 reads and executes a program stored in a main memory device 102, thereby realizing the functions of the information processing device 10.

[0030] The main memory device 102 is a device that stores programs and data, and is a ROM (Read Only Memory), a RAM (Random Access Memory), a non-volatile semiconductor memory (NVRAM (Non-Volatile RAM)), etc. The auxiliary memory device 103 is various non-volatile memories (NVRAM: Non-volatile memory) such as an SSD (Solid State Drive) or an SD memory card, a hard disk drive, an optical storage device (CD (Compact Disc), DVD (Digital Versatile Disc), etc.), a storage area of ​​a cloud server, etc.

[0031] The communication device 106 is a wired or wireless communication interface that realizes communication with other devices, such as a NIC (Network Interface Card), a wireless communication module, a USB (Universal Serial Interface) module, or a serial communication module.

[0032] The information processing device 10 communicates information with the optical device 20, the output device 40, and the input device 50 via the communication device 106, and executes various controls.

[0033] [Functional configuration] 3(c) shows the main functional configuration of the information processing device 10. As shown in the figure, the information processing device 10 includes a storage area 110 and a management unit 120.

[0034] The memory area 110 is formed in the main memory device 102 or the auxiliary memory device 103 of the information processing device 10. Various information is stored in the memory area 110. Specific examples include images acquired by the system 1, image coordinates, the amount of misalignment between images, and the position of a region of interest (all of which will be described in detail later).

[0035] In addition to the above functions, the information processing device 10 also has functions of an operating system, a file system, a device driver, a DBMS (DataBase Management System), and the like.

[0036] The management unit 120 performs processes such as image acquisition and management that are executed by the information processing device 10. The functions of the management unit 120 are realized by the processor 101 of the information processing device 10 reading and executing a program stored in the main storage device 102 or the auxiliary storage device 103 of the information processing device 10.

[0037] The management unit 120 communicates with the optical device 20, the output device 40, and the input device 50 via the communication device 106, and executes various controls over these devices.

[0038] [Processing details] The details of the processing executed in the system 1 will be described below with reference to the flowcharts of FIGS. 7A-7C.

[0039] A program stored in the main memory device 102 of the information processing device 10 is started, and the management unit 120 executes the processing of the system 1 as follows. Note that, hereinafter, the processing executed by the management unit 120 of the server 10 may be simply described as being executed by the "information processing device 10."

[0040] In step S1, a user (e.g., a pathologist) places a slide glass holding a first specimen SP1 on the stage 32. As shown in FIG. 4, a specimen is created by embedding biological tissue BT in paraffin RF to form a block, slicing this block, and staining each of the slices. The process of creating a specimen from biological tissue BT is not limited to the above; for example, a method of freezing the biological tissue BT and slicing this block may be selected. Therefore, typically, multiple slide glasses are prepared according to the number of stains. Here, biological tissue BT held on one slide glass is referred to as one specimen. In other words, specimens are prepared according to the number of stains.

[0041] Staining is performed using at least one of the following methods: hematoxylin-eosin staining (HE staining) and immunostaining, which uses an antigen-antibody reaction to detect a target substance to obtain information on the distribution of specific substances that make up biological tissue. In addition to these staining methods, other methods such as FISH (Fluorescence in situ Hybridization), DISH (Dual Color in situ Hybridization), CISH (Chromogenic in situ hybridization), and SISH (Silver-Enhanced in situ Hybridization) can also be used.

[0042] In the following description, the specimens held on the respective slide glasses will be given individual reference symbols SP1, SP2, SP3, etc. as shown in FIG.

[0043] Each slide holds a sliced ​​and stained specimen SP1, SP2, etc., as described above. The shape of the specimen on each slide is approximately the same, but not necessarily perfectly the same. Furthermore, the position and orientation (angle) of the specimen on each slide usually differ from one slide to another, influenced by the shape of the biological tissue BT and the slicing and staining processes. Furthermore, when placed on the stage 32, there may be a misalignment in the installation position of each slide. Therefore, there may be a misalignment in the position of the specimen (which can also be considered as coordinates indicating the position) in each image acquired by the optical device 20.

[0044] In step S2, the management unit 120 controls the imaging unit 39 via the optical system P1 to capture the first image of the entire specimen SP1. The specimen SP1 is captured in one shot, that is, one image is formed by capturing the image once. As a result of the capture, a macro image MA1 is acquired and stored in the memory area 110.

[0045] In step S3, a region of interest (also referred to as a position of interest) is set in the specimen SP1 or the macro image MA1. The region of interest is set by the user specifying a position on the macro image MA1 displayed on the output device 40. The region of interest is set by the user operating the input device 50. Regions of interest are set in n locations (n ​​is an integer equal to or greater than 1) desired by the user, and there is no restriction on the number of regions of interest that can be set.

[0046] In this example, it is assumed that the user sets three regions of interest A11-A13 (FIG. 5). The management unit 120 stores in the memory area 110 coordinates (X and Y coordinates) indicating the positions of the regions of interest A11-A13 in the specimen SP1 or the macro image MA1 set by the user, using the macro image MA1 as a reference.

[0047] Here, the regions of interest A11-A13 are not specific to specimen SP1, but are also regions set in each specimen SP2, SP3, ... They may also be considered to be regions set in the biological tissue BT. As described above, each specimen is cut out from the biological tissue BT, and therefore, in each specimen, the regions of interest A11-A13 are set in the same locations as in specimen SP1. As will be described in detail later, a user can acquire images of, for example, the region of interest A11 in each specimen via the microscope device 30 and compare the differences between the images, thereby making it possible to compare common regions of specimens stained using different staining techniques, which can be used for diagnosis, etc.

[0048] In step S4, the imaging unit 39 images each of the regions of interest A11-A13 of the specimen SP1 once via the optical system P2, thereby acquiring micro images MC11-MC13 and storing them in the memory area 110 (FIG. 5). At this time, the magnification of the objective lens used is also stored in the memory area 110. At this time, the management unit 120 controls the positional relationship with the optical system P2 by moving the stage 32 on which the specimen SP1 is placed within the XY plane based on the coordinates indicating the positions of the regions of interest A11-A13 stored in the memory area 110, focuses the objective lens 31 on the specimen SP1, and uses the imaging unit 39 to image the regions of interest A11-A13.

[0049] When storing the micro images MC11-MC13 in the storage area 110, the management unit 120 adds each focus position (Z coordinate) as position information of the attention areas A11-13 and stores the information in the storage area 110.

[0050] The entire optical system P2 may be configured to be movable in the X and Y directions.

[0051] Although an example has been described in which micro images MC11 to MC13 are acquired for each of the attention areas A11 to A13 after all of the attention areas A11 to A13 have been registered, the present invention is not limited to this.

[0052] For example, when setting the attention area A11, a micro image MC11 is acquired and stored in the memory area 110, and then when setting the attention area A12, a micro image MC12 is acquired and stored in the memory area 110, and then when setting the attention area A13, a micro image MC13 is acquired and stored in the memory area 110. (Combination of steps S3 and S4)

[0053] The management unit 120 displays the macro image MA1 read from the storage area 110 on the output device 40 and moves the stage carrying the specimen SP1 a predetermined amount in the Y direction to align it with the optical path of the optical system P2. When the user operates the input device 50 to specify a predetermined position on the macro image MA1, the management unit 120 controls the positional relationship with the optical system P2 by moving the stage carrying the specimen SP1 within the XY plane based on the coordinates indicating the specified position, thereby adjusting the focus of the objective lens 31 on the specimen SP1. Also, as shown in FIG. 9, the management unit 120 acquires a micro image of the area of ​​the specimen SP1 corresponding to the specified position via the optical system P2 using the imaging unit 39 and displays it on the output device 40. The user checks the micro image displayed on the output device 40, determines whether it is appropriate, and, if appropriate, operates the input device 50 to register the specified position (XY position, focus position). The magnification of the objective lens used is also registered. The management unit 120 sets the coordinates (XY coordinates, Z coordinate indicating the focus position) indicating the registered designated position as the attention area A11, and also saves the micro image displayed on the output device 40 as a micro image MC11 in the storage area 110. Note that the setting of the attention area A12, the setting of the attention area A13, etc. is also performed in a similar manner.

[0054] The micro images MC11-MC13 are images obtained by capturing the attention areas A11-A13, respectively. The micro images MC11-MC13 are images obtained at a higher magnification than the macro images, and therefore are obtained as images enlarged more than the macro image MA1. The micro images MC11-MC13 are also obtained at a higher resolution than the macro image MA1.

[0055] After the micro-images MC11-MC13 are acquired, the slide glass holding the specimen SP1 is removed from the stage 32 (S5), and a second slide glass holding the specimen SP2 is placed on the stage 32 (S6).

[0056] 7A, the process for the mth sample (m is an integer equal to or greater than 2) is generally described. In the following description of steps S6 to S25A, for ease of understanding, a specific example will be described in which m=2, i.e., the process for the second sample SP2, is performed.

[0057] In step S7, the imaging unit 39, controlled by the management unit 120, images the specimen SP2 via the optical system P1. The imaging of the specimen SP2 is performed in one shot, that is, by forming one image by one shooting. As a result of the imaging, a macro image MA2 is acquired and stored in the memory area 110.

[0058] In step S8, the amount of deviation between the macro images MA1 and MA2 stored in the storage area 110 is acquired. The management unit 120 extracts, from each of the macro images MA1 and MA2, a plurality of feature points (coordinates indicating positions within the image) that are predicted to be common to the specimens SP1 and SP2, and measures the deviation between the plurality of feature points (coordinates indicating positions within the image) of the macro image MA1 and the plurality of feature points (coordinates indicating positions within the image) of the macro image MA2 that are predicted to correspond thereto, thereby acquiring the amount of deviation, and stores the acquired amount of deviation in the storage area 110. Note that the method for calculating the amount of deviation of the macro image MA2 relative to the macro image MA1 is not limited to this, and the amount of deviation may be calculated using, for example, the outline of the specimen displayed in the macro image or the area of ​​the specimen. The management unit 120 corrects the position of the macro image MA2 relative to the macro image MA1 using the acquired amount of deviation, thereby making it possible to align the position of any region of interest in the macro image MA2 with the same region of interest in the macro image MA1.

[0059] To obtain the amount of deviation, the coordinate deviation (shift) in each direction of the X-axis and Y-axis between the images, the deviation in image rotation (angle), and the deviation in enlargement / reduction (magnification) are obtained as shown in Fig. 10. As a result, the amount of deviation between the images is grasped using three matrices: the X-axis and Y-axis coordinate translation matrix between the two images, the rotation matrix, and the enlargement / reduction matrix. Note that Fig. 10 shows micro images MC11 and MC12, but the amount of deviation, consisting of the deviation (shift) in the X-axis and Y-axis directions and the rotation (angle) deviation between specimen SP1 displayed in macro image MA1 and SP2 displayed in macro image MA2, is also grasped in the macro images.

[0060] Next, the management unit 120 calculates IoU (Intersection over Union) based on the amount of deviation calculated in step S8 (S9). More specifically, the management unit 120 binarizes the macro images MA1 and MA2, and further quantifies the overlapping area of ​​the specimens SP1 and S2 displayed on the macro images MA1 and MA2 by calculating IoU. Generally, the closer the IoU value is to 1 from 0, the higher the degree of alignment. Note that the method for calculating the degree of alignment is not limited to IoU, and it may also be calculated, for example, by the overlapping of the contours (lines) of the specimens SP1 and S2 displayed on the macro images MA1 and MA2.

[0061] Next, the management unit 120 compares the IoU with a predetermined reference value (S9A).

[0062] If the IoU is equal to or less than the reference value, the management unit 120 switches to manual correction mode (FIG. 8, "macro alignment"), reads out the macro images MA1 and MA2 from the storage area 110, and displays them on the output device 40 (S10).

[0063] Note that the user may be allowed to operate the input device 40 to switch to the manual correction mode regardless of whether the IoU is equal to or less than the reference value. Furthermore, after step S7, steps S8 and S9 may not be performed (omitted), and the user may switch to the manual correction mode by operating the input device 40. In other words, when the management unit 120 receives an instruction from the input device 40, it switches to the manual correction mode.

[0064] The user sets the transparency (greater than 0% and less than 100%) of the macro images MA1 and MA2 based on input to the input device 50. It is also possible to change (increase) the transparency of only one of the macro images MA1 and MA2. In this case, the macro image with the changed (increased) transparency is arranged on top of the macro image with the unchanged transparency. This transparency needs to be set so that the overlap between the specimen SP1 displayed in the macro image MA1 and the specimen SP2 displayed in the macro image MA2 can be clearly seen. Therefore, the transparency is set according to the state of the specimen.

[0065] The user manually aligns the macro images MA1 and MA2 via the input device 50 while viewing the macro images MA1 and MA2, which have been changed to a predetermined transparency (for example, 50%) and are displayed on the output device 40 (S11). At this time, the management unit 120 changes the relative position, angle, and enlargement / reduction (magnification) of the macro images MA1 and MA2 based on the input to the input device 50, and displays them on the output device 40. While viewing the display on the output device 40, the user performs the operation until the positions of the specimens SP1 and SP2 shown in the macro images MA1 and MA2 are sufficiently aligned.

[0066] After the manual alignment is completed, the management unit 120 recalculates the amount of deviation between the aligned macro images MA1 and MA2, overwrites the amount of deviation acquired in step S8, and stores the recalculated amount of deviation in the storage area 110 (S12).

[0067] If the IoU is greater than or equal to the reference value in step S9A, or after completion of step S12, the management unit 120 acquires coordinates indicating the position of the areas of interest A11-A13 relative to the macro image MA2 or the specimen SP2 based on the amount of deviation between the macro images MA1 and MA2, and stores them in the memory area 110.

[0068] In step S14, a second macro image MA2 displaying the positions of the attention areas A11-A13 acquired in step S13 is displayed on the output device 40. Note that step S14 may be omitted. In step S15, the management unit 120 acquires micro images of the specimen SP2. More specifically, the image capturing unit 39 captures images of the regions of interest A11-A13 in the specimen SP2 via the optical system P2, and stores the images in the storage area 110. Each time step S15 is performed, one micro image is acquired for the Nth location (N is an integer from 1 to n) among the regions of interest A11-A13 (n=3, so three locations). Finally, three micro images MC21-MC23 corresponding to the regions of interest A11-A13, respectively, are acquired each time the process performs step S15.

[0069] For ease of understanding, the following description of steps S14 to S24 uses a specific example in which a micro image MC21 at the first location (N=1) is captured.

[0070] Using this specific example, in the processing of step S15, the management unit 120 controls the positional relationship with the optical system P2 by moving the stage 32 on which the specimen SP2 is placed within the XY plane, focuses the objective lens 31 on the specimen SP2, and captures an image of the area of ​​interest A11 in the specimen SP2 using the imaging unit 39. By capturing the image of the area of ​​interest A11, a micro image MC21 is acquired and stored in the memory area 110.

[0071] In step S16, the management unit 120 calculates the overlapping area between the micro-images MC11 and MC21 stored in the storage area 110 using IoU (S16). The calculation content is the same as in step S9. For example, when calculating IoU as the degree of alignment between the micro-images MC11 and MC21, the management unit 120 binarizes the micro-images MC11 and MC21, and further digitizes the overlapping areas of the specimens SP1 and SP2 displayed in the micro-images MC11 and MC21 by calculating IoU. Note that in the case of the micro-images MC12 and MC22 executed in a process where N=2, and in the case of the micro-images MC13 and MC23 executed in a process where N=3, IoU is calculated using the same process as for the micro-images MC11 and MC21.

[0072] Next, the management unit 120 compares the IoU with a predetermined reference value (S16A). Note that the reference value used in step S16A does not necessarily match the reference value used in step S9A. The reference value in step S16A is preferably a stricter value (a value closer to 1) than the reference value used in step S9A, i.e., the allowable deviation is smaller. The reference value may be changed depending on the magnification of the objective lens 31 used to acquire the micro image. The higher the magnification, the stricter the reference value.

[0073] If the IoU is equal to or less than the reference value, the amount of deviation between the micro images MC11 and MC21 stored in the storage area 110 is calculated (S17). The management unit 120 extracts a plurality of feature points (coordinates indicating positions within the image) that are predicted to be common to the micro images MC1 and MC2, measures the deviation between the plurality of feature points (coordinates indicating positions within the image) of the micro image MC1 and the plurality of feature points (coordinates indicating positions within the image) of the micro image MC2 that are predicted to correspond thereto, thereby obtaining the amount of deviation, and stores the obtained amount of deviation in the storage area 110. The method for calculating the amount of deviation between the micro images is the same as in step S8. The management unit 120 corrects the coordinates of the position indicating the region of interest A11 in the specimen SP2 based on the amount of deviation, and controls the positional relationship with the optical system P2 by moving the stage 32 on which the specimen SP2 is placed within the XY plane.

[0074] Furthermore, the management unit 120 focuses the objective lens 31 on the specimen SP2, and uses the imaging unit 39 to image the area of ​​interest A11 in the specimen SP2 corrected in step S17, and re-acquires the micro image MC21 and stores it in the memory area 110 (S18).

[0075] In the next step S19, the management unit 120 calculates IoU as the degree of alignment between the micro image MC11 and the micro image MC21 acquired in step S18. The calculation method is the same as in step S9. The management unit 120 binarizes the micro image MC11 and the micro image MC21 acquired in step S18, and further digitizes the overlapping area of ​​the specimens SP1 and SP2 displayed in the micro images MC11 and MC21 by calculating IoU.

[0076] Next, management unit 120 compares IoU with a predetermined reference value (S19A). Note that the reference value used in step S19A does not necessarily match the reference value used in steps S9A and S16A. It is preferable that the reference value in step S19A be stricter than the reference value used in step S9A, i.e., the allowable deviation amount is smaller.

[0077] After step S16A, step S20 may be performed without performing (omitting) steps S17 to S19.

[0078] If IoU is equal to or less than the reference value, manual alignment is performed in step S20 and thereafter (S20). The management unit 120 switches to manual correction mode, reads out the micro image MC11 from the storage area 110, and outputs it to the output device 40. The management unit 120 also captures an image of the region of interest A11 of the specimen SP2 placed on the stage 32 with the imaging unit 30 via the optical system P2 to obtain a micro image MC21, which is then displayed on the output device 40 in real time. At this time, as shown in FIG. 11, the micro images MC11 and MC21 are displayed side by side on the output device 40. Furthermore, since the orientation (angle) of the specimen on the glass slide differs for each glass slide, the management unit 120 tilts the micro image MC11 by the amount of angle deviation so as to correspond to the orientation (angle) of the micro image MC12, as shown in FIG. 6, so that the user can easily compare the micro images displayed on the output device 40.

[0079] While visually viewing the micro images MC11 and MC21 on the output device 40, the user moves the micro image MC21 displayed in real time on the output device 40 by a predetermined amount using the input device 50, for example, by dragging, so that the area of ​​interest A11 shown in the micro image MC11 and the area of ​​interest A11 shown in the micro image MC21 become the same (S21). When the micro image MC21 is dragged by a predetermined amount using the input device 50, the management unit 120 moves the stage 32 in the XY plane in conjunction with the micro image MC21 (the amount of movement of the image is converted into the amount of movement of the stage). As a result, the positional relationship between the stage 32 (specimen SP2) and the optical system P2 changes, and therefore the area of ​​interest A11 in the specimen SP2 imaged by the imaging unit 30 via the optical system P2 changes, i.e., the micro image MC21 changes. Furthermore, the user changes (rotates) the micro image MC11 by a predetermined angle by dragging it with the input device 50 so that the orientation of the attention area A11 shown in the micro image MC11 and the orientation of the attention area A11 shown in the changed micro image MC21 become the same. When the micro image MC11 is dragged by a predetermined angle with the input device 50, the management unit 120 changes the angle of the micro image MC11 in conjunction with this. The user performs the operation until the attention area A11 shown in the micro image MC11 and the attention area A11 shown in the micro image MC21 become the same. Note that the stage 32 may be provided with not only a movement mechanism within the XY plane but also a rotation mechanism, so that when the micro image MC21 is dragged by a predetermined amount with the input device 50, the management unit 120 moves and rotates the stage 32 within the XY plane in conjunction with this.

[0080] After the user manually aligns the micro image MC11 in step S21, the management unit 120 calculates the amount of deviation between the attention area A11 shown in the micro image MC11 and the attention area A11 shown in the new micro image MC21 (S22).

[0081] Furthermore, the management unit 120 acquires the micro image MC21 again and stores it in the storage area 110 (S23). In this way, the micro image MC21 of the new area of ​​interest A11 in the specimen SP2 is acquired again.

[0082] In step S24A, it is determined whether micro images of all regions of interest (n locations) have been acquired. For example, if only micro image MC21 of region of interest A11 has been acquired, and if it is determined in step S16A that IoU is equal to or greater than the reference value, the management unit 120 overwrites the deviation amounts of macro images MA1 and MA2 calculated in step S8 or overwritten in step S12 as they are. On the other hand, if the deviation amounts of micro images MC11 and MC21 are calculated in step S17 or step S22, the deviation amounts are overwritten on the deviation amounts of macro images MA1 and MA2 calculated in step S8 or overwritten in step S12 (S24). In this way, the deviation amounts of macro images MA1 and MA2 calculated in step S8 or overwritten in step S12 are corrected using the smaller deviation amount (FIG. 8 "Reflected in macro alignment"). Although the displacement amount calculated for the first region of interest A11 is overwritten, the average value of the displacement amounts calculated for the first and second (multiple) regions of interest may be overwritten as the displacement amount.

[0083] In step S13, the management unit 120 overwrites the displacement calculated in this process with the displacement stored in the memory area 110. This updates the registered position of the region of interest A11-13 in the specimen SP2. After that, in step S14 and subsequent steps, the value of N is incremented to N=2, and processing begins to acquire the micro image M22 corresponding to the next region of interest A12.

[0084] If two or more micro images have been acquired, the management unit 120 returns the process to step S14, increments the value of N, and starts the process of acquiring a micro image corresponding to the next region of interest. As an example, if the second micro image MC22 has been acquired in step S24A, the management unit 120 returns the process to step S14, sets N to 3, and starts the process of acquiring a third micro image MC23 corresponding to the region of interest A13.

[0085] Also, when micro images of all areas of interest (n locations) have been acquired in step S24A, in this example, when all micro images MC21-MC23 of areas of interest A11-A13 have been acquired, the slide glass holding the second specimen SP2 is removed from the stage 32.

[0086] If it is determined in step S25A that specimen SP2 is not the last specimen, the management unit 120 returns the process to step S6, sets m to 3, and starts image acquisition processing for the third specimen S3. In this way, the value of m is incremented each time the process passes from step S25A to step S6, and the processing from steps S6 to S25 is executed sequentially for each specimen SP3, SP4, SP5, ...

[0087] If it is determined in step S25A that the processing for the last specimen has been executed, the management unit 120 executes processing to display the micro images side by side on the output device 40 (S26), as shown in Fig. 12. In Fig. 12, micro images MC11-MC101 of the region of interest A11 in each of the specimens SP1-S10 are displayed side by side. Note that the micro images MC11-MC101 can be digitally zoomed at any position within the image.

[0088] Alternatively, in S26, the management unit 120 executes a process of displaying the micro images superimposed on the output device 40 as shown in Fig. 10. In Fig. 10, the micro images MC11 and MC21 are displayed superimposed in the depth direction of the screen of the output device 40 (the depth direction of the paper in the figure).

[0089] As shown in Figure 12, micro images MC21-MC101 are images acquired for the region of interest A11 of the mth specimen SPm corresponding to the region of interest A11 shown in micro image MC11, by correction or alignment based on the above-mentioned amount of deviation. Furthermore, taking into account the angular (rotational) deviation between the orientation of the first specimen SP1 and the orientation of the mth specimen SPm, each micro image is displayed with its angle changed relative to micro image MC11. Therefore, each micro image is displayed with no or reduced deviation between the regions of interest between the images. The user can accurately compare each micro image MC11-MC101 while viewing the display on the output device 40.

[0090] 12, the management unit 120 displays on the output device 40 all of the micro images (MC11-MC101) of the area of ​​interest A11 (number 1 displayed on the macro image MA1) specified by the user via the input device 50 among the areas of interest A11, A12, and A13 (corresponding to numbers 1, 2, and 3 displayed on the macro image MA1, respectively) of the first macro image MA1 displayed in the lower right corner of the output device 40. Also, as shown in FIG. 12, when the management unit 120 displays on the output device 50 all of the micro images (MC11-MC101) corresponding to the area of ​​interest specified by the user via the input device 50, the management unit 120 executes a process of displaying a mask (a rectangular mask that covers the display range of the micro image) over the number displayed on the first macro image MA1 that corresponds to the area of ​​interest (S27).

[0091] In addition, the process of indicating the positions of the attention areas A11, A12, and A13 on the macro image MA1 and superimposing a rectangular graphic mask display MS that covers the display range of the micro image on one of the attention areas A11, A12, and A13 (corresponding to the numbers 1, 2, and 3 displayed on the macro image MA1, respectively) may be performed on the attention areas micro-observed (image acquired) by the user.

[0092] By checking the mask display MS, the user can distinguish between, for example, regions of interest for which image display (or observation) has been performed and regions of interest for which image display (or observation) has not yet been completed.

[0093] 13 and 14, the user can also display various annotations AN. The annotations AN include numerical values ​​indicating the distance between two points and various shapes. Note that the user may also be able to display notes on each micro image and write text.

[0094] 7A to 7C, the user may be able to set in advance to omit the IoU calculation (evaluation). That is, the user may operate the input device 40 to set in advance that the management unit 120 does not perform the IoU (Intersection over Union) calculation. In this case, after step S8 in FIGS. 7A to 7C, steps S9 to S12 are not performed (omitted), and the process proceeds to step S13. Next, as one method, after step S15, step S16 is not performed (omitted), and the process proceeds to step S17, and after step S18, step S19 is not performed (omitted), and the process proceeds to step S20. As another method, after step S15, steps S16 to S23 are not performed (omitted), and the process proceeds to step S24A. Furthermore, as another method, after step S15, step S16 is not performed (omitted), and the process proceeds to step S17, and after step S18, step S19 is not performed (omitted), and the process proceeds to step S24A. After step S24A, in step S24, the management unit 120 overwrites the deviation amounts of the macro images MA1 and MA2 calculated in step S8 as they are.

[0095] <Effects> The following aspects are disclosed in the embodiments.

[0096] (Aspect 1) The microscope device 30 includes an optical system P1 (corresponding to the first observation optical system) that acquires macro images, an optical system P2 (corresponding to the second observation optical system) that is different from the optical system P1 and acquires micro images, and an information processing device 10 (corresponding to the control unit) that activates the management unit 120.

[0097] The management unit 120 executes a process (S1) of acquiring a macro image MA1 of the specimen SP1 using the optical system P1, an acquisition process (S3) of acquiring first position information indicating the position of the area of ​​interest A11-A13 within the macro image MA1, and a process (S4) of controlling the positional relationship between the stage 32 on which the specimen SP1 is placed and the optical system P2 based on the first position information, and acquiring micro images MC11-MC13 of the area of ​​interest A11-A13 using the optical system P2.

[0098] In addition, the management unit 120 performs the following processes using the optical system P1: a process (S6) of acquiring a macro image MA2 of a specimen SP2 different from the specimen SP1; a process (S8, S12) of calculating a macro shift amount, which is the amount of positional shift between the macro image MA1 and the macro image MA2; and a process (S13) of acquiring second position information in the macro image MA2 based on the macro shift amount, which indicates the position of the attention area A11-A13 corresponding to the first position information.

[0099] In addition, based on the second position information, the management unit 120 controls the relative position between the stage 32 on which the specimen SP2 is placed and the optical system P2, and executes a process (S15, S18) of acquiring micro images MC21-MC23 of the areas of interest A11-A13 in the specimen SP2 using the optical system P2.

[0100] With the above configuration, the positions of the regions of interest A11-A13 in the specimen SP2 can be accurately determined based on the positional shift between the macro images MA1 and MA2. Therefore, the micro images MC21-MC23 can be captured without or with a small amount of shift relative to the micro images MC11-MC13, respectively. In this way, accurate image acquisition is possible.

[0101] (Aspect 2) In aspect 1, the management unit 120 acquires the first location information in response to an instruction from the input device 50 (corresponding to the input unit) in the acquisition process.

[0102] In the above configuration, the user can use the input device 50 to easily specify the attention areas A11-A13.

[0103] (Aspect 3) In Aspect 1 or 2, the resolution of each of the macro images MA1 and MA2 is lower than the resolution of any of the micro images MC11-MC13 and MC21-MC23.

[0104] With the above configuration, macro images can be quickly acquired and processed, making it possible to quickly grasp the overall image of specimens SP1, S2, etc.

[0105] (Aspect 4) In any of Aspects 1 to 3, the macro images MA1 and MA2 are each acquired collectively by the optical system P1.

[0106] Since the macro images MA1 and MA2 are acquired collectively, each image can be acquired quickly in the above configuration. Furthermore, since the collectively acquired macro images MA1 and MA2 have a smaller capacity than macro images acquired by dividing them, calculation of the amount of deviation between the macro images MA1 and MA2 and position correction (information processing) of the macro image MA2 relative to the macro image MA1 based on the calculated amount of deviation can be performed quickly.

[0107] (Aspect 5) In any of aspects 1 to 4, the management unit 120 calculates the amount of deviation (micro deviation) between the micro images MC11 and MC21, and further executes a process (S24) to correct the amount of deviation (macro deviation) between the macro images MA1 and MA2 based on this amount of deviation.

[0108] In the above configuration, the highly accurate micro-shift amount obtained from the micro-images MC11 and MC21 is used, so that the macro-shift of the macro-images MA1 and MA2 can be corrected more accurately.

[0109] (Aspect 6) In any of aspects 1 to 5, at least the region of interest A11 (first region of interest No. 1) and the region of interest A12 (first region of interest No. 2) are present in the macro image MA1. The management unit 120 acquires first position information No. 1 indicating the position of the region of interest A11 and first position information No. 2 indicating the position of the region of interest A12 (S3). The management unit 120 further acquires, from the macro image MA2, second position information No. 1 indicating the position of the region of interest A11 (second region of interest No. 1) corresponding to the first position information No. 1, and second position information No. 2 indicating the position of the region of interest A12 (second region of interest No. 2) corresponding to the first position information No. 2 (S13). The micro-shift amount is calculated based on the micro image MC11 of the first region of interest No. 1 and the micro image MC12 of the second region of interest No. 1. Based on the corrected macro deviation amount, a process (S13) is performed to correct the position information (second position information part 2) of the area of ​​interest A12 in the macro image MA2, and a process (S15) is performed to acquire a micro image MC22 corresponding to the second position information part 2 based on this corrected position information.

[0110] (Aspect 7) In any of aspects 1 to 6, the management unit 120 calculates a micro-shift amount, which is the amount of positional shift between the micro-image MC11 and the micro-image MC21, and further executes a process of controlling the positional relationship between the stage 32 on which the specimen SP2 is placed and the second optical system P2 based on the micro-shift amount (S17).

[0111] With the above configuration, micro images MC11 and MC21 with little deviation can be acquired.

[0112] (Embodiment 8) In any of embodiments 1 to 7, the management unit 120 reads out the micro image MC11 stored in the memory area 110 of the main memory device 102 or the auxiliary memory device 103, displays it on the output device 40 (corresponding to the display unit), and further executes a process (S20) of displaying on the output device 40 the micro image MC21 that is currently being acquired by the optical system P2.

[0113] In the above configuration, the user can easily align the two micro images MC11 and MC21 while checking the alignment on the screen.

[0114] (Aspect 9) In any of aspects 1 to 8, the management unit 120 controls the positional relationship between the stage 32 on which the specimen SP2 is placed and the optical system P2 based on instructions from the input device 50 (corresponding to the input unit), and further executes a process (S21) of moving the area of ​​interest A11 of the micro image MC21 displayed on the output device 40.

[0115] With the above configuration, the area of ​​interest A11 of the micro image MC21 can be moved quickly.

[0116] (Aspect 10) In any of aspects 1 to 9, the management unit 120, in response to instructions from the input device 50, increases the transparency of at least one of the macro images MA1 and MA2 and displays them on the output device 40, and further executes processes (S10, S11) to change the overlap of the specimens SP1 and SP2 displayed on the macro images MA1 and MA2.

[0117] With the above configuration, the misalignment between the macro images MA1 and MA2 can be easily visually recognized and corrected.

[0118] (Embodiment 11) In any of embodiments 1 to 10, the management unit 120 executes a process of calculating IoU, which is the degree of alignment of the macro images MA1 and MA2, based on the amount of deviation between the macro images (macro deviation amount) (S9).

[0119] In the above configuration, the degree of alignment can be measured by calculating IoU.

[0120] (Aspect 12) In any of aspects 1 to 11, if the calculated IoU is equal to or less than a preset reference value, the management unit 120 increases the transparency of at least one of the macro images MA1 and MA2 in response to an instruction from the input device 50, displays the images on the output device 40, and further executes the process (S10, S11) of displaying the specimens SP1 and SP2 displayed on the macro images MA1 and MA2 so that they overlap, and again calculates the macro deviation amount after the process (S12).

[0121] In the above configuration, the degree of alignment can be measured by calculating IoU.

[0122] (Embodiment 13) In any of embodiments 1 to 12, the management unit 120 further executes a process (S16, S19) of calculating IoU, which is the degree of alignment of the micro-images MC11, MC21, micro-images MC12, MC22, or micro-images MC13, MC23.

[0123] (Aspect 14) In any of aspects 1 to 13, if the calculated IoU is equal to or less than a preset reference value, the management unit 120 reads out the micro image MC11 stored in the memory area 110 and displays it on the output device 40, and also displays the micro image MC21 being acquired by the optical system P2 on the output device 40, and controls the positional relationship between the stage 32 on which the specimen SP2 is placed and the optical system P2 based on instructions from the input device 50, and further performs processing to move the area of ​​interest A11 of the micro image MC21 displayed on the output device 40 (S20, S21).

[0124] With the above configuration, the area of ​​interest A11 of the micro image MC21 can be easily moved.

[0125] (Embodiment 15) In any of embodiments 1 to 14, the management unit 120 further executes a process (S26) of displaying the micro images side by side.

[0126] In the above configuration, the user can easily compare and confirm the regions of interest A11-A13 between specimens.

[0127] (Mode 16) In any of modes 1 to 15, when the management unit 120 receives a process to acquire micro images MC11 and MC21 or a process to display the micro images side by side, it further executes a process to perform a predetermined display on the area of ​​interest A11 on the macro image MA1 displayed on the display unit (S27).

[0128] With the above configuration, the user can distinguish between, for example, regions of interest that have been observed and regions of interest that have not yet been observed. [Explanation of symbols]

[0129] 1 System 10. Information processing equipment 20 Optical equipment 30 Microscope equipment

Claims

1. a first observation optical system for acquiring a macro image; a second observation optical system that acquires a micro image and is different from the first observation optical system; a control unit, The control unit acquiring a first macro image of a first specimen using the first observation optical system; an acquisition process for acquiring first position information indicating a position of a first region of interest within the first macro image; a process of controlling a positional relationship between a stage on which the first specimen is placed and the second observation optical system based on the first position information, and acquiring a first micro-image of the first region of interest using the second observation optical system; acquiring a second macro image of a second specimen different from the first specimen using the first observation optical system; A process of calculating a macro deviation amount, which is a positional deviation amount between the first macro image and the second macro image; a process of acquiring second position information indicating a position of a second region of interest corresponding to the first position information in the second macro image based on the macro displacement amount; a process of controlling a positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the second position information, and acquiring a second micro-image of the second region of interest using the second observation optical system; Microscope to perform.

2. The control unit In the acquisition process, the first location information is acquired in response to an instruction from an input unit. The microscope according to claim 1 .

3. the resolution of each of the first macro image and the second macro image is lower than the resolution of both the first micro image and the second micro image; 3. The microscope according to claim 1 or 2.

4. the first macro image and the second macro image are acquired collectively by the first observation optical system, The microscope according to any one of claims 1 to 3.

5. The control unit Calculating a micro-shift amount, which is a positional shift amount between the first micro-image and the second micro-image; and further performing a process of correcting the macro-deviation amount based on the micro-deviation amount.

5. The microscope according to claim 1.

6. a plurality of first attention regions are present in the first macro image as at least a first attention region 1 and a first attention region 2; the control unit acquires first position information 1 indicating a position of the first region of interest 1 and first position information 2 indicating a position of the first region of interest 2; In the second macro image, second position information 1 indicating a position of a second region of interest 1 corresponding to the first position information 1 and second position information 2 indicating a position of a second region of interest 2 corresponding to the first position information 2 are acquired; the micro-shift amount is calculated based on the first micro-image of the first region of interest and the second micro-image of the second region of interest, a process of correcting the second position information part 2 in the second macro image based on the corrected macro deviation amount; and acquiring the second micro-image of the second region of interest based on the corrected second position information. The microscope according to claim 5.

7. The control unit Calculating a micro-shift amount, which is a positional shift amount between the first micro-image and the second micro-image; and further performing a process of controlling the positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the micro-displacement amount.

5. The microscope according to claim 1.

8. The control unit and further performing a process of reading out the first micro-image stored in the storage unit and displaying it on a display unit, and displaying the second micro-image being acquired by the second observation optical system on the display unit. A microscope according to any one of claims 1 to 7.

9. The control unit and further executing a process of controlling the positional relationship between a stage on which the second specimen is placed and the second observation optical system in response to an instruction from an input unit, and moving the second region of interest of the second micro-image displayed on the display unit. The microscope according to claim 8.

10. The control unit and further executing a process of increasing the transparency of at least one of the first macro image and the second macro image and displaying the image on a display unit in response to an instruction from an input unit, and changing an overlap between the first specimen displayed on the first macro image and the second specimen displayed on the second macro image.

10. The microscope according to claim 1.

11. The control unit further performing a process of calculating a degree of alignment between the first macro image and the second macro image based on the macro deviation amount. The microscope according to claim 1 .

12. The control unit If the calculated value is equal to or less than a preset reference value, a display process is further performed in which the transparency of at least one of the first macro image and the second macro image is increased and displayed on a display unit in response to an instruction from an input unit, and the first specimen displayed in the first macro image and the second specimen displayed in the second macro image are overlapped, and the macro deviation amount after the display process is calculated. The microscope according to claim 11.

13. The control unit further performing a process of calculating a degree of alignment between the first micro-image and the second micro-image; 7. The microscope according to claim 5 or 6.

14. The control unit When the calculated value is equal to or less than a predetermined reference value, the first micro-image stored in the storage unit is read out and displayed on a display unit, and the second micro-image being acquired by the second observation optical system is displayed on the display unit; and further executing a process of controlling the positional relationship between a stage on which the second specimen is placed and the second observation optical system in response to an instruction from an input unit, and moving the second region of interest of the second micro-image displayed on the display unit. A microscope according to any one of claims 1 to 13.

15. The control unit further performing a process of displaying the first micro image and the second micro image side by side; A microscope according to any one of claims 1 to 14.

16. The control unit When the process of acquiring the first micro image and the second micro image or the process of displaying them side by side is received, a process of performing a predetermined display on the first attention area on the first macro image displayed on a display unit is further executed.

16. A microscope according to any one of claims 1 to 15.

17. a first observation optical system for acquiring a macro image; a second observation optical system that acquires a micro image and is different from the first observation optical system; a control unit; and a method for causing the control unit to execute the method in a microscope including the control unit, acquiring a first macro image of a first specimen using the first observation optical system; an acquisition process for acquiring first position information indicating a position of a first region of interest within the first macro image; a process of controlling a positional relationship between a stage on which the first specimen is placed and the second observation optical system based on the first position information, and acquiring a first micro-image of the first region of interest using the second observation optical system; acquiring a second macro image of a second specimen different from the first specimen using the first observation optical system; A process of calculating a macro deviation amount, which is a positional deviation amount between the first macro image and the second macro image; a process of acquiring second position information indicating a position of a second region of interest corresponding to the first position information in the second macro image based on the macro displacement amount; a process of controlling a positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the second position information, and acquiring a second micro-image of the second region of interest using the second observation optical system; A method comprising:

18. a first observation optical system for acquiring a macro image; a second observation optical system that acquires a micro image and is different from the first observation optical system; a control unit, and a microscope including the control unit, acquiring a first macro image of a first specimen using the first observation optical system; an acquisition process for acquiring first position information indicating a position of a first region of interest within the first macro image; a process of controlling a positional relationship between a stage on which the first specimen is placed and the second observation optical system based on the first position information, and acquiring a first micro-image of the first region of interest using the second observation optical system; acquiring a second macro image of a second specimen different from the first specimen using the first observation optical system; A process of calculating a macro deviation amount, which is a positional deviation amount between the first macro image and the second macro image; a process of acquiring second position information indicating a position of a second region of interest corresponding to the first position information in the second macro image based on the macro displacement amount; a process of controlling a positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the second position information, and acquiring a second micro-image of the second region of interest using the second observation optical system; A program that executes the following.

Images