Microscope, method, and program

Abstract

This microscope comprises: a first observation optical system that acquires a macro image; a second observation optical system that is different from the first observation optical system and acquires a micro image; and a control unit, the control unit executing processing for acquiring a first macro image of a first sample using the first observation optical system, acquisition processing for acquiring first positional information indicating the position of a first region of interest in the first macro image, processing for controlling the positional relationship between a stage on which the first sample is mounted and the second observation optical system on the basis of the first positional information to acquire a first micro image of the first region of interest using the second observation optical system, processing for acquiring a second macro image of a second sample different from the first sample using the first observation optical system, processing for acquiring second positional information indicating the position of a second region of interest corresponding to the first positional information in the second macro image on the basis of the first macro image and the second macro image, and processing for controlling the positional relationship between the stage on which the second sample is mounted and the second observation optical system on the basis of the second positional information to acquire a second micro image of the second region of interest using the second observation optical system.

Description

Microscope, method, and program

[0001] The present invention relates to a microscope, a method, and a program.

[0002] Conventionally, a method for performing alignment of an image acquired by a microscope is known (Patent Document 1).

[0003] Patent No. 6643072

[0004] The techniques of this disclosure provide a novel method for image registration.

[0005] One embodiment of the present invention is a microscope comprising a first observation optical system that acquires a macro image, a second observation optical system that acquires a micro image different from the first observation optical system, and a control unit, wherein the control unit executes the following processes: a process of acquiring a first macro image of a first specimen using the first observation optical system; a process of acquiring first position information that indicates the position of a first region of interest within the first macro image; a process of controlling the positional relationship between a stage on which the first specimen is placed and the second observation optical system based on the first position information, and acquiring a first micro image of the first region of interest using the second observation optical system; a process of acquiring a second macro image of a second specimen that is different from the first specimen using the first observation optical system; a process of acquiring second position information that indicates the position of a second region of interest in the second macro image that corresponds to the first position information, based on the first macro image and the second macro image; and a process of controlling the positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the second position information, and acquiring a second micro image of the second region of interest using the second observation optical system.

[0006] Furthermore, one embodiment of the present invention is a method for a microscope including a first observation optical system for acquiring a macro image, a second observation optical system for acquiring a micro image different from the first observation optical system, and a control unit, the method being executed by the control unit, the method including: a process of acquiring a first macro image of a first specimen using the first observation optical system; an acquisition process of acquiring first position information indicating a position of a first region of interest within the first macro image; a process of controlling a positional relationship between a stage on which the first specimen is placed and the second observation optical system based on the first position information, and acquiring a first micro image of the first region of interest using the second observation optical system; a process of acquiring a second macro image of a second specimen different from the first specimen using the first observation optical system; a process of calculating a macro shift amount that is a positional shift between the first macro image and the second macro image based on the macro shift amount; a process of acquiring second position information indicating a position of a second region of interest in the second macro image that corresponds to the first position information; and a process of controlling a positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the second position information, and acquiring a second micro image of the second region of interest using the second observation optical system.

[0007] a control unit; and a program for causing the control unit to execute the following operations: acquiring a first macro-image of a first specimen using the first observation optical system; acquiring first position information indicating the position of a first region of interest within the first macro-image; controlling the positional relationship between a stage on which the first specimen is placed and the second observation optical system based on the first position information, and acquiring a first micro-image of the first region of interest using the second observation optical system; acquiring a second macro-image of a second specimen different from the first specimen using the first observation optical system; calculating a macro-shift amount that is the amount of positional shift between the first macro-image and the second macro-image based on the macro-shift amount; acquiring second position information indicating the position of a second region of interest in the second macro-image that corresponds to the first position information, and controlling the positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the second position information, and acquiring a second micro-image of the second region of interest using the second observation optical system.

[0008] 1 is a diagram illustrating the overall configuration of a system according to an embodiment, showing a situation in which an optical system P1 is used; FIG. 2 is a diagram illustrating the overall configuration of a system according to an embodiment, showing a situation in which an optical system P2 is used; FIG. 3 is a diagram illustrating the hardware configuration of an information processing device; (a) a block diagram illustrating the overall configuration of the system, (b) a block diagram illustrating the hardware configuration of the information processing device, and (c) a block diagram illustrating the functional configuration (software configuration) of the information processing device; FIG. 4 is a diagram illustrating the configuration of a biological tissue and a specimen; FIG. 5 is a diagram illustrating the configuration of a specimen and a region of interest; FIG. 6 is a diagram illustrating the configuration of a biological tissue and a micro image; FIG. 7 is a flowchart of processing executed in an embodiment; FIG. 8 is a flowchart of processing executed in an embodiment; FIG. 9 is a diagram illustrating an overview of alignment executed in a macro image and a micro image; FIG. 10 is an example of a screen displayed on an output device, showing a situation in which one micro image is displayed; FIG. 11 is an example of a screen displayed on an output device, showing a situation in which two micro images are displayed superimposed; FIG. 12 is an example of a screen displayed on an output device, showing a situation in which two micro images are displayed side by side; FIG. 13 is an example of a screen displayed on an output device, showing a situation in which a plurality of micro images are displayed side by side; 1 is an example of a screen displayed on an output device, showing a situation in which a plurality of micro images are displayed side by side.

[0009] Hereinafter, the present invention will be described based on one embodiment thereof with reference to the drawings.

[0010] 1 and 2 show the configuration of a system 1 according to one embodiment of the present invention. The system 1 includes a microscope device 30 incorporating an information processing device 10, an output device 40, and an input device 50. The output device 40, the input device 50, and the microscope device 30 are connected to each other via communication means such as a bus (not shown) so as to be able to send and receive data to and from each other.

[0011] The output device 40 acquires images captured by the microscope device 30 and processes the images for display. The output device 40 is an interface that outputs various types of information, and is, for example, a screen display device (liquid crystal monitor, LCD (Liquid Crystal Display)), a printer, etc.

[0012] The input device 50 is an interface that accepts input of information, and is, for example, a keyboard, a mouse, a touch panel, a card reader, a voice input device (such as a microphone), a voice recognition device, or the like.

[0013] The microscope device 30 is a device that uses a microscope to capture (photograph) biological tissue BT (FIG. 4) as a subject. One example of the subject is biological tissue such as a cell.

[0014] As shown in FIG. 3, the microscope device 30 includes an information processing device 10 and an optical device 20 .

[0015] The optical device 20 includes an objective lens 31 (including multiple objective lenses with different magnifications, which can be switched), a second objective lens 34, a stage 32, a light source 35 that illuminates the specimen, reflecting mirrors 36 and 37, and an imaging unit 39 (including a CCD sensor, a CMOS sensor, etc.), and is capable of photographing the specimen placed on the stage 32.

[0016] The stage 32 is a member having a flat upper surface, and can move in the directions of the X and Y axes in Figures 1 and 2. The X and Y axes are parallel to the upper surface of the stage 32 and are perpendicular to each other.

[0017] The optical device 20 includes two optical systems P1 and P2. As shown in Fig. 1, the optical system P1 has a reduction lens (not shown) and is used to form a macro image of the entire specimen on the imaging unit 39. As shown by the dashed dotted line in Fig. 1, the optical system P1 guides light from the specimen placed on the stage 32 to the imaging unit 39 via the reduction lens (not shown) and a reflecting mirror 36.

[0018] The optical system P2 is used to magnify a portion of the specimen and form a micro-image on the imaging unit 39. As shown by the dashed line in Figure 2, the optical system P2 guides light from the specimen placed on the stage 32 to the imaging unit 39 via the objective lens 31, the reflecting mirror 37, and the second objective lens 34 in this order.

[0019] The magnification of the optical system P2 varies depending on the magnification of the objective lens 31 used, and is, for example, 4x, 10x, 20x, or 40x.

[0020] The optical device 20 is equipped with two optical systems P1 and P2 to acquire macro and micro images, but the observable range may be changed by switching the lenses that make up a single optical system.

[0021] The macro image is acquired by capturing an optical image of the entire specimen formed by the optical system P1 with the imaging unit 39, and by performing a single image capture. In other words, the macro image is not formed by combining multiple images of the specimen captured for each predetermined region (each divided region), but rather the entire specimen is captured at once.

[0022] Therefore, the capacity of the macro image is approximately 1 / the number of images (for example, 40 to 100) compared to when multiple images are combined to form the macro image, and since image combining is not required, processing in the information processing device 10 (including image display on the output device 40), which will be described later, can be performed at high speed.

[0023] The micro image is captured as an image optically enlarged more than the macro image by the objective lens 31 and the second objective lens 34. Furthermore, the micro image is acquired at a higher resolution than the macro image.

[0024] The image capturing unit 39 can be switched between a low-resolution mode and a high-resolution mode, and the resolution of the macro image is 10 to 80 μm / px depending on which mode is selected. The resolution of the micro image is 0.1 to 4 μm / px depending on the combination of the selected mode of the image capturing unit 39 and the magnification (observable range) of the objective lens used.

[0025] The time required from step S7 to step S15 (from acquiring macro images of m target books to acquiring micro images of m target books) shown in Figures 7A-C described below is at least 20 seconds if there is one area of ​​interest (one micro image).

[0026] The information processing device 10 has a function of controlling the optical device 20 and acquiring an image.

[0027] 3B shows an example of hardware used to realize the information processing device 10. As shown in the figure, the information processing device 10 includes a processor 101, a main memory device 102, an auxiliary memory device 103, and a communication device 106. These are connected to each other so as to be able to communicate with each other via communication means such as a bus (not shown).

[0028] Furthermore, the information processing device 10 does not necessarily need to have all of its configuration realized by hardware, and all or part of its configuration may be realized by virtual resources such as a cloud server of a cloud system.

[0029] The processor 101 is configured using a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc. The processor 101 reads and executes programs stored in the main memory device 102, thereby realizing the functions of the information processing device 10.

[0030] The main storage device 102 is a device that stores programs and data, and is a read-only memory (ROM), a random access memory (RAM), a non-volatile semiconductor memory (NVRAM), etc. The auxiliary storage device 103 is a variety of non-volatile memories (NVRAM) such as a solid-state drive (SSD) or an SD memory card, a hard disk drive, an optical storage device (a compact disc (CD), a digital versatile disc (DVD), etc.), a storage area of ​​a cloud server, etc.

[0031] The communication device 106 is a wired or wireless communication interface that enables communication with other devices, such as a NIC (Network Interface Card), a wireless communication module, a USB (Universal Serial Interface) module, or a serial communication module.

[0032] The information processing device 10 communicates information with the optical device 20, the output device 40, and the input device 50 via the communication device 106, and executes various controls.

[0033] 3C shows the main functional configuration of the information processing device 10. As shown in the figure, the information processing device 10 includes a storage area 110 and a management unit 120.

[0034] The storage area 110 is formed in the main storage device 102 or the auxiliary storage device 103 of the information processing device 10. Various information is stored in the storage area 110. Specific examples include images acquired by the system 1, image coordinates, the amount of misalignment between images, and the position of a region of interest (all of which will be described in detail later).

[0035] In addition to the above functions, the information processing device 10 also has functions such as an operating system, a file system, a device driver, and a DBMS (DataBase Management System).

[0036] The management unit 120 performs processes such as image acquisition and management that are executed by the information processing device 10. The functions of the management unit 120 are realized by the processor 101 of the information processing device 10 reading and executing a program stored in the main storage device 102 or the auxiliary storage device 103 of the information processing device 10.

[0037] The management unit 120 communicates with the optical device 20, the output device 40, and the input device 50 via the communication device 106, and executes various controls over these devices.

[0038] [Processing Details] The processing executed in the system 1 will be described in detail below with reference to the flowcharts of FIGS. 7A to 7C.

[0039] A program stored in the main memory device 102 of the information processing device 10 is started, and the management unit 120 executes the processing of the system 1 as follows. Note that, hereinafter, the processing executed by the management unit 120 of the server 10 may be simply described as being executed by the "information processing device 10."

[0040] In step S1, a user (e.g., a pathologist) places a slide glass holding a first specimen SP1 on the stage 32. As shown in FIG. 4 , a specimen is created by embedding biological tissue BT in paraffin RF to form a block, slicing the block, and staining each slice. The process for creating a specimen from biological tissue BT is not limited to the above; for example, a method of freezing the biological tissue BT and slicing the block may be selected. Therefore, typically, multiple slide glasses are prepared according to the number of stains. Here, biological tissue BT held on one slide glass is referred to as one specimen. In other words, specimens are prepared according to the number of stains.

[0041] The staining is performed by at least one of hematoxylin-eosin staining (HE staining) and "immunostaining," which aims to obtain distribution information of specific substances that constitute biological tissue and stains using an antigen-antibody reaction with a detection target substance. In addition to these staining methods, other methods such as FISH (fluorescence in situ hybridization), DISH (dual color in situ hybridization), CISH (chromogenic in situ hybridization), and SISH (silver-enhanced in situ hybridization) may also be used.

[0042] In the following description, the specimens held on the respective slide glasses will be given individual reference symbols SP1, SP2, SP3, . . . as shown in FIG.

[0043] Each slide holds a sliced ​​and stained specimen SP1, SP2, ... as described above. The shape of the specimen on each slide is approximately the same, but not necessarily perfectly identical. Furthermore, the position and orientation (angle) of the specimen on each slide usually differ from one slide to another, influenced by the shape of the biological tissue BT and the slicing and staining processes. Furthermore, when placed on the stage 32, there may be a misalignment in the installation position of each slide. Therefore, the position of the specimen (which can also be considered as coordinates indicating the position) in each image acquired by the optical device 20 may be misaligned.

[0044] In step S2, the management unit 120 controls the imaging unit 39 via the optical system P1 to capture the first image of the entire specimen SP1. The specimen SP1 is captured in one shot, that is, one image is formed by capturing the image once. As a result of the capture, a macro image MA1 is acquired and stored in the memory area 110.

[0045] In step S3, a region of interest (also referred to as a position of interest) is set in the specimen SP1 or the macro image MA1. The region of interest is set by the user specifying a position on the macro image MA1 displayed on the output device 40. The region of interest is set by the user operating the input device 50. Regions of interest are set in n locations (n ​​is an integer equal to or greater than 1) desired by the user, and there is no limit to the number of regions of interest that can be set.

[0046] In this example, it is assumed that the user sets three areas of interest A11-A13 (FIG. 5). The management unit 120 stores in the storage area 110 coordinates (X and Y coordinates) indicating the positions of the areas of interest A11-A13 in the specimen SP1 or macro image MA1 set by the user, using the macro image MA1 as a reference.

[0047] Here, the regions of interest A11-A13 are not specific to specimen SP1, but are also regions set in each of specimens SP2, SP3, ... They may also be considered to be regions set in the biological tissue BT. As described above, each specimen is cut out from the biological tissue BT, and therefore, the regions of interest A11-A13 are set in the same locations in each specimen as in specimen SP1. As will be described in detail later, a user can acquire images of, for example, the region of interest A11 in each specimen via the microscope device 30 and compare the differences between the images, thereby making it possible to compare common regions in specimens stained using different staining techniques, which can be utilized for diagnosis, etc.

[0048] In step S4, the imaging unit 39 images each of the regions of interest A11-A13 of the specimen SP1 once via the optical system P2, thereby acquiring micro images MC11-MC13, which are then stored in the storage area 110 ( FIG. 5 ). At this time, the magnification of the objective lens used is also stored in the storage area 110. At this time, the management unit 120 controls the positional relationship with the optical system P2 by moving the stage 32 on which the specimen SP1 is placed within the XY plane based on the coordinates indicating the positions of the regions of interest A11-A13 stored in the storage area 110, focuses the objective lens 31 on the specimen SP1, and images the regions of interest A11-A13 using the imaging unit 39.

[0049] When storing the micro images MC11-MC13 in the storage area 110, the management unit 120 adds each focus position (Z coordinate) as position information of the attention areas A11-A13 and stores the information in the storage area 110.

[0050] The entire optical system P2 may be configured to be movable in the X and Y directions.

[0051] Although an example has been described in which micro images MC11 to MC13 are acquired for each of the attention areas A11 to A13 after all of the attention areas A11 to A13 have been registered, the present invention is not limited to this.

[0052] For example, when setting the attention area A11, a micro image MC11 is acquired and stored in the memory area 110, and then when setting the attention area A12, a micro image MC12 is acquired and stored in the memory area 110, and then when setting the attention area A13, a micro image MC13 is acquired and stored in the memory area 110. (Combination of steps S3 and S4)

[0053] The management unit 120 displays the macro image MA1 read from the storage area 110 on the output device 40 and moves the stage carrying the specimen SP1 a predetermined amount in the Y direction to align it with the optical path of the optical system P2. When the user operates the input device 50 to specify a predetermined position on the macro image MA1, the management unit 120 controls the positional relationship with the optical system P2 by moving the stage carrying the specimen SP1 within the XY plane based on the coordinates indicating the specified position, thereby adjusting the focus of the objective lens 31 on the specimen SP1. Also, as shown in FIG. 9 , the management unit 120 acquires a micro image of the area corresponding to the specified position of the specimen SP1 via the optical system P2 using the imaging unit 39 and displays it on the output device 40. The user checks the micro image displayed on the output device 40, determines whether it is appropriate, and, if appropriate, operates the input device 50 to register the specified position (XY position, focus position). The magnification of the objective lens used is also registered. The management unit 120 sets the coordinates (X and Y coordinates, Z coordinate indicating the focus position) indicating the registered designated position as the attention area A11, and stores the micro image displayed on the output device 40 as a micro image MC11 in the storage area 110. Note that the setting of the attention area A12, the setting of the attention area A13, etc. is also performed in a similar manner.

[0054] The micro images MC11-MC13 are images obtained by capturing the attention areas A11-A13, respectively. The micro images MC11-MC13 are images obtained at a higher magnification than the macro image, and therefore are acquired as images that are enlarged more than the macro image MA1. The micro images MC11-MC13 are also acquired at a higher resolution than the macro image MA1.

[0055] After the micro-images MC11-MC13 are acquired, the slide glass holding the specimen SP1 is removed from the stage 32 (S5), and a second slide glass holding the specimen SP2 is placed on the stage 32 (S6).

[0056] 7A, the process for the mth sample (m is an integer equal to or greater than 2) is generally described. In the following description of steps S6 to S25A, for ease of understanding, a specific example will be described in which m=2, i.e., the process for the second sample SP2, is performed.

[0057] In step S7, the imaging unit 39, controlled by the management unit 120, images the specimen SP2 via the optical system P1. The imaging of the specimen SP2 is performed in one shot, that is, by forming one image by one shooting. As a result of the imaging, a macro image MA2 is acquired and stored in the memory area 110.

[0058] In step S8, the amount of deviation between the macro images MA1 and MA2 stored in the storage area 110 is acquired. The management unit 120 extracts, from each of the macro images MA1 and MA2, multiple feature points (coordinates indicating positions within the image) predicted to be common to the specimens SP1 and SP2, and measures the deviation between the multiple feature points (coordinates indicating positions within the image) of the macro image MA1 and the multiple feature points (coordinates indicating positions within the image) predicted to correspond to these feature points of the macro image MA2, thereby acquiring the amount of deviation and storing the acquired amount of deviation in the storage area 110. Note that the method for calculating the amount of deviation of the macro image MA2 relative to the macro image MA1 is not limited to this; for example, the deviation may be calculated using the outline or area of ​​the specimen displayed in the macro image. The management unit 120 corrects the position of the macro image MA2 relative to the macro image MA1 using the acquired amount of deviation, thereby aligning the position of any region of interest in the macro image MA2 with the same region of interest in the macro image MA1.

[0059] In acquiring the amount of deviation, the coordinate deviation (shift) in each direction of the X-axis and Y-axis between the images, the deviation in image rotation (angle), and the deviation in enlargement / reduction (magnification) are acquired as shown in Fig. 10. As a result, the amount of deviation between the images is grasped using three matrices: an X-axis and Y-axis coordinate translation matrix between the two images, a rotation matrix, and a scaling matrix. Note that Fig. 10 shows micro images MC11 and MC12, but the amount of deviation, consisting of the deviation (shift) in the X-axis and Y-axis directions and the deviation in rotation (angle) between the specimen SP1 displayed in macro image MA1 and the specimen SP2 displayed in macro image MA2, is also grasped in the same way for macro images.

[0060] Next, the management unit 120 calculates IoU (Intersection over Union) based on the amount of deviation calculated in step S8 (S9). Specifically, the management unit 120 binarizes the macro images MA1 and MA2, and then quantifies the overlapping area of ​​the specimens SP1 and S2 displayed on the macro images MA1 and MA2 by calculating IoU. Generally, the closer the IoU value is to 1 from 0, the higher the degree of alignment. The method for calculating the degree of alignment is not limited to IoU, and it may also be calculated, for example, by the overlapping of the contours (lines) of the specimens SP1 and S2 displayed on the macro images MA1 and MA2.

[0061] Next, the management unit 120 compares the IoU with a predetermined reference value (S9A).

[0062] If the IoU is equal to or less than the reference value, the management unit 120 switches to manual correction mode ("macro alignment" in FIG. 8), reads out the macro images MA1 and MA2 from the storage area 110, and displays them on the output device 40 (S10).

[0063] Note that regardless of whether the IoU is equal to or less than the reference value, the user may be allowed to switch to the manual correction mode by operating the input device 40. Furthermore, after step S7, steps S8 and S9 may not be performed (omitted), and the user may be allowed to switch to the manual correction mode by operating the input device 40. In other words, when the management unit 120 receives an instruction from the input device 40, it switches to the manual correction mode.

[0064] The user sets the transparency (greater than 0% and less than 100%) of the macro images MA1 and MA2 based on input to the input device 50. It is also possible to change (increase) the transparency of only one of the macro images MA1 and MA2. In this case, the macro image with the changed (increased) transparency is arranged on top of the macro image with the unchanged transparency. This transparency needs to be set so that the overlap between the specimen SP1 displayed in the macro image MA1 and the specimen SP2 displayed in the macro image MA2 can be fully seen. Therefore, the transparency is set according to the state of the specimen.

[0065] The user manually aligns the macro images MA1 and MA2 via the input device 50 while viewing the macro images MA1 and MA2, which have been changed to a predetermined transparency (for example, 50%) and are displayed on the output device 40 (S11). At this time, the management unit 120 changes the relative position, angle, and enlargement / reduction (magnification) of the macro images MA1 and MA2 based on the input to the input device 50, and displays them on the output device 40. While viewing the display on the output device 40, the user continues to perform operations until the positions of the specimens SP1 and SP2 shown in the macro images MA1 and MA2 are sufficiently aligned.

[0066] After the manual alignment is completed, the management unit 120 recalculates the amount of misalignment between the aligned macro images MA1 and MA2, overwrites the amount of misalignment acquired in step S8, and stores the recalculated amount of misalignment in the storage area 110 (S12).

[0067] If IoU is equal to or greater than the reference value in step S9A, or after completion of step S12, the management unit 120 acquires coordinates indicating the position of the areas of interest A11-A13 relative to the macro image MA2 or the specimen SP2 based on the amount of deviation between the macro images MA1 and MA2, and stores them in the memory area 110.

[0068] In step S14, a second macro image MA2, which displays the positions of the attention areas A11-A13 acquired in step S13, is displayed on the output device 40. Note that step S14 may be omitted. In step S15, the management unit 120 acquires micro images of the specimen SP2. Specifically, the attention areas A11-A13 of the specimen SP2 are imaged by the imaging unit 39 via the optical system P2 and stored in the storage area 110. Each time step S15 is performed, one micro image is acquired for the Nth location (N is an integer from 1 to n) among the attention areas A11-A13 (n=3, so three locations). Finally, three micro images MC21-MC23 corresponding to the attention areas A11-A13, respectively, are acquired each time the process passes through step S15.

[0069] For ease of understanding, the following description of steps S14 to S24 will use as a specific example the case of capturing a micro image MC21 at the first location (N=1).

[0070] Using this specific example, in the processing of step S15, the management unit 120 controls the positional relationship with the optical system P2 by moving the stage 32 on which the specimen SP2 is placed within the XY plane, focuses the objective lens 31 on the specimen SP2, and captures an image of the area of ​​interest A11 in the specimen SP2 using the imaging unit 39. By capturing the image of the area of ​​interest A11, a micro image MC21 is acquired and stored in the memory area 110.

[0071] In step S16, the management unit 120 calculates the overlapping area between the micro-images MC11 and MC21 stored in the storage area 110 using IoU (S16). The calculation details are the same as those in step S9. For example, when calculating IoU as the degree of alignment between the micro-images MC11 and MC21, the management unit 120 binarizes the micro-images MC11 and MC21, and then digitizes the overlapping areas of the specimens SP1 and SP2 displayed in the micro-images MC11 and MC21 by calculating IoU. Note that in the case of the micro-images MC12 and MC22 executed in a process where N = 2, and in the case of the micro-images MC13 and MC23 executed in a process where N = 3, IoU is calculated using the same process as for the micro-images MC11 and MC21.

[0072] Next, the management unit 120 compares the IoU with a predetermined reference value (S16A). Note that the reference value used in step S16A does not necessarily match the reference value used in step S9A. It is preferable that the reference value in step S16A is a stricter value (a value closer to 1) than the reference value used in step S9A, i.e., the allowable deviation is smaller. The reference value may also be changed depending on the magnification of the objective lens 31 used to acquire the micro image. The higher the magnification, the stricter the reference value.

[0073] If the IoU is equal to or less than the reference value, the amount of deviation between the micro-images MC11 and MC21 stored in the storage area 110 is calculated (S17). The management unit 120 extracts multiple feature points (coordinates indicating positions within the images) predicted to be common to the micro-images MC1 and MC2, measures the deviation between the multiple feature points (coordinates indicating positions within the images) of the micro-image MC1 and the multiple feature points (coordinates indicating positions within the images) predicted to correspond to these feature points of the micro-image MC2, and stores the obtained deviation amount in the storage area 110. The method for calculating the deviation amount between the micro-images is the same as in step S8. The management unit 120 corrects the coordinates of the position indicating the region of interest A11 in the specimen SP2 based on the deviation amount, and controls the positional relationship with the optical system P2 by moving the stage 32 on which the specimen SP2 is placed within the XY plane.

[0074] Furthermore, the management unit 120 focuses the objective lens 31 on the specimen SP2, and uses the imaging unit 39 to image the area of ​​interest A11 in the specimen SP2 corrected in step S17, and re-acquires the micro image MC21 and stores it in the memory area 110 (S18).

[0075] In the next step S19, the management unit 120 calculates IoU as the degree of alignment between the micro image MC11 and the micro image MC21 acquired in step S18. The calculation method is the same as in step S9. The management unit 120 binarizes the micro image MC11 and the micro image MC21 acquired in step S18, and further digitizes the overlapping areas of the specimens SP1 and SP2 displayed in the micro images MC11 and MC21 by calculating IoU.

[0076] Next, the management unit 120 compares the IoU with a predetermined reference value (S19A). Note that the reference value used in step S19A does not necessarily match the reference value used in steps S9A and S16A. It is preferable that the reference value in step S19A be a stricter value than the reference value used in step S9A, i.e., the allowable deviation amount is smaller.

[0077] After step S16A, step S20 may be performed without performing (omitting) steps S17 to S19.

[0078] If IoU is equal to or less than the reference value, manual alignment is performed in step S20 and subsequent steps (S20). The management unit 120 switches to manual correction mode, reads out the micro image MC11 from the storage area 110, and outputs it to the output device 40. The management unit 120 then captures an image of the area of ​​interest A11 of the specimen SP2 placed on the stage 32 via the optical system P2 using the imaging unit 30 to obtain a micro image MC21, which is then displayed on the output device 40 in real time. At this time, as shown in FIG. 11 , the micro images MC11 and MC21 are displayed side by side on the output device 40. Furthermore, since the orientation (angle) of the specimen on the glass slide differs for each glass slide, the management unit 120 tilts the micro image MC11 by the amount of angle deviation to correspond to the orientation (angle) of the micro image MC12, as shown in FIG. 6 , so that the user can easily compare the micro images displayed on the output device 40.

[0079] While visually viewing the microimages MC11 and MC21 on the output device 40, the user moves the microimage MC21 displayed in real time on the output device 40 by a predetermined amount using the input device 50, for example, by dragging it, so that the area of ​​interest A11 shown in the microimage MC11 and the area of ​​interest A11 shown in the microimage MC21 become the same (S21). When the microimage MC21 is dragged by the input device 50 by a predetermined amount, the management unit 120 moves the stage 32 in the XY plane in conjunction with the microimage MC21 (the amount of movement of the image is converted into the amount of movement of the stage). As a result, the positional relationship between the stage 32 (specimen SP2) and the optical system P2 changes, and the area of ​​interest A11 in the specimen SP2 imaged by the imaging unit 30 via the optical system P2 changes, i.e., the microimage MC21 changes. Furthermore, the user changes (rotates) the micro image MC11 by a predetermined angle by dragging it using the input device 50 so that the orientation of the attention area A11 shown in the micro image MC11 and the orientation of the attention area A11 shown in the changed micro image MC21 become the same. When the micro image MC11 is dragged by a predetermined angle using the input device 50, the management unit 120 changes the angle of the micro image MC11 in conjunction with this. The user performs operations until the attention area A11 shown in the micro image MC11 and the attention area A11 shown in the micro image MC21 become the same. Note that the stage 32 may be provided with not only a movement mechanism within the XY plane but also a rotation mechanism, so that when the micro image MC21 is dragged by a predetermined amount using the input device 50, the management unit 120 moves and rotates the stage 32 within the XY plane in conjunction with this.

[0080] After the user manually aligns the micro image MC11 in step S21, the management unit 120 calculates the amount of deviation between the attention area A11 shown in the micro image MC11 and the attention area A11 shown in the new micro image MC21 (S22).

[0081] Furthermore, the management unit 120 acquires the micro image MC21 again and stores it in the storage area 110 (S23). In this way, the micro image MC21 of the new area of ​​interest A11 in the specimen SP2 is acquired again.

[0082] In step S24A, it is determined whether micro images of all the regions of interest (n locations) have been acquired. For example, if only micro image MC21 of region of interest A11 has been acquired, and if the management unit 120 determines in step S16A that IoU is equal to or greater than the reference value, the misalignment amounts of macro images MA1 and MA2 calculated in step S8 or overwritten in step S12 are overwritten as they are. On the other hand, if the misalignment amounts of micro images MC11 and MC21 are calculated in step S17 or step S22, the misalignment amounts of macro images MA1 and MA2 calculated in step S8 or overwritten in step S12 are overwritten with the calculated misalignment amounts (S24). In this way, the misalignment amounts of macro images MA1 and MA2 calculated in step S8 or overwritten in step S12 are corrected using the smaller misalignment amount (FIG. 8 "Reflected in macro alignment"). Although the displacement amount calculated for the first region of interest A11 is overwritten, the average value of the displacement amounts calculated for the first and second (multiple) regions of interest may be overwritten as the displacement amount.

[0083] In step S13, the management unit 120 overwrites the displacement calculated in this process, thereby updating the displacement stored in the memory area 110. This updates the registered positions of the regions of interest A11-13 in the specimen SP2. After that, in step S14 and subsequent steps, the value of N is incremented to N=2, and processing begins to acquire the micro-image M22 corresponding to the next region of interest A12.

[0084] If two or more micro images have been acquired, the management unit 120 returns the process to step S14, increments the value of N, and starts the process of acquiring a micro image corresponding to the next region of interest. For example, if the second micro image MC22 has been acquired in step S24A, the management unit 120 returns the process to step S14, sets N to 3, and starts the process of acquiring a third micro image MC23 corresponding to the region of interest A13.

[0085] Also, when micro images of all areas of interest (n locations) have been acquired in step S24A, in this example, when all micro images MC21-MC23 of areas of interest A11-A13 have been acquired, the slide glass holding the second specimen SP2 is removed from the stage 32.

[0086] If it is determined in step S25A that specimen SP2 is not the last specimen, the management unit 120 returns the process to step S6, and starts image acquisition processing for the third specimen S3 with m set to 3. In this way, the value of m is incremented each time the process passes from step S25A to step S6, and the processing from step S6 to S25 is executed sequentially for each specimen SP3, SP4, SP5, ...

[0087] If it is determined in step S25A that the processing for the last specimen has been completed, the management unit 120 executes processing to display the micro images side by side on the output device 40 (S26), as shown in Fig. 12. In Fig. 12, micro images MC11-MC101 of the region of interest A11 in each of the specimens SP1-S10 are displayed side by side. Note that the micro images MC11-MC101 can be digitally zoomed at any position within the image.

[0088] Alternatively, in S26, the management unit 120 executes a process of displaying the micro images superimposed on the output device 40 as shown in Fig. 10. In Fig. 10, the micro images MC11 and MC21 are displayed superimposed in the depth direction of the screen of the output device 40 (the depth direction of the paper in the figure).

[0089] As shown in FIG. 12, microimages MC21-MC101 are images acquired for the region of interest A11 of the mth specimen SPm corresponding to the region of interest A11 shown in microimage MC11, through correction or alignment based on the aforementioned amount of deviation. Furthermore, the angles of the microimages MC21-MC101 are changed relative to the microimage MC11, taking into account the angular (rotational) deviation between the orientation of the first specimen SP1 and the orientation of the mth specimen SPm. Therefore, each microimage is displayed with no or reduced deviation between the regions of interest between the images. The user can accurately compare each microimage MC11-MC101 while viewing the display on the output device 40.

[0090] 12, the management unit 120 displays on the output device 40 all of the micro images (MC11-MC101) of the area of ​​interest A11 (the number 1 displayed on the macro image MA1) specified by the user via the input device 50 among the areas of interest A11, A12, and A13 (corresponding to the numbers 1, 2, and 3 displayed on the macro image MA1, respectively) of the first macro image MA1 displayed in the lower right corner of the output device 40. When the management unit 120 displays on the output device 50 all of the micro images (MC11-MC101) corresponding to the areas of interest specified by the user via the input device 50, as shown in FIG. 12, the management unit 120 executes a process of displaying a mask (a rectangular mask that covers the display range of the micro image) over the number displayed on the first macro image MA1 that corresponds to the area of ​​interest (S27).

[0091] In addition, the process of indicating the positions of the attention areas A11, A12, and A13 on the macro image MA1 and superimposing a rectangular graphic mask display MS that covers the display range of the micro image on any of the attention areas A11, A12, and A13 (corresponding to the numbers 1, 2, and 3 displayed on the macro image MA1, respectively) may be performed on the attention areas that have been micro-observed (image acquired) by the user.

[0092] By checking the mask display MS, the user can distinguish between, for example, areas of interest for which image display (or observation) has been performed and areas of interest for which image display (or observation) has not yet been completed.

[0093] 13 and 14, the user can also display various annotations AN. Annotations AN can be numerical values ​​indicating the distance between two points or various shapes. Note that the user may also be able to display notes on each micro image and write text.

[0094] Note that the user may be able to pre-set the process shown in FIGS. 7A-C so that the IoU calculation (evaluation) is omitted. That is, the user may be able to pre-set the management unit 120 not to calculate IoU (Intersection over Union) by operating the input device 40. In this case, after step S8 in FIGS. 7A-C, steps S9 to S12 are not performed (omitted), and the process proceeds to step S13. Next, as one method, after step S15, step S16 is not performed (omitted), and the process proceeds to step S17; after step S18, step S19 is not performed (omitted), and the process proceeds to step S20. As another method, after step S15, steps S16 to S23 are not performed (omitted), and the process proceeds to step S24A. Furthermore, as another method, after step S15, step S16 is not performed (omitted), and the process proceeds to step S17; after step S18, step S19 is not performed (omitted), and the process proceeds to step S24A. After step S24A, in step S24, the management unit 120 overwrites the amounts of deviation between the macro images MA1 and MA2 calculated in step S8 as they are.

[0095] <Effects> The following aspects are disclosed in the embodiments.

[0096] (Aspect 1) The microscope device 30 includes an optical system P1 (corresponding to the first observation optical system) that acquires macro images, an optical system P2 (corresponding to the second observation optical system) that is different from the optical system P1 and acquires micro images, and an information processing device 10 (corresponding to the control unit) that activates the management unit 120.

[0097] The management unit 120 executes a process (S1) of acquiring a macro image MA1 of the specimen SP1 using the optical system P1, an acquisition process (S3) of acquiring first position information indicating the position of the areas of interest A11-A13 within the macro image MA1, and a process (S4) of controlling the positional relationship between the stage 32 on which the specimen SP1 is placed and the optical system P2 based on the first position information, and acquiring micro images MC11-MC13 of the areas of interest A11-A13 using the optical system P2.

[0098] In addition, the management unit 120 performs a process (S6) of acquiring a macro image MA2 of a specimen SP2 different from the specimen SP1 using the optical system P1, and a process (S13) of acquiring second position information in the macro image MA2 based on the macro images MA1 and MA2, the second position information indicating the position of the attention area A11-A13 corresponding to the first position information.

[0099] In addition, based on the second position information, the management unit 120 controls the relative position between the stage 32 on which the specimen SP2 is placed and the optical system P2, and performs a process (S15, S18) of acquiring micro images MC21-MC23 of the target areas A11-A13 in the specimen SP2 using the optical system P2.

[0100] With the above configuration, the positions of the regions of interest A11-A13 in the specimen SP2 can be accurately determined based on the positional deviation of the macro images MA1 and MA2. Therefore, the micro images MC21-MC23 can be captured without deviation or with only a small amount of deviation relative to the micro images MC11-MC13, respectively. In this way, accurate image acquisition is possible.

[0101] (Aspect 2) In aspect 1, the management unit 120 executes a process (S8, S12) to calculate the macro shift amount, which is the amount of positional shift between macro image MA1 and macro image MA2, and acquires second position information based on the macro shift amount (S13).

[0102] In the above configuration, the positions of the regions of interest A11-A13 in the specimen SP2 can be accurately determined based on the positional deviation between the macro images MA1 and MA2.

[0103] (Aspect 3) In aspect 1 or 2, the management unit 120 acquires the first location information in response to an instruction from the input device 50 (corresponding to the input unit) in the acquisition process.

[0104] In the above configuration, the user can use the input device 50 to easily specify the attention areas A11-A13.

[0105] (Aspect 4) In any of Aspects 1 to 3, the resolution of each of the macro images MA1 and MA2 is lower than the resolution of any of the micro images MC11-MC13 and MC21-MC23.

[0106] With the above configuration, macro images can be quickly acquired and processed, making it possible to quickly grasp the overall image of the specimens SP1, S2, etc.

[0107] (Aspect 5) In any of Aspects 1 to 4, the macro images MA1 and MA2 are each acquired simultaneously by the optical system P1.

[0108] Since the macro images MA1 and MA2 are acquired collectively, each image can be acquired quickly in the above configuration. Furthermore, since the collectively acquired macro images MA1 and MA2 have a smaller capacity than macro images acquired by dividing them, it is possible to quickly calculate the amount of deviation between the macro images MA1 and MA2 and correct the position of the macro image MA2 relative to the macro image MA1 (information processing) based on the calculated amount of deviation.

[0109] (Aspect 6) In any of aspects 1 to 5, the management unit 120 calculates the amount of deviation (micro deviation) in the micro images MC11 and MC21, and further executes a process (S24) to correct the amount of deviation (macro deviation) between the macro images MA1 and MA2 based on this amount of deviation.

[0110] In the above configuration, the highly accurate micro-shift amount from the micro-images MC11 and MC21 is used, so the macro-shift of the macro-images MA1 and MA2 can be corrected more accurately.

[0111] (Aspect 7) In any of Aspects 1 to 6, at least attention area A11 (first attention area 1) and attention area A12 (first attention area 2) are present in macro image MA1. The management unit 120 acquires first position information 1 indicating the position of attention area A11 and first position information 2 indicating the position of attention area A12 (S3). The management unit 120 further acquires second position information 1 indicating the position of attention area A11 (second attention area 1) corresponding to the first position information 1, and second position information 2 indicating the position of attention area A12 (second attention area 2) corresponding to the first position information 2, from macro image MA2 (S13). The micro-shift amount is calculated based on the micro-image MC11 of the first attention area 1 and the micro-image MC12 of the second attention area 1. Based on the corrected macro deviation amount, a process (S13) is performed to correct the position information (second position information part 2) of the area of ​​interest A12 in the macro image MA2, and a process (S15) is performed to acquire the micro image MC22 corresponding to the second position information part 2 based on this corrected position information.

[0112] (Aspect 8) In any of Aspects 1 to 7, the management unit 120 calculates a micro-shift amount, which is the amount of positional shift between the micro-image MC11 and the micro-image MC21, and further executes a process of controlling the positional relationship between the stage 32 on which the specimen SP2 is placed and the second optical system P2 based on the micro-shift amount (S17).

[0113] With the above configuration, micro images MC11 and MC21 with little deviation can be acquired.

[0114] (Aspect 9) In any of aspects 1 to 8, the management unit 120 reads out the micro image MC11 stored in the memory area 110 of the main memory device 102 or the auxiliary memory device 103, displays it on the output device 40 (corresponding to the display unit), and further executes a process (S20) of displaying the micro image MC21 being acquired by the optical system P2 on the output device 40.

[0115] With the above configuration, the user can easily align the two micro images MC11 and MC21 while checking the alignment on the screen.

[0116] (Aspect 10) In any of aspects 1 to 9, the management unit 120 controls the positional relationship between the stage 32 on which the specimen SP2 is placed and the optical system P2 based on instructions from the input device 50 (corresponding to the input unit), and further executes a process (S21) of moving the area of ​​interest A11 of the micro image MC21 displayed on the output device 40.

[0117] With the above configuration, the area of ​​interest A11 of the micro image MC21 can be moved quickly.

[0118] (Aspect 11) In any of aspects 1 to 10, the management unit 120, in response to instructions from the input device 50, further executes processes (S10, S11) to increase the transparency of at least one of the macro images MA1 and MA2 and display it on the output device 40, and change the overlap of the specimens SP1 and SP2 displayed in the macro images MA1 and MA2.

[0119] With the above configuration, the misalignment between the macro images MA1 and MA2 can be easily visually recognized and corrected.

[0120] (Aspect 12) In any of aspects 1 to 11, the management unit 120 executes a process of calculating IoU, which is the degree of alignment of the macro images MA1 and MA2, based on the amount of deviation between the macro images (macro deviation amount) (S9).

[0121] In the above configuration, the degree of alignment can be measured by calculating IoU.

[0122] (Aspect 13) In any of Aspects 1 to 12, if the calculated IoU is equal to or less than a preset reference value, the management unit 120 increases the transparency of at least one of the macro images MA1 and MA2 in response to an instruction from the input device 50 and displays them on the output device 40, and further executes processing (S10, S11) to display the specimens SP1 and SP2 displayed in the macro images MA1 and MA2 so that they overlap, and again calculates the macro deviation amount after processing (S12).

[0123] In the above configuration, the degree of alignment can be measured by calculating IoU.

[0124] (Aspect 14) In any of aspects 1 to 13, the management unit 120 further executes a process (S16, S19) of calculating IoU, which is the degree of alignment of the micro-images MC11, MC21, the micro-images MC12, MC22, or the micro-images MC13, MC23.

[0125] (Aspect 15) In any of aspects 1 to 14, if the calculated IoU is equal to or less than a predetermined reference value, the management unit 120 reads out the micro image MC11 stored in the memory area 110 and displays it on the output device 40, and also displays the micro image MC21 being acquired by the optical system P2 on the output device 40, and controls the positional relationship between the stage 32 on which the specimen SP2 is placed and the optical system P2 based on instructions from the input device 50, and further performs processing to move the area of ​​interest A11 of the micro image MC21 displayed on the output device 40 (S20, S21).

[0126] With the above configuration, the area of ​​interest A11 of the micro image MC21 can be easily moved.

[0127] (Aspect 16) In any of aspects 1 to 15, the management unit 120 further executes a process (S26) of displaying the micro images side by side.

[0128] In the above configuration, the user can easily compare and confirm the regions of interest A11-A13 between specimens.

[0129] (Aspect 17) In any of aspects 1 to 16, when the management unit 120 receives a process to acquire micro images MC11 and MC21 or a process to display the micro images side by side, it further executes a process to perform a predetermined display on the area of ​​interest A11 on the macro image MA1 displayed on the display unit (S27).

[0130] With the above configuration, the user can distinguish between, for example, regions of interest that have been observed and regions of interest that have not yet been observed.

[0131] 1 System 10 Information processing device 20 Optical device 30 Microscope device

Claims

1. A microscope comprising: a first observation optical system that acquires macro images; a second observation optical system that acquires micro images different from the first observation optical system; and a control unit, wherein the control unit executes the following processes: a process of acquiring a first macro image of a first specimen using the first observation optical system; an acquisition process of acquiring first position information that indicates the position of a first region of interest within the first macro image; a process of controlling the positional relationship between a stage on which the first specimen is placed and the second observation optical system based on the first position information, and acquiring a first micro image of the first region of interest using the second observation optical system; a process of acquiring a second macro image of a second specimen that is different from the first specimen using the first observation optical system; a process of acquiring second position information that indicates the position of a second region of interest in the second macro image that corresponds to the first position information, based on the first macro image and the second macro image; and a process of controlling the positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the second position information, and acquiring a second micro image of the second region of interest using the second observation optical system.

2. The microscope of claim 1, wherein the control unit calculates a macro shift amount, which is the amount of positional shift between the first macro image and the second macro image, in the acquisition process, and acquires the second position information indicating the position of the second region of interest based on the macro shift amount.

3. The microscope according to claim 1, wherein the control unit acquires the first position information in response to an instruction from an input unit in the acquisition process.

4. A microscope according to any one of claims 1 to 3, wherein the resolution of each of the first macro image and the second macro image is lower than the resolution of both the first micro image and the second micro image.

5. A microscope according to any one of claims 1 to 4, wherein the first macro image and the second macro image are acquired simultaneously by the first observation optical system.

6. A microscope as described in any one of claims 1 to 5, wherein the control unit calculates a micro-shift amount, which is the amount of positional shift between the first micro-image and the second micro-image, and further performs a process of correcting a macro-shift amount, which is the amount of positional shift between the first macro-image and the second macro-image, based on the micro-shift amount.

7. The microscope of claim 6, wherein the first region of interest exists in the first macro image as at least a first region of interest #1 and a first region of interest #2, the control unit acquires first position information #1 indicating the position of the first region of interest #1 and first position information #2 indicating the position of the first region of interest #2, and acquires, in the second macro image, second position information #1 indicating the position of the second region of interest #1 corresponding to the first position information #1 and second position information #2 indicating the position of the second region of interest #2 corresponding to the first position information #2, the micro-shift amount is calculated based on the first micro-image of the first region of interest #1 and the second micro-image of the second region of interest #1, and executes a process of correcting the second position information #2 in the second macro image based on the corrected macro-shift amount, and a process of acquiring the second micro-image of the second region of interest #2 based on the corrected second position information #2.

8. A microscope as claimed in any one of claims 1 to 5, wherein the control unit calculates a micro-shift amount, which is the amount of positional shift between the first micro-image and the second micro-image, and further performs processing to control the positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the micro-shift amount.

9. A microscope as described in any one of claims 1 to 8, wherein the control unit reads out the first micro-image stored in the memory unit and displays it on the display unit, and further performs processing to display on the display unit the second micro-image that is being acquired by the second observation optical system.

10. The microscope of claim 9, wherein the control unit controls the positional relationship between the stage on which the second specimen is placed and the second observation optical system in response to instructions from the input unit, and further performs processing to move the second region of interest of the second micro-image displayed on the display unit.

11. A microscope as claimed in any one of claims 1 to 10, wherein the control unit, in response to an instruction from an input unit, increases the transparency of at least one of the first macro image and the second macro image and displays it on the display unit, and further performs processing to change the overlap between the first specimen displayed in the first macro image and the second specimen displayed in the second macro image.

12. The microscope according to claim 2, wherein the control unit further executes a process of calculating the degree of alignment between the first macro image and the second macro image based on the amount of macro deviation.

13. The microscope of claim 12, wherein the control unit, when the calculated value is equal to or less than a preset reference value, increases the transparency of at least one of the first macro image and the second macro image and displays them on the display unit in response to an instruction from the input unit, and further performs display processing such that the first specimen displayed in the first macro image and the second specimen displayed in the second macro image overlap, and calculates the macro shift amount after the display processing.

14. The microscope according to claim 6 or 7, wherein the control unit further executes a process of calculating the degree of alignment between the first micro-image and the second micro-image.

15. A microscope as described in any one of claims 1 to 14, wherein the control unit, when the calculated value is equal to or less than a predetermined reference value, reads out the first micro-image stored in the memory unit and displays it on the display unit, and also displays the second micro-image being acquired by the second observation optical system on the display unit, and further controls the positional relationship between the stage on which the second specimen is placed and the second observation optical system according to instructions from the input unit, and performs processing to move the second region of interest of the second micro-image displayed on the display unit.

16. The microscope according to any one of claims 1 to 15, wherein the control unit further executes a process of displaying the first micro-image and the second micro-image side by side.

17. A microscope as described in any one of claims 1 to 16, wherein the control unit, when receiving the process of acquiring the first micro image and the second micro image or the process of displaying them side by side, further executes a process of performing a predetermined display on the first region of interest on the first macro image displayed on the display unit.

18. A method for a microscope equipped with a first observation optical system for acquiring a macro image, a second observation optical system different from the first observation optical system for acquiring a micro image, and a control unit, the method being executed by the control unit, the method comprising: a process of acquiring a first macro image of a first specimen using the first observation optical system; an acquisition process of acquiring first position information indicating the position of a first region of interest in the first macro image; a process of controlling the positional relationship between a stage on which the first specimen is placed and the second observation optical system based on the first position information, and acquiring a first micro image of the first region of interest using the second observation optical system; a process of acquiring a second macro image of a second specimen different from the first specimen using the first observation optical system; a process of calculating a macro shift amount which is the amount of positional shift between the first macro image and the second macro image based on the macro shift amount; a process of acquiring second position information indicating the position of a second region of interest in the second macro image which corresponds to the first position information, and a process of controlling the positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the second position information, and acquiring a second micro image of the second region of interest using the second observation optical system. A method comprising:

19. A program for a microscope equipped with a first observation optical system for acquiring macro images, a second observation optical system different from the first observation optical system for acquiring micro images, and a control unit, which causes the control unit to execute the following processes: a process for acquiring a first macro image of a first specimen using the first observation optical system; an acquisition process for acquiring first position information indicating the position of a first region of interest within the first macro image; a process for controlling the positional relationship between a stage on which the first specimen is placed and the second observation optical system based on the first position information, and acquiring a first micro image of the first region of interest using the second observation optical system; a process for acquiring a second macro image of a second specimen different from the first specimen using the first observation optical system; a process for calculating a macro shift amount which is the amount of positional shift between the first macro image and the second macro image based on the macro shift amount; a process for acquiring second position information indicating the position of a second region of interest in the second macro image which corresponds to the first position information; and a process for controlling the positional relationship between the stage on which the second specimen is placed and the second observation optical system based on the second position information, and acquiring a second micro image of the second region of interest using the second observation optical system.

Images