Magnification observation device, magnified image observation method, magnified image observation program, computer-readable recording medium, and storage device

Abstract

To reduce stress during execution of autofocus.SOLUTION: An enlarging observation device 100 comprises: a focusing degree evaluation unit 91 that calculates a focusing degree feature quantity indicating a degree of focusing of image data indicating an image of an observation object formed through an objective lens unit 25; a focus adjustment mechanism that moves a relative distance between a focal position of the objective lens unit 25 and the observation object in either a direction in which both approach each other or a direction in which both separate from each other along an optical axis of the objective lens unit 25 to adjust the focus of the image data; and a frame skip unit 92 that skips update of live display on a display unit 70 performed by a display control unit 52 on the basis of, comparison between the focusing degree feature quantity of the image data after the movement performed by the focus adjustment mechanism and the focusing degree feature quantity of the image data before the movement displayed on the display unit 70, which are sequentially calculated by the focusing degree evaluation unit 91 when the focus adjustment mechanism performs the movement in either the direction in which both approach each other or the direction in which both separate from each other along the optical axis direction.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a magnification observation device, a magnified image observation method, a magnified image observation program, a computer-readable recording medium, and a device storing the program. [Background technology]

[0002] Optical microscopes using optical lenses and digital microscopes are used as magnification observation devices for magnifying and displaying subjects such as microscopic specimens and workpieces. Digital microscopes receive reflected or transmitted light from an observation object placed on an XY stage, which is incident through the optical system of the objective lens, using an image sensor such as a CCD or CMOS that electrically reads each pixel arranged two-dimensionally, and then displays the electrically read image on a display. Some such digital microscopes are equipped with an autofocus function that adjusts the distance in the Z direction so that the focal point of the objective lens is on the object on the XY stage, thereby obtaining a focused image (see, for example, Patent Document 1).

[0003] In Patent Document 1, the movement of the imaging field of view is detected, and when the movement of the XY stage is completed, the relative height of the objective lens unit and the object to be measured is changed as needed to acquire an all-in-focus image. Because the entire series of operations is performed automatically, the user can quickly observe by simply moving the imaging field of view to the desired observation position.

[0004] When generating such an all-points image, the relative height at which the focus value of each pixel is maximized within a certain height range is searched for, resulting in out-of-focus and in-focus images being displayed alternately.

[0005] However, if the image is in focus to a certain extent when the imaging field of view is aligned with the measurement position, when the autofocus is automatically performed, the image that was in focus to a certain extent will suddenly become out of focus. In this case, from the user's perspective, it may seem as if the image that was in focus has been deliberately made out of focus, which can be frustrating for the user. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-127771 Summary of the Invention [Problem to be solved by the invention]

[0007] An object of the present invention is to provide a magnification observation device, a magnified image observation method, a magnified image observation program, a computer-readable recording medium, and a device storing the same, which reduce stress when performing autofocus.

[0008] A magnification observation device according to one aspect of the present invention includes a stage unit for placing an observation object thereon, an objective lens unit arranged facing the observation object on the stage unit, a camera unit that captures an image of the observation object formed through the objective lens unit and generates image data representing the image, a field of view moving mechanism that changes the relative position of the optical axis of the objective lens unit on the stage unit and moves the stage unit so as to move the observation field of the camera unit, a display control unit that displays an image of the observation object on a display unit based on the image data generated by the camera unit, and a focus degree evaluation unit that calculates a focus degree feature amount that indicates a focus degree of image data representing the image of the observation object formed through the objective lens unit. At least one of the stage unit and the objective lens unit is moved along the optical axis of the objective lens unit, The relative distance between the focal position of the objective lens unit and the object to be observed is adjusted along the optical axis of the objective lens unit in either a direction toward or a direction away from each other. changea focus adjustment mechanism for adjusting the focus of the image data by adjusting the focus of the image data; The relative distance is Along the optical axis in either the near or far direction After the relative distance is changed in the direction, a post-movement focus-degree feature amount calculated by the focus-degree evaluation unit based on image data showing an image of the observation object formed through the objective lens unit is compared with a maximum focus-degree feature amount among focus-degree feature amounts calculated by the focus-degree evaluation unit based on image data showing an image of the observation object formed through the objective lens unit before the relative distance is changed in the direction by the focus adjustment mechanism, and if the post-movement focus-degree feature amount is smaller than the maximum focus-degree feature amount, The display control unit skips updating of the live display on the display unit. and discarding the post-movement focus-degree feature amount, and if the post-movement focus-degree feature amount is greater than the maximum focus-degree feature amount, updating the maximum focus-degree feature amount with the post-movement focus-degree feature amount. With the above configuration, when the focus is adjusted by the focus adjustment mechanism, an image with a better focus degree is obtained. to Since the display content is updated only when the camera is turned on, there is no stress caused by the display temporarily becoming out of focus when performing conventional autofocus, allowing for comfortable focus adjustment.

[0009] According to another aspect of the present invention, a magnified image observation method includes: capturing an image of an observation object placed on a stage unit with a camera unit via an objective lens unit and displaying the image on a display unit; changing the relative distance between the focal position of the objective lens unit and the observation object along the optical axis of the objective lens unit with a focus adjustment mechanism to adjust the focus of image data; and relatively moving the optical axis of the objective lens unit and the stage unit with a field movement mechanism so that the position of the optical axis of the objective lens unit on the stage unit changes and the observation field output to the display unit moves; and, in this magnified image observation method, an image of the observation field including the observation object is captured based on image data generated by the camera unit at the relative distance between the stage unit and the objective lens unit changed by the focus adjustment mechanism. The aforementioned a step of displaying the image on a display unit; a step of calculating a focus degree feature amount indicating a focus degree of image data showing an image of an observation object formed through the objective lens unit by a focus degree evaluation unit, and adjusting the focus of the image by the focus adjustment mechanism; the maximum of the focus degree feature amount calculated by the focus degree evaluation unit based on image data indicating an image of the observation object formed through the objective lens unit after the relative distance between the focal position of the objective lens unit and the observation object is changed along the optical axis of the objective lens unit by the focus adjustment mechanism, and the focus degree feature amount calculated by the focus degree evaluation unit based on image data indicating an image of the observation object formed through the objective lens unit before the relative distance between the focal position of the objective lens unit and the observation object is changed along the optical axis of the objective lens unit by the focus adjustment mechanism; skipping updating of the live display on the display unit based on a comparison with the focus degree feature amount; ofAs a result, when adjusting focus with the focus adjustment mechanism, the display content of the display unit is updated only when an image with an improved focus degree feature amount is obtained, which eliminates the stress of switching to a display that temporarily becomes out of focus when performing conventional autofocus, and realizes comfortable focus adjustment.

[0010] Furthermore, according to a magnified image observation program according to another aspect of the present invention, there is provided a magnified image observation program for operating a magnifying observation device comprising: a stage unit for placing an observation object on the stage unit; an objective lens unit arranged to face the observation object on the stage unit; a camera unit that captures an image of the observation object formed through the objective lens unit and generates image data representing the image; a display unit that displays an image of an observation field including the observation object based on the image data generated by the camera unit; a field of view moving mechanism that changes the position of the optical axis of the objective lens unit on the stage unit and relatively moves the optical axis of the objective lens unit and the stage unit so that the observation field output to the display unit moves; and a focus adjustment mechanism that changes the relative distance between the focal position of the objective lens unit and the observation object along the optical axis of the objective lens unit to adjust the focus of the image data, wherein the magnified image observation program operates a magnified image observation device comprising: a stage unit for placing an observation object on the stage unit; an objective lens unit that is arranged to face the observation object on the stage unit; The aforementioned a function of displaying the image on a display unit; a function of calculating a focus degree feature amount indicating a focus degree of image data showing an image of an observation object formed through the objective lens unit by a focus degree evaluation unit, and adjusting the focus of the image by the focus adjustment mechanism; the maximum of the focus degree feature amount calculated by the focus degree evaluation unit based on image data indicating an image of the observation object formed through the objective lens unit after the relative distance between the focal position of the objective lens unit and the observation object is changed along the optical axis of the objective lens unit by the focus adjustment mechanism, and the focus degree feature amount calculated by the focus degree evaluation unit based on image data indicating an image of the observation object formed through the objective lens unit before the relative distance between the focal position of the objective lens unit and the observation object is changed along the optical axis of the objective lens unit by the focus adjustment mechanism; The computer can be made to realize a function of skipping updating of the live display on the display unit based on a comparison with the focus-degree feature amount. With the above configuration, when adjusting focus with the focus adjustment mechanism, the display content on the display unit is updated only if an image with an improved focus-degree feature amount is obtained, thereby realizing comfortable focus adjustment without the stress of switching to a display that is temporarily out of focus when performing conventional autofocus.

[0011] Furthermore, according to another aspect of the present invention, a computer-readable recording medium or device on which the program is recorded stores the program. Recording media include magnetic disks such as CD-ROMs, CD-Rs, CD-RWs, flexible disks, magnetic tapes, MOs, DVD-ROMs, DVD-RAMs, DVD±Rs, DVD±RWs, HD DVDs (AODs), Blu-rays (product names), UHD BDs (product names), USB memory sticks, SSD memory sticks, optical disks, magneto-optical disks, semiconductor memories, and other media capable of storing programs. Programs may be distributed by being stored on the recording media, or by being downloaded via a network such as the Internet. Recording media also include devices capable of recording programs, such as general-purpose or dedicated devices on which the program is implemented in an executable form, such as software or firmware. Each process or function included in the program may be executed by computer-executable program software, or each process may be implemented by hardware such as a predetermined gate array (FPGA, ASIC), or by a combination of program software and partial hardware modules that implement some of the hardware elements. In this specification, a computer-readable medium also includes a non-transitory tangible medium or a transitory propagating signal. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a perspective view of the appearance of a magnification observation device according to a first embodiment. [Figure 2] FIG. 2 is a block diagram of the magnification observation device of FIG. [Figure 3] FIG. 2 is a schematic diagram illustrating a configuration of an illumination unit. [Figure 4] FIG. 1 is a schematic diagram of a ring illumination. [Figure 5] FIG. 10 is a schematic diagram showing a user interface screen of a magnified image observation program including a navigation area. [Figure 6]2 is a schematic diagram showing the relationship between a stage unit, an observation object, a navigation image, and an observation field of view. FIG. [Figure 7] 10 is a flowchart showing a procedure for registering a 3D navigation image. [Figure 8] FIG. 10 is a schematic diagram of a 3D navigation image registration screen. [Figure 9] FIG. 10 is a schematic diagram of a 3D navigation image registration screen. [Figure 10] FIG. 10 is a schematic diagram of a 3D navigation image registration screen. [Figure 11] 10 is a flowchart showing a normal autofocus procedure. [Figure 12] 10 is a flowchart showing a procedure for short-distance autofocus. [Figure 13] FIG. 10 is a block diagram of a magnification observation device according to a second embodiment. [Figure 14] FIG. 14A is a schematic side view showing the sequence of the head unit during movement, and FIG. 14B is a schematic side view showing the stopping sequence. [Figure 15] FIG. 10 is a schematic diagram of a 3D navigation image registration screen. [Figure 16] FIG. 10 is a schematic diagram of a 3D navigation image registration screen. [Figure 17] 10A to 10C are schematic side views showing a sequence during movement of the head unit in the general-purpose mode. [Figure 18] 10 is a flowchart showing details of focus tracking processing. [Figure 19] 10A and 10B are schematic side views showing sequences during movement in which focus tracking is performed when a 3D navigation image is present and when it is not present. [Figure 20] FIG. 10 is a schematic side view showing focus tracking with a focus jump function. [Figure 21] FIG. 10 is a schematic side view showing focus tracking with an offset function. [Figure 22] 22A is a schematic diagram showing a focus frame when autofocus is being performed, FIG. 22B is a schematic diagram showing a focus frame when autofocus is not being performed, and FIG. 22C is a schematic diagram showing a focus frame when autofocus is being performed using focus control different from that in FIG. 22A. [Figure 23] 10 is a timing chart showing the relationship between commands and operations during focus tracking. [Figure 24] FIG. 24A is a schematic perspective view showing the operation of the head unit in the plane tracking mode, and FIG. 24B is a graph showing the estimated tilt of the plane. [Figure 25] 10 is a user interface screen showing a plane fit function. [Figure 26] 10 is a flowchart showing a procedure for normal autofocus with frame skip. [Figure 27] 10 is a flowchart showing a procedure for short-distance autofocus with frame skip. [Figure 28] 10 is a flowchart showing a frame skip procedure. [Figure 29] FIG. 10 is a schematic perspective view showing the operation of the head unit during search autofocus. [Figure 30] 10 is a timing chart showing the operation of search autofocus. [Figure 31] 10 is a flowchart showing a procedure of a focus follow-up mode. [Figure 32] 10 is a flowchart showing a procedure for one-shot composition. [Figure 33] FIG. 10 is a front view showing a magnification observation device according to a third embodiment. [Figure 34] FIG. 10 is a side view showing a magnification observation device according to a fourth embodiment. [Figure 35] 35 is a schematic diagram showing a side camera image captured by the side camera unit of FIG. 34. FIG. [Figure 36] FIG. 10 is a schematic diagram illustrating depth image synthesis. [Figure 37] FIG. 10 is a schematic diagram illustrating whether or not offset processing can be performed. DETAILED DESCRIPTION OF THE INVENTION

[0013] Embodiments of the present invention will be described below with reference to the drawings. However, the embodiments described below exemplify a magnification observation device, a magnification image observation method, a magnification image observation program, a computer-readable recording medium, and a device storing the same, which embody the technical concepts of the present invention. The present invention does not specify the magnification observation device, the magnification image observation method, the magnification image observation program, the computer-readable recording medium, and the device storing the same. Furthermore, this specification does not specify the components described in the claims as those of the embodiments. The dimensions, materials, shapes, and relative positions of the components described in the embodiments, unless otherwise specified, are not intended to limit the scope of the present invention and are merely illustrative examples. The size and positional relationships of components shown in the drawings may be exaggerated for clarity. Furthermore, in the following description, identical names and symbols indicate identical or similar components, and detailed descriptions will be omitted as appropriate. Furthermore, the elements constituting the present invention may be configured with the same components, with one component serving multiple functions, or the functions of one component may be shared among multiple components.

[0014] The magnification observation device used in the embodiments of the present invention is connected to a computer, printer, external storage device, or other peripheral device connected thereto for operation, control, display, and other processing, for example, via a serial connection such as IEEE1394, RS-232x, RS-422, or USB, a parallel connection, or a network such as 10BASE-T, 100BASE-TX, or 1000BASE-T, and the connection is electrically, magnetically, or optically connected for communication. The connection is not limited to a physical connection using a wire, but may also be a wireless connection using radio waves, infrared rays, optical communications, or the like, such as a wireless LAN such as IEEE802.x or Bluetooth (registered trademark). Furthermore, recording media for exchanging data and saving settings may include memory cards, magnetic disks, optical disks, magneto-optical disks, semiconductor memories, and the like. Note that in this specification, the terms "magnification observation device" and "magnified image observation method" refer not only to the magnification observation device itself, but also to a magnification observation system that combines it with peripheral devices such as a computer and external storage device.

[0015] Furthermore, in this specification, the term "magnification observation device" is not limited to the system itself that performs magnification observation, nor to devices or methods that perform input / output, display, calculation, communication, and other processing related to imaging using hardware. Devices and methods that implement processing using software are also within the scope of the present invention. For example, devices and systems that incorporate software, programs, plug-ins, objects, libraries, applets, compilers, modules, macros that run on specific programs, etc. into general-purpose circuits or computers to enable imaging itself or related processing also fall under the category of the magnification observation device of the present invention. Furthermore, in this specification, the term "computer" includes not only general-purpose and dedicated electronic computers, but also workstations, terminals, and other electronic devices. Furthermore, in this specification, the term "program" is not limited to a standalone program, but can also be used in the following ways: as part of a specific computer program, software, service, etc.; ... [Embodiment 1]

[0016] A magnification observation device 100 according to a first embodiment of the present invention will be described below with reference to FIGS. 1 and 2. As shown in FIG. 1, the magnification observation device 100 is broadly divided into an imaging system 1 and a control system 2. The imaging system 1 includes an illumination unit 60 for illuminating an observation object WK, such as a specimen, sample, workpiece, or other subject, and a head unit 4 for capturing an image of the observation object WK illuminated by the illumination unit 60. As shown in FIG. 2, the head unit 4 includes a camera unit 10 including an image sensor 12 and a microscope lens unit 20 detachably attached to the tip of the camera unit 10. The microscope lens unit 20 constitutes an imaging optical system (lens optical system) 11 made up of multiple optical lenses. Here, the microscope lens unit 20 includes an objective lens unit 25 as shown in FIG. 3. The head unit 4 also functions as an imaging means for receiving reflected or transmitted light of the illumination light.

[0017] The magnification observation device also includes a focus adjustment mechanism. The focus adjustment mechanism adjusts the focus of image data by moving the relative distance between the focal position of an optical imaging system, such as the objective lens unit 25, and the object of observation either toward or away from each other along the optical axis of the optical imaging system. The focus adjustment mechanism is composed of, for example, an upper Z elevator 16, such as an upper Z stage, or a lower stage elevator 35, such as a lower Z stage. The focus adjustment mechanism may move the head unit 4 along the optical axis of the optical imaging system, or may move the objective lens unit 25 along the optical axis of the optical imaging system relative to the head unit 4. Alternatively, the focus adjustment mechanism may move a lens in the objective lens unit 25 along the optical axis of the optical imaging system. Furthermore, if the objective lens unit 25 includes a variable-focus lens, such as a liquid lens, the focus of the liquid lens may be controlled by a control circuit to move the relative distance between the focal position of the optical imaging system, such as the objective lens unit 25, and the object of observation either toward or away from each other along the optical axis of the optical imaging system. The focus adjustment mechanism includes a liquid lens or the like whose focus is controlled by a focus adjustment mechanism or a control circuit. (Camera section 10)

[0018] As shown in Fig. 3, the camera unit 10 includes an image sensor 12 that electrically reads reflected light incident via an imaging optical system 11 from an observation object WK illuminated by an illumination unit 60. In this example, the image sensor 12 uses a CMOS, but other light-receiving elements such as a CCD can also be used. The objective lens unit 25 is disposed facing the observation object on the stage unit 30. The camera unit 10 also captures an image of the observation object formed via the objective lens unit 25 and generates image data representing this image.

[0019] The imaging system 1 also includes a stage unit 30 on which the observation object WK is placed, an upper Z stage as a first focus adjustment unit that adjusts the focus by changing the relative distance in the optical axis direction between this stage unit 30 and the head unit 4, and an upper Z elevator 16 for driving this upper Z stage. Light is incident on the observation object WK placed on this stage unit 30 via an imaging optical system 11, and reflected light reflected by the observation object WK or transmitted light irradiated from the bottom side of the observation object WK is electrically read by an imaging element 12 of the camera unit 10.

[0020] The control system 2 further includes a main body 50 having a display 70 that displays an enlarged image captured by the camera 10. The camera 10 is connected to the main body 50 via a cable 3. The display 70 displays an image of the observation field including the object to be observed based on image data generated by the camera 10. In the example of FIG. 1, the display 70 is provided integrally with the main body 50, but the display may also be a separate member from the main body 50. For example, the main body 50 may include a display control unit 52 that generates the display content to be displayed on the display 70, as well as a display connection interface that connects the display 70 to the main body 50.

[0021] The main body 50 also includes a processor 80. As shown in the block diagram of FIG. 2, the processor 80 realizes multiple functions (details will be described later). The main body 50 can be a general-purpose computer with a dedicated program installed, or a specially designed device. In this example, a general-purpose computer with a magnified image observation program for operating the magnification observation device installed is used as the main body. The main body 50 includes the processor 80, a display control unit 52, a storage unit 53, an interface 54, an operation unit 55, and a memory unit 56.

[0022] The cable unit 3 is an electrical cable for transmitting image information obtained by the image sensor 12 of the camera unit 10 to the body unit 50. In addition to the electrical cable, the cable unit 3 may also include an optical cable for transmitting illumination light from the body unit 50 to the head unit 4. In this case, the cable unit 3 may be an integrated unit of an electrical cable and an optical cable, or these may be provided separately. (Display unit 70)

[0023] The display unit 70 can be a monitor such as an LCD display, organic EL display, or CRT. The main body 50 is also connected to an operation unit 55 that enables the user to perform various operations. The operation unit 55 is an input device such as a console or a mouse. In this example, the display unit 70 and operation unit 55 can be integrated into the main body 50 or can be external components. Furthermore, if the display unit 70 is configured as a touch panel, the display unit 70 and operation unit 55 can be configured as an integrated unit.

[0024] The operation unit 55 is connected to the main unit 50 or a computer via a wired or wireless connection, or is fixed to the computer. Typical operation units 55 include various pointing devices, such as a mouse, keyboard, slide pad, TrackPoint, tablet, joystick, console, jog dial, digitizer, light pen, numeric keypad, touchpad, and AccuPoint. These operation units 55 can be used not only to operate the magnification observation operation program, but also to operate the magnification observation device itself and its peripheral devices. Furthermore, a touch screen or touch panel can be used for the display itself that displays the interface screen, allowing the user to input and operate the screen by directly touching it with their hands, or voice input or other existing input means can be used, or both. In the example of Figure 1, the operation unit 55 is composed of a mouse, keyboard, and joystick 55b. (Lighting section 60)

[0025] The illumination unit 60 generates illumination light that illuminates the observation object WK that is imaged on the imaging element 12. A schematic configuration of the illumination unit 60 is shown in FIG. 3. The illumination unit 60 includes an illumination control unit 66. The illumination control unit 66 controls the illumination light according to set illumination conditions. The illumination conditions adjust the brightness of the illumination light emitted from the illumination unit 60. For example, the brightness of the illumination light can be adjusted by adjusting the irradiation time or irradiation intensity of the illumination unit 60. The illumination unit 60 may include an LED as a light source. The light emission of the light source may be controlled by the illumination control unit 66, or the brightness of the irradiated light may be controlled by a light-blocking member such as a shutter.

[0026] For illumination, epi-illumination such as ring illumination (dark-field illumination) or coaxial illumination (bright-field illumination) can be used. Furthermore, illumination methods such as transmitted illumination and differential interference illumination can be appropriately used for the illumination unit 60. Epi-illumination is an illumination method in which illumination light falls on the observation object from above, and includes ring illumination, coaxial illumination, etc. Coaxial illumination and ring illumination have different angles of incidence on the observation object, resulting in different shadows. These can be used depending on the observation object. Furthermore, the microscope lens unit 20 detachably attached to the tip of the camera unit 10 and the objective lens unit 25 detachably attached to the microscope lens unit 20 may be compatible with both coaxial illumination and ring illumination, only coaxial illumination, or only ring illumination. Trans-illumination is an illumination method in which illumination light is irradiated onto the observation object from below. In the example of FIG. 3, the illumination unit 60 includes a coaxial epi-illumination unit 62 and a ring illumination unit 63. The coaxial epi-illumination unit 62 is bent via a beam splitter 61 so as to be parallel to the optical axis AX of the objective lens unit 25. The illumination unit 60 can be either built into the head unit 4 or configured as a separate unit that can be detached from the head unit 4. The light source of the illumination unit 60 may be an LED or an LD, etc., provided in each of the coaxial epi-illumination unit 62 and the ring illumination unit 63, or a common illumination light source may be built into the main body unit 50 so that illumination light is transmitted to the illumination unit in the head unit via an optical cable.

[0027] Coaxial lighting and ring lighting can also be equipped with oblique lighting. An example of a coaxial lighting and ring lighting equipped with such oblique lighting functionality is shown in the schematic diagram of Figure 4. The lighting unit 60 shown in this figure comprises a circular coaxial epi-illumination unit 62 located on the inner ring side and a similarly circular ring lighting unit 63 located on the outer ring side. Both the coaxial epi-illumination unit 62 and the ring lighting unit 63 are divided into sections along the circumference, and by switching the lighting of each divided block, it is possible to provide oblique lighting with different lighting directions.

[0028] In this way, the illumination unit 60 is capable of switching the illumination direction in which illumination light is irradiated toward the observation object. Furthermore, the illumination unit 60 is capable of irradiating with either a first illumination pattern in which the illumination direction in which illumination light is irradiated is sequentially switched, or a second illumination pattern in which illumination is irradiated in one of the illumination directions constituting the first illumination pattern. The illumination pattern switching is performed by the illumination control unit 66.

[0029] In the example of FIG. 4 , the ring illumination unit 63 is divided into four illumination blocks 63a, 63b, 63c, and 63d, and each of the illumination blocks 63a to 63d can be individually lit. By turning on one of the illumination blocks 63a to 63d and turning off the others, the illumination direction in which the illumination light is irradiated toward the observation object can be changed. The number of divisions of the ring illumination unit is not limited to four, and may be three or less, five or more. This can be changed as appropriate depending on the observation purpose, etc. The lighting sequence of the illumination blocks 63a to 63d in the first illumination pattern is predetermined, and may be, for example, clockwise lighting or figure-eight lighting. The lighting sequence of the illumination blocks 63a to 63d in the first illumination pattern may be stored in the memory unit 56 as lighting sequence information. Furthermore, the first illumination pattern may be selected by sequentially lighting up each illumination block of the coaxial epi-illumination unit 62, sequentially lighting up each illumination block of the ring illumination unit 63, or sequentially lighting up each illumination block of the coaxial epi-illumination unit 62 and each illumination block of the ring illumination unit 63, depending on the microscope lens unit 20 and the objective lens unit 25. For example, ring illumination may be used preferentially. In this case, when an objective lens unit 25 compatible with only coaxial illumination is used, the coaxial epi-illumination unit 62 may be selected as illumination for the first illumination pattern, and when an objective lens unit 25 compatible with both coaxial illumination and ring illumination is used, the ring illumination unit 63 may be selected as illumination for the first illumination pattern. (Lighting control unit 66)

[0030] The illumination control unit 66 can switch the illumination light control between a first sequence and a second sequence. In the first sequence, the illumination unit 60 operates in a first illumination pattern. A live image of the observation object illuminated with the first illumination pattern is displayed on the display unit 70. This first sequence is also referred to as search lighting (details will be described later). In the second sequence, the illumination unit 60 operates in a second illumination pattern using one of the illumination directions selected based on the feature values ​​of each image data calculated by the feature calculation unit 88 (details will be described later). An image of the observation object illuminated with the second illumination pattern is displayed on the display unit 70. This makes it possible to automatically select and display an illumination direction that makes it easier to check the surface condition of the observation object. For example, by changing the illumination direction in real time, an observation environment is realized in which unevenness, scratches, dents, etc. on the surface of the observation object are less likely to be overlooked.

[0031] When the observation field of view is moved along the field of view movement trajectory, the illumination control unit 66 controls the illumination unit 60 to emit light with a first illumination pattern. This allows the illumination direction to be automatically changed while the observation field of view is moving, making it possible to observe the surface condition of the object under observation with different illumination, making it easier to find scratches, chips, etc. Furthermore, when the illumination control unit 66 stops the movement of the observation field of view by the field of view movement mechanism 5, it analyzes multiple image data with different illumination directions stored in the buffer memory 57, selects an image with an illumination direction in which the flaw is most clearly visible, and controls the illumination unit 60 to emit light with a second illumination pattern fixed to that illumination direction. As a result, when the movement of the observation field of view is stopped, image data with an illumination direction in which the flaw is most clearly visible is displayed, realizing observation that is ideal for flaw search.

[0032] The illumination unit 60 shown in FIG. 1 includes a coaxial incident illumination unit 62 (see FIG. 3) for irradiating the observation object WK with coaxial incident light, and a ring illumination unit 63 for irradiating ring-shaped illumination light from a ring-shaped light source. These illumination units are connected to the main body 50 via cables. The main body 50 is equipped with a connector for connecting the cables. The ring illumination unit 63 can switch between all-around illumination and side illumination. To achieve this, the ring illumination unit 63 can be configured to have multiple LEDs arranged in a ring shape and some of the LEDs can be turned on and off, or to have a turret-type mask that cuts off part of the illumination light. The illumination control unit 66 controls the lighting and switching of these illumination lights. (Field of view movement mechanism 5)

[0033] The magnification observation device 100 also includes a field-of-view moving mechanism 5 that moves the observation field displayed on the display unit 70. The field-of-view moving mechanism 5 changes the relative position between the objective lens unit 25 and the stage unit 30 to change the position of the optical axis AX of the objective lens unit 25 on the mounting surface of the stage unit 30. When the field-of-view moving mechanism 5 moves the relative position between the objective lens unit 25 and the stage unit 30, an updated image can be displayed on the display unit 70 in the observation field after the movement. In the example of FIG. 1, an XY stage that moves the stage unit 30 is used as the field-of-view moving mechanism 5. However, the present invention may also use a field-of-view moving mechanism that moves the objective lens unit instead of or in addition to this. It is sufficient for the field-of-view moving mechanism to be able to move the relative position of the objective lens unit and the stage unit as viewed from the optical axis side, i.e., the observation field within the XY plane.

[0034] The XY stage is an electrically driven stage that can move the mounting surface of the stage unit 30 in the X-axis direction and the Y-axis direction. The field of view moving mechanism 5 can also be provided with a θ stage that can rotate the stage unit 30.

[0035] Furthermore, in addition to being able to move the stage unit 30 in the XY plane by the field of view moving mechanism 5, the stage unit 30 can also be moved in the height direction, that is, in the Z direction, by the lower stage elevator 35.

[0036] The operation of the lower stage elevator 35 will now be described. The main body 50 changes the relative distance in the optical axis direction between the stage 30 and the head 4, which has the imaging optical system 11 and the imaging element 12, in this case, the height in the z direction, by inputting control data related to the control of the stepping motor 37 to the motor control circuit 36. Specifically, the main body 50 controls the rotation of the stepping motor 37 by inputting control data necessary for controlling the lower stage elevator 35 to the motor control circuit 36, thereby raising and lowering the height z (position in the z direction) of the stage 30. The stepping motor 37 generates a rotation signal corresponding to the rotation. Based on the rotation signal input via the motor control circuit 36, the main body 50 stores the height z of the stage 30 as information related to the relative distance in the optical axis direction between the stage 30 and the imaging optical system 11. The stage 30 functions as an observation positioning unit that positions the observation position with respect to the observation object WK.

[0037] Furthermore, in this embodiment, changing the height of the stage unit 30 not only changes the relative distance between the stage unit 30 and the imaging optical system 11 in the optical axis direction, but also changes the height of the imaging optical system, i.e., the height of the head unit 4. The head unit 4 is connected to the main body unit 50 by the cable unit 3. As a result, data acquired by the head unit 4 is sent to the main body unit 50 via the cable unit 3, and the necessary processing can be performed on the main body unit 50 side. Note that the stage unit may be provided in the microscope main body, or it may be provided in the head unit that is a separate member from the main body, or an imaging unit without a stage may be provided in the head unit. An imaging unit without a stage may be attached to a mounting stand or may be handheld by the user.

[0038] The image sensor 12 can electrically read the amount of received light for each pixel arranged two-dimensionally in the x and y directions. The image of the observation object WK formed on the image sensor 12 is converted into an electrical signal according to the amount of received light at each pixel of the image sensor 12, and the signal is further converted into digital data by the image sensor control circuit 13. The main body 50 stores the digital data converted by the image sensor control circuit 13 as received light data D in the storage unit 53 together with pixel arrangement information (x, y) as two-dimensional position information of the observation object WK in a plane (x, y directions in FIG. 2) approximately perpendicular to the optical axis direction (z direction in FIG. 2). Here, the plane approximately perpendicular to the optical axis direction does not necessarily have to be a plane strictly at 90° with respect to the optical axis; it may be an observation plane within a range of inclination that allows the shape of the observation object WK to be recognized with the imaging optical system and the resolution of the image sensor 12.

[0039] In the above explanation, an example of the stage unit 30 has been given in which the observation object WK is placed on the stage unit 30, but it is also possible to use a configuration in which, for example, an arm is attached instead of the stage unit and the observation object WK is fixed to the tip of the arm. Furthermore, the head unit 4 is not only used by being attached to the camera attachment unit 43, but can also be detachable and positioned at a desired position and angle by hand or other methods. (Movement direction instruction section 55a)

[0040] The control system 2 is provided with an operation unit 55. The operation unit 55 is an input device connected to the main body 50. The operation unit 55 functions as a movement direction indicator 55a that accepts user input indicating the movement direction of the observation field on the display unit 70. The movement direction of the field of view movement mechanism 5 is indicated according to the direction input from this operation unit 55. The operation unit 55 can be a joystick, a touchpad, a mouse, a keyboard (arrow keys or specific keys), or the like. In particular, by using a joystick 55b as the operation unit 55, the user can easily intuitively indicate the movement direction of the observation field by the direction in which he tilts the joystick 55b. The movement speed can also be specified by the angle at which the joystick 55b is tilted from the vertical position.

[0041] Furthermore, the stage unit 30 can be moved in the height direction, i.e., the Z direction, by the lower stage elevator 35, and also in a plane. Specifically, it is provided with an XY stage that can move in the X-axis direction and the Y-axis direction. It can also be provided with a rotatable stage (θ stage) that rotates the stage unit 30.

[0042] In this example, both the upper Z elevator 16 and the lower stage elevator 35 are electrically driven. However, in the present invention, it is sufficient to acquire the relative distance between the focal position of the objective lens unit 25 and the stage unit 30, and it is not essential that both the upper Z elevator and the lower stage elevator are electrically driven. For example, either the upper Z elevator or the lower stage elevator may be configured to be driven manually.

[0043] 2, the main body 50 includes a processor 80, a display control unit 52, a storage unit 53, an interface 54, an operation unit 55, and a memory unit 56. This magnification observation device 100 captures an observation image using an image sensor 12 that electrically reads reflected light or transmitted light from an observation object WK fixed to a stage unit 30 that is incident via an imaging optical system 11, and displays the image on a display unit 70.

[0044] The storage unit 53 functions as a memory unit, for example, by saving image data displayed on the display unit 70 by the display control unit 52 as a still image or a moving image. The storage unit 53 also saves image data of composite images generated by the image processing unit 84 for different observation objects, in association with condition data including one or more pieces of information relating to the conditions when the composite images were captured.

[0045] The interface 54 is a connection section that allows the main body section 50 to communicate data with the head section 4, the lower stage elevator 35, etc. The memory section 56 is composed of RAM, ROM, etc. This memory section 56 includes a buffer memory 57 that sequentially stores image data captured by the camera section 10 in different illumination directions while the field of view movement mechanism 5 is moving. The operation section 55 is also a member that sets the imaging conditions for setting the conditions for capturing an image with the camera section 10, as well as other necessary settings and operations.

[0046] The buffer memory 57 temporarily stores multiple pieces of image data for different lighting directions to be displayed on the display unit 70 in the first sequence. The image data buffered in the buffer memory 57 is used when determining the lighting direction, and is therefore stored in correspondence with or associated with each lighting direction. The size of the buffer memory 57 is sufficient to store image data for at least one cycle, and the image data may be overwritten for the next cycle. Here, one cycle refers to a period covering one cycle of each of the different types of lighting included in the first lighting pattern, and each of the different types of lighting included in the first lighting pattern is turned on at least once during one cycle. In other words, by buffering image data in the buffer memory 57 for a period of one cycle or more, it is possible to buffer sufficient image data for determining the lighting direction.

[0047] If multiple image data sets with different illumination directions are stored in the buffer memory 57 when the region is specified by the region designation unit, the feature calculation unit 88 calculates the feature values ​​of the image data within the specified region for each of the stored image data, selects one of the illumination directions, and transitions from the first sequence to the second sequence. Features of image data can be variances of brightness, contrast values, etc. This allows image data to be stored in the buffer memory 57 while sequentially switching illumination directions, and when the user specifies the desired region, the illumination direction can be selected using the already stored image data. This allows for rapid switching to an appropriate illumination direction without waiting for new image data to be captured. If, when the region is specified by the region designation unit, image data for a period less than one cycle is stored in the buffer memory 57 or no image data is stored, the first sequence is maintained until image data corresponding to one cycle can be buffered in the buffer memory 57. Then, the feature calculation unit 88 calculates the feature values ​​of the image data within the specified region for each of the image data sets stored in the buffer memory 57, selects one of the illumination directions, and transitions from the first sequence to the second sequence. (Display control unit 52)

[0048] The display control unit 52 outputs image data generated by the camera unit 10 to the display unit 70. The display unit 70 displays image data of the observation field output from the display control unit 52. This display control unit 52 can be configured with a GPU or the like. In the example of FIG. 2, an example is described in which the display control unit 52 is configured as a separate member from the processor unit. Such a display control unit 52 is configured with, for example, a GPU. However, the present invention is not limited to this configuration, and the display control unit 52 may be incorporated into the processor unit. For example, the display control unit 52 may be integrated into a CPU or MPU that constitutes the processor. (Processor unit 80)

[0049] The processor unit 80 realizes functions such as a movement control unit 83, an image processing unit 84, a focus control unit 90, a height information acquisition unit 89, a feature calculation unit 88, a judgment unit 97, and a one-shot synthesis unit 98. The movement control unit 83 controls the movement of the field of view movement mechanism 5 according to the movement direction instruction from the movement direction instruction unit 55a. The focus control unit 90 controls the focus adjustment mechanism to adjust the focus of the image. For example, it performs autofocus to automatically achieve focus. The focus control unit 90 realizes functions such as a focus degree evaluation unit 91, a frame skip unit 92, a movement stop detection unit 94, and a focus sequence execution unit 93. The focus degree evaluation unit 91 calculates a focus degree feature that indicates the focus degree of the image data displayed on the display unit 70 by the display control unit 52.

[0050] When the focus adjustment mechanism moves the image data in either the near or far direction along the optical axis, the frame skip unit 92 causes the display control unit 52 to skip updating of the live display on the display unit 70, based on a comparison of the focus degree feature amount of the image data after the movement by the focus adjustment mechanism, which is sequentially calculated by the focus degree evaluation unit 91, with the focus degree feature amount of the image data before the movement that is displayed on the display unit 70. As a result, when adjusting the focus with the focus adjustment mechanism, the display content on the display unit 70 is updated only if an image with an improved focus degree feature amount is obtained.

[0051] Furthermore, when adjusting the focus of an image with the focus adjustment mechanism, if the focus degree feature amount of the image data after movement by the focus adjustment mechanism, which is sequentially calculated by the focus degree evaluation unit 91, is worse than the focus degree feature amount of the image data before movement displayed on the display unit 70, the frame skip unit 92 does not cause the display control unit 52 to update the display content on the display unit 70. This achieves comfortable focus adjustment without the stress of switching to a display that is temporarily out of focus when conventional autofocus is performed.

[0052] Here, the focus degree feature amount is an index indicating the degree of focus of an image, and known parameters such as focus value and phase difference can be used. For example, a focus value such as contrast calculated based on image data can be used. In this case, when adjusting the focus of an image using the focus adjustment mechanism, the frame skip unit 92 causes the display control unit 52 to update the display content on the display unit 70 based on a comparison of the focus value of the image data after movement by the movement mechanism, which is sequentially calculated by the focus degree evaluation unit 91, with the focus value of the image data before movement, which is displayed on the display unit 70. This makes it possible to perform autofocus based on focus values ​​such as contrast obtained from image data without using a dedicated sensor or the like.

[0053] Furthermore, when a phase difference is used as the focus degree feature, an autofocus sensor is provided to detect the difference between the in-focus position and the current position. In this case, the frame skip unit 92 sequentially measures the phase difference using the autofocus sensor when adjusting the focus of the image using the focus adjustment mechanism. When the measurement value after movement by the focus adjustment mechanism is improved compared to the in-focus position compared to the measurement value displayed on the display unit 70 before the movement, the display control unit 52 updates the display content on the display unit 70. This method does not require movement in the Z direction, thereby achieving high-speed autofocus.

[0054] While the observation field is being moved by the field of view moving mechanism 5, the focus sequence execution unit 93 executes a focus sequence during field of view movement, which adjusts the relative distance between the focal position of the optical imaging system, such as the objective lens unit 25, and the observation object WK using the focus adjustment mechanism based on information regarding the XY position of the observation field, and causes the display control unit 52 to display a live image of the observation object on the display unit 70 based on image data obtained.

[0055] The movement stop detector 94 is a component that detects movement or stop of the observation field. When the movement stop detector 94 detects movement of the observation field, a focus sequence during field movement is executed, in which the focus adjustment mechanism adjusts the relative distance between the focal position of the optical imaging system, such as the objective lens unit 25, and the observation object WK based on at least one of image data captured and generated by the camera unit 10 while the observation field is moving and information related to the XY position of the observation field. Then, based on the image data captured and generated by the camera unit 10 while the observation field is moving, the display controller 52 displays a live image of the observation object WK on the display unit 70. On the other hand, when the movement stop detector 94 detects a stop of movement of the observation field, the focus sequence during field movement is terminated, and a stop sequence is executed, in which the focus adjustment mechanism stops adjusting the relative distance between the focal position of the optical imaging system, such as the objective lens unit 25, and the observation object WK. In this way, focus adjustment, which was previously not possible while the observation field was moving, can be achieved.

[0056] The movement stop detection unit 94 may detect both the movement state and the stop state of the observation field using a single component, or may be provided with a movement detection unit that detects the movement state of the observation field and a stop detection unit that detects the stop state of the movement of the observation field.

[0057] When executing a focus sequence during field of view movement, in which the focus adjustment mechanism adjusts the relative distance between the focal position of the optical imaging system, such as the objective lens unit 25, and the observation object WK based on information about the XY position of the observation field while the illumination control unit 66 is executing the first sequence, the illumination unit 60 operates in a first illumination pattern, and when the movement stop detection unit 94 detects that the movement of the observation field has stopped, the focus sequence during field of view movement is terminated and a stop sequence is executed. In the stop sequence, for example, to execute autofocus, the illumination unit 60 is temporarily switched to omnidirectional illumination, in which the coaxial epi-illumination unit 62, ring illumination unit 63, etc. are all lit, and after autofocus is completed, the illumination unit 60 is again operated in the first illumination pattern. When the movement stop detection unit 94 detects that the observation field of view has moved, the focus sequence execution unit 93 executes the focus sequence during field of view movement while the illumination control unit 66 continues operating the illumination unit 60 in the first illumination pattern.

[0058] The image processing unit 84 also realizes a function for synthesizing an image from multiple images (synthetic image mode). For example, for an image with a shallow depth of focus, it is possible to synthesize only the in-focus portions of multiple images taken while changing the focal position based on the focus information to obtain an image with a deep depth of focus (depth synthetic image). It is also possible to obtain images with enhanced resolution or an expanded dynamic range using so-called super-resolution technology. Examples of synthetic images generated by the image processing unit 84 in this way include depth synthetic images, 3D synthetic images, pixel-shifted images, super-resolution images, and HDR images. Furthermore, the image processing unit 84 may be provided with a function for measuring the object being observed. For example, a measurement tool with an array of buttons for performing several measurements can be displayed on the image, and desired measurements can be performed by operating the buttons on the displayed image.

[0059] During execution of the first sequence by the illumination control unit 66, the illumination unit 60 is operated in a first illumination pattern to display a live image, and during the second sequence, the illumination unit 60 is operated in a second illumination pattern in one of the illumination directions selected based on the feature amounts of each image data calculated by the feature amount calculation unit 88, and the image processing unit 84 realizes a function of synthesizing an image from a plurality of images (synthetic image mode).The display unit 70 also displays a composite image such as a depth composite image, a 3D composite image, a pixel shift image, a super-resolution image, or an HDR image of the object to be observed illuminated with the second illumination pattern.

[0060] The transition from the first sequence to the second sequence may be triggered by an instruction to execute the composite image mode, or may be triggered by designating an ROI using the region designation unit after the image composition mode has been set in advance.

[0061] The height information acquisition unit 89 acquires height information at different XY positions of the observation target WK as three-dimensional reference information, e.g., height image data of the observation target WK. When the field of view movement mechanism 5 moves the observation field displayed on the display unit 70 via the display control unit 52, the focus adjustment mechanism can be adjusted to adjust the focal position based on the height information of the XY position corresponding to the destination field of view movement position among the three-dimensional reference information acquired by the height information acquisition unit 89. Here, the height information indicates the position in the z direction from the mounting surface of the stage unit 30. With this configuration, by acquiring height image data, which is three-dimensional information of the observation target WK, in advance, focus adjustment can be performed based on the height information of this height image data, and when the observation field of view is moved, focused image data can be displayed on the display unit 70. For example, the height information acquisition unit 89 acquires height information corresponding to the XY position as three-dimensional reference information from wide-area image data with three-dimensional information captured by the low-magnification objective lens unit 25. The height information acquisition unit 89 can acquire height information corresponding to XY positions as three-dimensional reference information for wide-area image data with three-dimensional information that has been captured in advance, such as a navigation image described below. Note that when displaying an area that does not include height information acquired by the height information acquisition unit 89, the focus adjustment mechanism may be configured to perform focus adjustment based on height information estimated based on height information around the area. If the movement of the mounting surface of the stage unit 30 is not taken into consideration, such as when the mounting surface of the stage unit 30 is fixed, the height information does not necessarily need to be the position in the z direction from the mounting surface of the stage unit 30, and may be, for example, the position in the z direction in the device coordinate system of the magnification observation device.

[0062] Furthermore, the focus adjustment mechanism may be configured to determine the direction of movement according to changes in the focus feature amount based on image data obtained when the field of view is moved by the field of view movement mechanism, thereby enabling focused image data to be displayed on the display unit even when the field of view is moved by the field of view movement mechanism.

[0063] Furthermore, the focus adjustment mechanism may be configured to estimate a predetermined geometric shape from the measured values, and determine and move the target height value based on the field of view movement position and the estimated information. Here, the predetermined geometric shape can be a plane, sphere, cylinder, cone, parabola, etc.

[0064] When the illumination control unit 66 executes a focus sequence during field of view movement using a predetermined geometric shape via the focus adjustment mechanism while the illumination unit 60 is operating in a first illumination pattern, the focal position is adjusted based on the height information of the observation object WK at each XY position measured in advance and the geometric shape, such as the planar shape. When the movement stop detection unit 94 detects that the movement of the observation field of view has stopped, the focus sequence during field of view movement is terminated and a stop sequence is executed. In the stop sequence, when measuring the height information of the observation object WK and performing autofocus, the illumination unit 60 is temporarily switched to omnidirectional illumination, with the coaxial epi-illumination unit 62, ring illumination unit 63, etc., fully lit. After the height information measurement and autofocus are completed, the illumination unit 60 is again operated in the first illumination pattern. The three-dimensional reference information is updated based on the measured height information. When the movement stop detection unit 94 detects the movement state of the observation field of view, the illumination control unit 66 continues operating the illumination unit 60 in the first illumination pattern, and the focus sequence execution unit 93 executes the focus sequence during field of view movement.

[0065] Furthermore, the focus adjustment mechanism may be configured to transition to a stop sequence when movement of the observation field by the field of view movement mechanism is stopped, and to automatically execute autofocus control as the stop sequence. The focus adjustment mechanism may also be configured to immediately stop autofocus control when an instruction to stop autofocus is received. For example, while autofocus control is being performed, autofocus may be stopped and a focus sequence during field of view movement may be executed when movement of the observation field by the field of view movement mechanism is resumed. In this case, if the focus sequence during field of view movement is a focus sequence during field of view movement in which the focus adjustment mechanism adjusts the relative distance between the focal position of an optical imaging system, such as the objective lens unit 25, and the observation object WK based on image data captured by the camera unit 10 while the observation field of view is being moved, the focus adjustment mechanism adjusts the relative distance between the focal position of an optical imaging system, such as the objective lens unit 25, and the observation object WK to the relative distance that maximizes the focus value due to the stopped autofocus. Furthermore, in the case of a focus sequence during field of view movement in which the focus adjustment mechanism adjusts the relative distance between the focal position of an optical imaging system such as the objective lens unit 25 and the observed object WK based on height information of an XY position corresponding to the field of view movement position of the destination out of the three-dimensional reference information acquired by the height information acquisition unit 89, the focus adjustment mechanism adjusts the relative distance between the focal position of an optical imaging system such as the objective lens unit 25 and the observed object WK to the relative distance corresponding to the field of view movement position of the destination based on the three-dimensional reference information. This makes it possible to continue observation in a state as focused as possible even when autofocus is stopped.

[0066] When the field of view movement mechanism 5 moves the observation field of view displayed on the display unit 70 by the display control unit 52, the determination unit 97 determines whether or not to change the relative distance between the focal position of the objective lens unit 25 and the observation object beyond a predetermined condition using the focus adjustment mechanism, based on height information of a position corresponding to the field of view movement position at the destination, among the three-dimensional reference information acquired by the height information acquisition unit 89. Furthermore, the one-shot synthesis unit 98 performs depth synthesis after automatically setting upper and lower height limits for the image displayed in the observation field of view based on height information of the observation object.

[0067] The processor unit 80 can be configured with a general-purpose CPU, MPU, SoC, or a gate array such as an ASIC or FPGA customized for a specific application. While this example illustrates a configuration in which a single CPU serves as the processor unit and realizes multiple functions, the present invention is not limited to this configuration, and the processor unit may also be configured with multiple CPUs. The multiple CPUs may be multiple physical CPUs or a so-called multi-core MPU incorporating multiple CPU cores in a single package. In this case, each function may be realized by multiple CPUs or CPU cores, or different functions may be assigned to each CPU or CPU core. Furthermore, the processor unit may be configured with a combination of a CPU and a GPU. In this case, the GPU may perform the functions of the display control unit 52 described above and may also be configured to execute some or all of the functions assigned to the processor unit. (Navigation image registration function)

[0068] The magnification observation device according to this embodiment also includes a navigation image registration function for registering a wide-area image to facilitate searching for the observation area of ​​the observation object. The navigation image registration function acquires a wide-area image of the observation object in advance, captured using a low-magnification objective lens unit 25, and displays this wide-area image on a screen separate from the current observation field, such as a navigation window. Clicking a desired position on the navigation window moves the XY stage to that position. Details of the navigation image registration function will be explained based on the user interface screen of the magnification image observation program shown in FIG. 5. The navigation image registration screen 230 shown in this figure includes an image display area 231, an operation area 232, and a navigation area 233. The image display area 231 is an area for displaying an image. The operation area 232 is an area for displaying various operation buttons, explanations, and the like. The navigation area 233 is an area for displaying the navigation image NI, which is a wide-area image.

[0069] A position corresponding to the observation field displayed in the image display area 231 is displayed as a rectangle FR on the navigation image NI displayed in the navigation area 233. The display of the position corresponding to the observation field displayed in the image display area 231 is not limited to a rectangle FR, but may be displayed as cross lines intersecting in a crisscross pattern on the navigation image NI displayed in the navigation area 233. In this case, the cross lines are displayed so that the position on the navigation image NI where the cross lines intersect corresponds to the center of the observation field displayed in the image display area 231. This allows the user to relatively grasp the positional relationship of the currently observed position and the location of the observed object. In FIG. 5, chip resistors CR1 to CR3 soldered on a circuit board are displayed on the navigation image NI, and the center of chip resistor CR1 is located at the center of the observation field. Furthermore, when the XY stage is moved to change the observation field displayed in the image display area 231, the position of the rectangle FR on the navigation area 233 is updated to the moved position accordingly. Furthermore, when a desired position is designated in the navigation area 233 with a mouse cursor or the like, the XY stage is moved to the corresponding position. It is preferable that such a navigation image NI captures a wide range of the object to be observed, but it does not have to be a full view of the object to be observed.The relationship between the stage unit 30, the object to be observed WK, the navigation image NI, and the observation field of view OA is shown in Figure 6. (3D navigation image registration function)

[0070] Furthermore, the magnification observation device according to this embodiment can also have a 3D navigation image registration function for registering a wide-area image having height information. The 3D navigation image registration function may be integrated with the navigation image registration function. That is, when registering a navigation image as a wide-area image, it can also be registered as a 3D navigation image having height information.

[0071] Here, the procedure for registering a 3D navigation image in which each pixel of a wide-area image has height information will be described with reference to the flowchart in Fig. 7 and the user interface screens in Figs. 8 to 10. First, on the 3D navigation image registration screen 240 in Fig. 8, the "3D navigation registration" button 243 provided in the operation area 242 is pressed to execute 3D navigation image registration. Note that an out-of-focus image is displayed in the image display area 241. In this example, a chip resistor CR surface-mounted on a board is displayed as the observation object in the image display area 241, and the dashed line indicates that the image is out of focus.

[0072] Next, in step S701, the selection of the lens magnification of the objective lens unit 25 is accepted. Here, when the "3D navigation registration" button 243 is pressed on the 3D navigation image registration screen 240 of FIG. 8, a low-magnification imaging window 244 is displayed as shown in FIG. 9. From this screen, the user can set the imaging settings for the navigation image, such as lens magnification and focus adjustment. The low-magnification imaging window 244 includes a lens magnification adjustment field 245 and a focus adjustment field 246 for focus adjustment. If necessary, the user can physically switch the objective lens 25 to a low-magnification objective lens unit 25 suitable for capturing wide-area images. The magnification observation device may present a predetermined magnification of the objective lens unit 25 as a default value. Then, in step S702, the lens magnification of the objective lens unit 25 is changed based on the user's selection.

[0073] Next, in step S703, a designation of the Z-direction movement range is accepted. Here, the user first moves the stage unit 30 or the objective lens unit 25 to the observation field where the wide-area image is desired to be captured. The low-magnification imaging window 244 displays a display guiding the user to move the stage unit 30 or the objective lens unit 25 to the lowest position. The lowest position refers to the position within the Z-direction movement range where the relative distance between the objective lens unit 25 and the observation object is closest, and thus corresponds to the lowest value of the height information. Meanwhile, the highest position within the Z-direction movement range where the relative distance between the objective lens unit 25 and the observation object is farthest is the highest position, and thus corresponds to the highest value of the height information. The focus adjustment field 246 of the low-magnification imaging window 244 shown in FIG. 9 may include a button for moving to the lowest position to acquire height information. At this time, the focus adjustment field 246 may also display a side camera image SI captured using a side camera (described later). This makes it easier to adjust the objective lens so that it does not collide with the object being observed when moving it to its lowest position before acquiring height information. As described above, the lower limit position of the Z-direction movement range can be specified by adjusting the relative distance between the objective lens unit 25 and the object being observed to the actual lower limit position, or the lower limit position can be specified using the side camera image SI. The upper limit position of the Z-direction movement range is specified via the focus adjustment field 246. In the example of FIG. 9, the upper limit position can be specified in three levels: high, medium, and low, and buttons corresponding to each level are displayed. Low is selected by default. The upper limit position can also be specified using the side camera image SI, for example.

[0074] Then, in step S704, an instruction to register a 3D navigation image is accepted. If no instruction is received, step S704 is repeated. When the user presses the execute button 247 in the low-magnification imaging window 244 of FIG. 9 to instruct execution of 3D navigation image registration, in step S705, the Z stage is moved in a direction that increases the relative distance between the objective lens unit 25 and the stage unit 30. Then, in step S706, an image is captured, and the focus value of each pixel is calculated based on the captured image. At this time, the height position at the time the image was captured may be acquired. For example, Z-direction position information may be acquired using a linear encoder attached along the Z direction to the upper Z elevator 16 of the upper Z stage or the lower stage elevator 35 of the lower Z stage, and the height position of the stage unit 30 from the mounting surface at the time the image was captured may be acquired based on this position information. Next, in step S707, it is determined whether the upper limit position of the movement range in the Z direction has been reached. If not, the process returns to step S705 and is repeated. If the upper limit of the movement range is reached, in step S708, 3D navigation data is generated based on the brightness value corresponding to the height position at which the focus value is maximized for each pixel. This 3D navigation data is registered as a 3D navigation image. The 3D navigation image is a wide-area image for indicating the position of the observation field displayed in the image display area 241, and is displayed in the navigation area 250 with each pixel corresponding to an XY position. The 3D navigation image also includes the height position corresponding to each pixel and can be used as three-dimensional reference information. Finally, in step S709, a navigation image is displayed on the display unit 70 based on the 3D navigation data, and the 3D navigation image registration process is terminated. As shown in FIG. 10, cross lines intersecting in a crisscross pattern may be displayed on the navigation image displayed in the navigation area 250. In this case, the cross lines are displayed so that the position on the navigation image where the cross lines intersect corresponds to the center of the observation field displayed in the image display area 241. This allows the user to relatively grasp the relative positional relationship of the currently observed position and the location of the object being observed. When the 3D navigation image registration process is completed, a focused image is displayed in the image display area 241, as shown in FIG.At this time, focus adjustment is performed based on the focus value within the area RI surrounded by a dashed line near the center of the image display area 241, and an in-focus image is displayed. Therefore, in the example of Figure 10, a chip resistor CR is located in the area RI surrounded by a dashed line near the center of the image display area 241, and the surface of the chip resistor CR is in focus, while the land on the substrate CB on which the chip resistor CR is mounted is out of focus. While an example has been shown in which image data captured by a low-magnification objective lens unit 25 is used as the 3D navigation image, this is not limiting. For example, a wide-area image corresponding to the navigation image may be generated by concatenating multiple image data. When generating a 3D navigation image, height image data corresponding to each imaging position is generated, and the 3D navigation image is generated by concatenating the height image data. A flatness correction function may be added to more accurately acquire 3D data such as 3D navigation data. Due to the field curvature of the lens of the microscope lens unit 20, even when 3D navigation data is acquired relative to a flat surface, the periphery of the field of view is curved. Therefore, a flatness correction function may be used to correct the 3D navigation data by an amount equivalent to the curvature. Furthermore, when performing depth stacking (described later), the upper and lower limits of the Z range for depth stacking may be automatically set based on the registered 3D navigation data. For example, when performing depth stacking in a certain observation field of view for an observation object WK as shown in FIG. 36, the 3D data can be referenced to obtain the height distribution included in this observation field of view, and the upper height Zmax and lower height Zmin of the Z range for depth stacking can be determined. This type of generation of a depth stacked image is called one-shot synthesis (details will be described later). (Autofocus adjustment mechanism)

[0075] This magnification observation device 100 has an autofocus function. Specifically, autofocus is performed by a focus adjustment mechanism that adjusts the focus of an image by changing the relative distance between the focal position of the objective lens unit 25 and the stage unit 30 along the optical axis of the objective lens unit 25. In addition to calculating the focus value based on image data generated by the camera unit 10, autofocus may also be performed by measuring the relative distance between the objective lens unit 25 and the object to be observed using an external distance measuring sensor. Known methods such as a contrast method or a phase difference method can be used as appropriate to achieve autofocus. The magnification observation device according to the above-mentioned first embodiment uses the contrast method.

[0076] This magnification observation device 100 performs normal autofocus, which searches for a focused position within a normal height range, and short-distance autofocus, which searches for a focused position within a narrower height range than the normal autofocus. First, the normal autofocus procedure will be described based on the flowchart in FIG. 11. Here, the user, for example, temporarily lowers the objective lens unit 25 to bring it close to the observation target on the mounting surface of the stage unit 30. Then, the user operates the Z stage using a focus adjustment mechanism or the like to automatically search for a focused position. First, in step S1101, the Z stage is guided to move so as to reduce the relative distance between the objective lens unit 25 and the stage unit 30. Next, in step S1102, it is determined whether or not an autofocus start command has been issued. For example, it is detected that the user has pressed the autofocus start button. If an autofocus start command has been issued, the process proceeds to step S1103; if not, the process repeats step S1102.

[0077] Next, in step S1103, the Z stage is moved by a focus adjustment mechanism or the like in a direction that increases the relative distance between the objective lens unit 25 and the stage unit 30. Then, in step S1104, images are captured at predetermined timing during the movement of the Z stage, and a focus value is calculated based on the captured images. Furthermore, in step S1105, it is determined whether the movement of the Z stage has ended within the height range for normal autofocusing. If not, the process returns to step S1103 and repeats the above process. Then, if the movement within the normal autofocusing range has ended, the process proceeds to step S1106, where the height position at which the focus value is maximized among the calculated focus values ​​is determined, and the Z stage is moved to this position by a focus adjustment mechanism or the like. Finally, in step S1107, a live image is displayed on the display unit 70. In this manner, normal autofocusing is performed. (Short range autofocus)

[0078] Next, short-distance autofocus will be described with reference to the flowchart in FIG. 12. Short-distance autofocus is applied to a system that, when the object is roughly in focus, searches a relatively narrow Z-direction range and adjusts the height to a position where the object is in focus. In short-distance autofocus, precise measurement of height information may be performed by narrow-pitch searching. First, in step S1201, the Z stage is moved in the forward direction. Next, in step S1202, images are captured at predetermined timings while the Z stage is moving, and a focus value is calculated based on the captured images. Then, in step S1203, it is determined whether the movement of the Z stage has ended within the height range for short-distance autofocus. If not, the system returns to step S1201 and repeats the above process.

[0079] If the movement of the short-distance autofocus range is completed, the process proceeds to step S1204, where the Z stage is moved in the direction opposite to the forward direction. Furthermore, in step S1205, images are captured at predetermined timings during this Z stage movement, and the focus value is calculated based on the captured images. Next, in step S1206, it is determined whether the movement of the Z stage has completed within the height range for short-distance autofocus. If not, the process returns to step S1204 and the above process is repeated.

[0080] When the movement of the short-distance autofocus range is completed, the process proceeds to step S1207, where the height position at which the focus value is maximized among the calculated focus values ​​is determined, and the Z stage is moved to this position. Finally, in step S1208, a live image is displayed on the display unit 70. In this manner, short-distance autofocus is executed. [Embodiment 2]

[0081] Furthermore, the method for achieving autofocus is not limited to the above-described method, and other methods, such as a phase-difference method, may also be used. For example, FIG. 13 shows a magnification observation device 200 according to a second embodiment, which uses the phase-difference method. The magnification observation device shown in this figure includes an autofocus sensor 15. Note that in the magnification observation device 200 according to the second embodiment, the same components as those described in the first embodiment are designated by the same reference numerals, and detailed descriptions thereof are omitted as appropriate. The autofocus sensor 15 is composed of a phase-difference autofocus sensor. The phase-difference autofocus sensor splits and receives light incident from the objective lens unit 25, and the focus direction and amount are determined as focus-degree features from the distance between the two formed images by the focus-degree evaluation unit 91. Compared to the contrast method described above, this method allows for faster focusing because it does not require moving the objective lens unit 25 to search for the focus.

[0082] The autofocus sensor is not limited to a phase-difference autofocus sensor, and may be, for example, a distance measurement sensor that measures the distance between the objective lens unit and the object being observed. In this case, autofocus can be achieved by measuring the distance between the objective lens unit and the object being observed using the distance measurement sensor, determining the difference between this and a pre-measured focal length of the objective lens unit, and then moving the Z stage or the like by the corresponding distance using an autofocus adjustment mechanism. The following description uses a contrast method as an example, but it goes without saying that the present invention can also be applied to other known autofocus methods, such as a phase-difference method.

[0083] Typically, autofocusing is performed while the XY stage is stopped. That is, autofocusing is not performed while the head unit 4 or the objective lens unit 25 is moving, but is performed while the XY stage is stopped. In other words, autofocusing is not performed during the observation field movement sequence, but is performed during the stop sequence. Such autofocusing can be performed by an explicit command from the user, such as pressing an autofocus execution button, or it can be performed automatically when the magnification observation device detects that the XY stage movement has stopped and the movement of the observation field is stopped. This allows a focused image to be automatically displayed on the display unit 70 when the movement of the observation field is stopped, without the user having to issue an autofocus command each time, improving usability. The movement stop detection unit 94 can be used to move or stop the observation field. (Focus tracking function)

[0084] On the other hand, the magnification observation device according to this embodiment can perform autofocus not only when the XY stage is stationary, but also when the XY stage is moving, i.e., when the observation field is moving. In this specification, the function of adjusting the focus while the observation field is moving within the XY plane is called focus tracking. Focus tracking tracks the movement of the head unit 4 in the direction in which the focus is achieved.

[0085] This focus control will be explained with reference to Figures 14A and 14B. Focus tracking includes a moving sequence, which is a focus control performed while the observation field of view is moving, and a stop sequence, which is performed when the movement of the observation field of view is stopped. Figure 14A shows the moving sequence, and Figure 14B shows the stop sequence. For ease of explanation, these figures show an example in which the observation field of view is moved by moving the objective lens unit 25. However, the movement of the observation field of view is not limited to this, and it goes without saying that the XY stage may also be moved. Figure 14A shows a substrate CB on which chip resistors CR1 and CR2 are mounted on the mounting surface of the stage unit 30, and the objective lens unit 25 moves relative to the stage unit 30 in the order 25-1, 25-2, 25-3, and 25-4. Arrow 26 indicates the relative movement trajectory of the objective lens unit 25 with respect to the stage unit 30. As shown in FIG. 14A, while the observation field is moving, the objective lens unit 25 is moved vertically to follow the general shape of the observation object, such as the substrate CB or chip resistors CR1 and CR2, to change the relative distance between the objective lens unit 25 and the observation object, thereby adjusting the focal position. Furthermore, as shown in FIG. 14B, when the movement of the observation field is stopped, short-distance autofocus is performed as a stop sequence to accurately adjust the focal position. Arrow 27 indicates the change in the relative distance of the objective lens unit 25 to the stage unit 30 during short-distance autofocus. In this case, more accurate autofocus results can be obtained than with the moving sequence. Note that performing autofocus during the stop sequence is not necessarily required; it is also possible to set the system not to perform autofocus when the observation field is stopped.

[0086] Furthermore, the focus tracking function itself may be switchable between ON and OFF. Hereinafter, the state in which the focus tracking function is ON is referred to as the focus tracking mode. To execute the focus tracking mode, for example, press the "focus tracking" button 248 on the 3D navigation image registration screen 240 of FIG. 10. FIG. 15 shows how, in the focus tracking mode, focus control is performed so that the area RI surrounded by a dashed line near the center of the image display area 241 is in focus. In the example of FIG. 15, the substrate CB is located in the area RI surrounded by a dashed line near the center of the image display area 241, and the lands on the substrate on which the chip resistors are mounted are in focus, but the surface of the chip resistors CR is out of focus. 10, during focus tracking mode, if the observation field of view is moved from this state so that the chip resistor CR is positioned within the area RI surrounded by a dashed line near the center of the image display area 241, focus adjustment is performed while the observation field of view is moved, and the surface of the chip resistor CR is in focus while the lands SD on the substrate CB on which the chip resistor CR is mounted are out of focus. When a user instruction is received using the "One-shot synthesis" button 249 shown in FIG. 16, depth stacking processing is performed and a depth stacked image is displayed in the image display area 241 (details will be described later).

[0087] In the focus tracking mode, there are various methods for focusing control that operate the objective lens unit 25 or the Z stage to achieve autofocus. The magnification observation device according to this embodiment allows selection of multiple focusing controls in the focus tracking mode. In other words, multiple focus tracking modes with different focusing control methods are executable. In the focus tracking mode, an appropriate selected focusing control sequence is executed while the field of view is moving in a direction perpendicular to the optical axis. Similarly, when the movement of the observation field of view is stopped, an appropriate focusing control sequence is executed depending on the focus tracking mode. Specifically, examples of focus tracking modes include a general-purpose mode, a 3D shape tracking mode, and a planar tracking mode. In each of these focus tracking modes, a focusing control sequence appropriate for the observation mode may be automatically selected during or when the observation field of view is moving. (general purpose mode)

[0088] Figure 17 shows the movement sequence and stop sequence in the general-purpose mode. Figure 17 shows the state in which a stepped substrate CB is placed on the mounting surface of the stage unit 30, and the objective lens unit 25 moves relative to the stage unit 30 in the order 25-1, 25-2, 25-3, 25-4, 25-5, and 25-6. Arrow DR1 indicates the direction of movement of the field of view of the objective lens unit 25 relative to the stage unit 30. Arrow DR2 indicates the trajectory of relative movement of the objective lens unit 25 relative to the stage unit 30. Each arrow 27' indicates the direction of change in the relative distance of the objective lens unit 25 relative to the stage unit 30. Figure 17 also shows the focus value at each point in time. As shown in this figure, in the general-purpose mode, focusing control is performed based on the focus value. That is, the change in the focus value after movement of the observation field of view is used to control the focus adjustment mechanism so that if the focus value increases, movement in the Z direction continues, and if the focus value decreases, movement in the opposite direction is performed. First, the objective lens unit 25 is moved in either the up or down direction. If the focus value decreases as a result of the movement, the objective lens unit 25 moves in the opposite direction. The focus value is sequentially acquired while the observation field of view is moved, and the focus value is controlled vertically to increase. In the moving sequence in general-purpose mode, the objective lens unit 25 continues to move in either the vertical or horizontal direction relative to the stage unit 30, and the movement direction is controlled based on the change in focus value. As a result, focusing control is possible without peak search. Note that the trajectory of the relative movement of the objective lens unit 25 indicated by the arrow DR2 is exaggerated for convenience and appears wavy, but in reality, the trajectory roughly matches the outline of the object being observed. Furthermore, if the focus value decreases significantly, the pitch may be increased for control. After the movement of the observation field of view stops, the system transitions to a stop sequence. In the stop sequence, autofocus is performed, and the focus value is controlled to be maximized at the observation position, after which the focusing operation stops.

[0089] Next, the specific procedure for autofocus in general-purpose mode will be described with reference to FIG. 18. First, the focus value is buffered in the initial state. Then, in step S1801, the upper Z stage is moved in a predetermined direction. Next, in step S1802, an image is captured, and the focus value is calculated based on the captured image. Then, in step S1803, a change in the focus value is calculated based on the current focus value and previous focus values. The focus value is buffered each time and used as the previous focus value. Then, in step S1804, it is determined whether the focus value has increased. If it has increased, proceed to step S1805, and the movement direction of the upper Z stage is maintained. On the other hand, if the focus value has decreased, proceed to step S1806, and the movement direction of the upper Z stage is reversed. While the field of view is moving, the tracking process is repeated from step S1801 as a moving sequence. In this way, the moving sequence monitors the focus value even during Z stage movement, and moves the Z stage in the direction of focus, thereby preventing out-of-focus images. When the movement of the field of view stops, the buffered focus value is cleared as a stop sequence and the tracking process ends. (3D shape following mode)

[0090] In the 3D shape tracking mode, height information of the object being observed is acquired in advance and this height information is used to adjust the focus. Height information is acquired by simultaneously acquiring a height image when capturing a 3D navigation image as a wide-area image of the object being observed at low magnification. The wide-area image may also be a connected image containing height information. In this way, in the 3D shape tracking mode, autofocus is performed using a 3D navigation image containing height information captured in advance. On the other hand, the general-purpose mode described above does not use such height information, so there is no need to acquire a height image in advance. For example, on the 3D navigation image registration screen 240 in Figure 8, if the "Use registered 3D data" checkbox 254 is checked, focus tracking is performed in the 3D shape tracking mode using the height information of the 3D navigation image. On the other hand, if the "Use registered 3D data" checkbox 254 is checked, focus tracking is performed in the general-purpose mode, which does not use the height information of the 3D navigation image.

[0091] The 3D shape tracking mode is shown in FIG. 19. FIG. 19 shows a state in which a stepped substrate CB, on which chip resistors CR1 to CR3 are mounted, is placed on the mounting surface of the stage unit 30, and the objective lens unit 25 moves relative to the stage unit 30 in the order of 25-1, 25-2, 25-3, 25-4, 25-5, 25-6, 25-7, 25-8, and 25-9. Arrow 26″ indicates the relative movement trajectory of the objective lens unit 25 with respect to the stage unit 30. Arrow 27″ indicates the change in the relative distance of the objective lens unit 25 with respect to the stage unit 30 during short-distance autofocus. FIG. 19 also shows the corresponding focus values ​​at each point in time. In this figure, the area where 3D data exists is indicated by a thick dashed line. In the range where 3D data exists as height information, i.e., three-dimensional reference information, the focus position is adjusted according to this height information. The relative positions of the objective lens unit 25 with respect to the stage unit 30 at 25-1, 25-2, 25-3, 25-4, and 25-5 correspond to the range in which 3D data exists as three-dimensional reference information. On the other hand, when the observation field of view is moved to a range in which 3D data does not exist as three-dimensional reference information, height information cannot be obtained, so the system switches to the general-purpose mode described above and performs focusing control. In other words, focusing control is performed using the change in focus value after the observation field of view is moved. In the example of Figure 19, the relative positions of the objective lens unit 25 with respect to the stage unit 30 at 25-6, 25-7, 25-8, and 25-9 correspond to the range in which 3D data does not exist as three-dimensional reference information. Note that the trajectory of relative movement of the objective lens unit 25 in the range where there is no 3D data, indicated by arrow 26", is exaggerated for convenience and appears wavy, but in reality the trajectory roughly matches the outline of the object being observed except for areas with large steps. Also, in the 3D shape tracking mode, when the movement of the observation field of view stops, the system transitions to a stop sequence. In this way, even if there is a mixture of ranges where there is 3D data and ranges where there is no 3D data, seamless optimal focus control can be achieved by automatically switching the focus tracking mode to the optimal mode. Furthermore, the color of the focus frame may be changed depending on the presence or absence of 3D data (details will be described later). For example, the color of the dashed line RI near the center of the image display area 241 shown in Figure 15 may be changed. (Focus jump function)

[0092] In the 3D shape tracking mode, focusing is controlled solely by the height position of the 3D data. While focus values ​​are normally used for control, as in the general-purpose mode, focusing control can also be performed by jumping the focus position using the height position of the 3D data when certain conditions are met. For example, a focus jump function can be added that jumps the focus position using the height position of the 3D data when there is a large step. This focus jump function is explained with reference to Figure 20. Figure 20 shows a stepped substrate CB with chip resistors CR1 to CR3 mounted on the mounting surface of the stage unit 30, and the objective lens unit 25 moves relative to the stage unit 30 in the order 25-1, 25-2, and 25-3. Arrow 28-1 indicates the change in the relative distance of the objective lens unit 25 relative to the stage unit 30 corresponding to the focus jump. In this figure, the area where 3D data exists is indicated by a bold dashed line. The previously acquired 3D data of the observation object is acquired, for example, through a low-magnification objective lens unit 25, so there may be discrepancies between the actual shape of the observation object and the data. On the other hand, focusing control based on focus values ​​measured in real time results in focusing control that matches the actual shape of the observed object. However, in situations where the focus value changes significantly over a short period of time, such as when there is a large step, it becomes difficult to track the actual shape of the observed object. For example, increasing the tracking responsiveness makes the focusing control more susceptible to oscillation and instability, while stabilizing the focusing control reduces the tracking responsiveness, making the focusing control unable to track when there is a large step. Therefore, the determination unit 97 acquires height information from the corresponding portion of the 3D data at the observation position to which the field of view is moved and calculates the height by measuring the focus value in the general mode described above. The determination unit 97 then compares the height of the 3D data with the height obtained by measuring the focus value of the observed object in the general mode described above and determines whether the difference falls within a predetermined range. If the difference falls within the predetermined range, the height based on the focus value is adopted. On the other hand, if the height based on the focus value significantly deviates from the height of the 3D data, i.e., exceeds a predetermined height range, the height of the 3D data is adopted.In this way, the relative distance of the objective lens unit 25 to the stage unit 30 is controlled during field of view movement from a relative distance based on the focus value measured in real time to a relative distance corresponding to the height information of the 3D data at the XY position of the field of view movement destination. That is, a jump is made from a height based on the focus value to a height based on the 3D data. Figure 20 schematically shows the decision threshold DS, which determines whether to adopt focus control based on the focus value measured in real time (control equivalent to the general-purpose mode) or focus control based on the height of the 3D data. When the objective lens unit 25 reaches 25-1, a flat area that fits within the decision threshold DS is expanded, so focus control equivalent to the general-purpose mode is performed for this area. Furthermore, there is a step between the objective lens unit 25's positions 25-1 and 25-2, and a slope between the objective lens unit 25's positions 25-2 and 25-3. A focus jump is performed when the field of view moves and passes over such a step or slope that exceeds the decision threshold DS. For example, at position 25-1, a lower limit height TH1 and an upper limit height TH2 are set, and as the field of view moves, the height range defined by TH1 and TH2 is exceeded, so a focus jump is executed (28-1). At position 25-2, the field of view does not move beyond the height range defined by TH3 and TH4, so a focus jump is not executed (28-2). In this way, by using the height information from the 3D data, highly reliable focusing control is possible, avoiding the degradation of tracking response at such boundaries. When slowly passing over a slope, the height of the 3D data at the XY position of the field of view movement falls within the determination threshold value DS, so a focus jump is not executed. (Offset function)

[0093] Furthermore, the 3D shape tracking mode can also add a function to offset the height of the 3D data. This offset function will be described with reference to FIG. 21. The previously acquired 3D data of the substrate CB, including the chip resistors CR1-CR2 as the observation target, may have a uniform deviation due to differences in the actual observation target's placement, etc. Here, it is assumed that the 3D data is acquired as indicated by the bold dashed line, as in FIG. 20. When the movement of the observation field of view is stopped at a position where 3D data exists, an autofocus operation in the stop sequence is performed at this position to acquire height information. Arrow 27''' indicates the change in the relative distance of the objective lens unit 25 to the stage unit 30 during short-distance autofocus. An offset amount for the 3D data is calculated so that the height of the 3D data at this position matches the calculated height information. The 3D data is then offset based on the offset amount. For example, the calculated offset amount is stored in the memory unit 56. When using the height information of the field of view to be moved in the 3D shape tracking mode, focusing control is performed based on the height information of the 3D data at the XY position of the field of view to be moved and the offset amount stored in the memory unit 56. In the example shown in FIG. 21 , the 3D data indicated by the thick dashed line is offset upward to the position indicated by the thin dashed line. In this case, a positive offset value corresponding to the offset amount is stored in the memory unit 56. Furthermore, if the offset is downward, a negative offset value corresponding to the offset amount is stored in the memory unit 56. By configuring the height information of the 3D data in this manner, it is possible to obtain a more accurate surface height of the observed object and perform accurate focusing control. The decision as to whether to perform offsetting may be based on the shape of the vicinity of a certain position from which the height information was obtained. For example, as shown in FIG. 37 , when height information of a certain position is obtained in the stopping sequence, the 3D data may be referenced to obtain shape information of the vicinity of this position, and if the surface is horizontal, offsetting may be performed, but if the surface is inclined, offsetting may not be performed. This configuration allows the offset amount of the 3D data to be calculated with high accuracy. (Focus frame)

[0094] Furthermore, during autofocusing, a focus frame indicating the focused portion of the observation field of view may be displayed. In this case, the display mode of the focus frame may be changed according to the autofocus control. An example of such a change in display mode is shown in FIGS. 22A to 22C. For example, during autofocusing, focus frame FF1 is displayed as an opaque, thick frame as shown in FIG. 22A. When the observation field of view is stopped, focus frame FF2 is displayed as a semi-transparent frame as shown in FIG. 22B. This change in the display mode of the focus frame allows the user to visually determine whether the autofocus function is currently active. In addition to opaque and semi-transparent, other visual distinctions may be made, such as thick or thin lines, blinking, highlighting, or graying. The display mode of the focus frame may also be changed between the moving sequence and the stopped sequence. For example, in FIG. 22A, focus frame FF3 is displayed as an opaque, thin-lined light blue frame during focus control using 3D data (3D shape tracking mode), and focus frame FF1 is displayed as an opaque, thick-lined light blue frame during the stop sequence. Furthermore, when focus control using a focus value (general-purpose mode) is being performed, the focus frame FF3 is displayed in an opaque thin green line, and when the stop sequence is being performed, it is displayed in an opaque thick green line. Furthermore, after the autofocus operation is completed, the opaque state may be switched to a normal translucent display. For example, when focus control using 3D data is being performed, the display switches from a thick opaque light blue FF1 to a translucent light blue FF2 when the autofocus operation is completed. Furthermore, when focus control using a focus value (general-purpose mode) is being performed, the display switches from a thick opaque green FF1 to a translucent green FF2 when the autofocus operation is completed. Furthermore, such a focus frame may be displayed in the display area while a video whose display content is updated continuously is being displayed. Furthermore, in the image display area or navigation area, the frame display style displayed when an ROI is specified may be configured to change depending on whether autofocus is being performed, the type of focus control, etc.

[0095] The relationship between each command and operation during focus tracking in the 3D shape tracking mode is described below with reference to the timing chart in FIG. 23. Here, 3D data is acquired in advance by registering a 3D navigation image, and the observation field of view is moved by moving the stage unit 30. The initial state in the timing chart in FIG. 23 shows the stage unit 30 stopped at the observation position where 3D data exists. In this state, a user instruction to perform normal autofocus is received, and normal autofocus is executed, resulting in a state where the object being observed is in focus at the observation position where 3D data exists. Subsequently, when a user instruction to start focus tracking is received, a translucent light blue focus frame is superimposed on the live image of the object being observed displayed in the image display area. The stage unit 30 moves in response to a movement command to the stage unit 30. When the movement stop detection unit 94 determines that the state of the stage unit 30 is moving, the moving sequence in the 3D shape tracking mode is executed. At this time, the focus frame changes from translucent light blue to opaque light blue. On the other hand, even if the movement command to the stage unit 30 is temporarily discontinued and the stage unit 30 temporarily stops, as shown by the dashed circle in Figure 23, the movement stop detection unit 94 determines that the stage unit 30 is moving unless it stops for a certain period of time. If the stage unit 30 stops for a certain period of time, the movement stop detection unit 94 determines that the stage unit 30 is stopped. When the movement stop detection unit 94 determines that the stage unit 30 is stopped, the focus tracking transitions from the moving sequence to the stop sequence. When the transition to the stop sequence is made, short-range autofocus is performed, and an offset amount is calculated based on the height information obtained through the short-range autofocus and the height information of the 3D data. During the stop sequence, the short-range autofocus is performed using a thick line, but then the focus frame changes to a thin, translucent light blue line. After that, the stage unit 30 moves in response to the movement command to the stage unit 30. When the movement stop detection unit 94 determines that the stage unit 30 is moving, the movement sequence in the 3D shape tracking mode is executed. At this time, the focus frame changes from translucent light blue to opaque light blue.When the observation field of view moves from a range with 3D data to a range without 3D data, the in-motion sequence in the 3D shape tracking mode transitions to focus control equivalent to the general-purpose mode, and the focus frame changes from light blue to green. If the stage unit 30 stops for a certain period of time, the movement stop detection unit 94 determines that the stage unit 30 is in a stopped state. When the movement stop detection unit 94 determines that it has stopped, focus tracking transitions from the in-motion sequence to the stop sequence. When the transition to the stop sequence is made, short-range autofocus is performed. In the stop sequence, the line is thick during short-range autofocus, but then the system enters an idle state, and the focus frame changes to a thin, translucent light blue line. When a command to end focus tracking is received from the user, focus tracking control ends. (Plane following mode)

[0096] Furthermore, the magnification observation device according to this embodiment is equipped with a plane tracking mode for focus control. In the plane tracking mode, a moving plane of the focal position of the objective lens unit 25 is estimated as three-dimensional reference information by fitting a predetermined geometric shape, such as a plane or a sphere, to height information at different XY positions of the observation object. A target height value is determined and moved based on the information on the field of view movement position and the estimated moving plane. Height information of the destination of the observation field of view is estimated using a geometric shape fitted to previously measured measurement points. This mode is applied when the observation object has a geometrically shaped surface. In the plane tracking mode, as shown in FIG. 24A, a plane is estimated as three-dimensional reference information from multiple focus positions, and the Z coordinate is moved along the estimated plane while the XY stage is moving. After the XY stage stops moving, normal autofocusing is performed. In the example of FIG. 24A, SQ1 indicates multi-point height acquisition (autofocus operation), SQ3 indicates height acquisition (autofocus operation), and SQ5 indicates height acquisition (autofocus operation). SQ2 and SQ4 indicate sequences during movement, and SQ3 and SQ5 indicate sequences for stopping. To execute this plane tracking mode, for example, the "Plane Fit" button 252 in the operation area 242 on the user interface screen shown in FIG. 25 is pressed. This activates the plane tracking mode. Specifically, the heights of multiple points within the observation field indicated by the dotted line in SQ1 in FIG. 24A are acquired, and plane A is estimated as shown in FIG. 24B. Next, the objective lens unit 25 moves through the section SQ2, tracking plane A estimated in SQ1. In FIG. 24A, I indicates the control direction of the objective lens unit 25 during the section SQ2. That is, I indicates the combination of the control direction of the field of view movement mechanism during the section SQ2 and the control direction for changing the relative distance between the focal position of the objective lens unit 25 and the observation target. After moving through the section SQ2, the objective lens unit 25 is stopped at SQ3. Autofocus is then performed at the stopped position to acquire height information. In Fig. 24A, III indicates the direction of control that changes the relative distance between the focal position of the objective lens unit 25 and the observation object. In addition, plane B is estimated as updated three-dimensional reference information as shown in Fig. 24B, in combination with the heights of multiple points acquired in SQ1.Next, the objective lens unit 25 moves through the section SQ4, tracking the plane B estimated in SQ3. Finally, after moving through the section SQ4, the objective lens unit 25 stops at SQ5. Autofocus is performed at the stopped position to obtain height information. This is then combined with the heights of the multiple points obtained in SQ1 and SQ3 to estimate plane C as updated three-dimensional reference information, as shown in Figure 24B. In this way, in the plane tracking mode, autofocus is performed to obtain the height when the observation field of view stops moving, and the plane is re-estimated each time. Furthermore, while the observation field of view is moving, it is controlled vertically to follow the estimated plane. In this way, focus adjustment, which was previously not possible during movement of the observation field of view, can be achieved. Note that the heights of multiple points may also be obtained in SQ3 and SQ5, as in SQ1. (Frame skip function)

[0097] Furthermore, the magnification observation device according to this embodiment is equipped with a frame skip function that, when performing autofocus, updates the image display only when a more focused image than the currently displayed image is obtained, and does not update the image display when a less focused image than the currently displayed image is obtained. This makes it possible, for example, when performing autofocus in a stop sequence, to switch to a more focused image without presenting the user with an out-of-focus image during focus search.

[0098] Here, as procedures for performing frame skip processing during autofocusing, a procedure for performing normal autofocusing including frame skip processing and a procedure for performing short-distance autofocusing will be described. (Normal autofocus with frame skipping)

[0099] First, the procedure for performing normal autofocus including frame skip processing will be described with reference to the flowchart in FIG. 26. Here, the height direction range for normal autofocus is set in advance. First, in step S2601, the Z stage is moved in a direction that increases the distance between the objective lens unit 25 and the stage unit 30. Next, in step S2602, a predetermined frame skip is executed while the Z stage is moving (details will be described later with reference to FIG. 28). Then, in step S2603, it is determined whether the movement of the Z stage has completed the set normal autofocus range. If it has not yet moved, the process returns to step S2601 and repeats the above steps. On the other hand, if the movement of the predetermined autofocus range has completed, the process proceeds to step S2604, where the Z stage is moved to a height position where the focus value is maximized. Since this height position is the in-focus position, a live image is displayed in step S2605, and the normal autofocus processing ends. (Short range autofocus with frame skipping)

[0100] Next, a procedure for performing short-distance autofocusing including frame skip processing will be described with reference to the flowchart of Fig. 27. Here too, the range in the height direction for performing short-distance autofocusing is set in advance.

[0101] First, in step S2701, the Z stage is moved in the forward direction. The forward direction is, for example, the direction in which the objective lens unit 25 moves away from the observation object. Next, in step S2702, a predetermined frame skip is executed while the Z stage is moving (details will be described later with reference to FIG. 28). Then, in step S2703, it is determined whether the movement of the Z stage has completed the set short-distance autofocus range. If movement has not yet occurred, the process returns to step S2701 and the above process is repeated.

[0102] On the other hand, if movement through the predetermined autofocus range has been completed, the process proceeds to step S2704, where the Z stage is moved in the opposite direction to the above. For example, the Z stage is moved in a direction in which the objective lens unit 25 approaches the observation object. Next, in step S2705, a predetermined frame skip is executed while the Z stage is moving (details will be described later with reference to FIG. 28). Then, in step S2706, it is determined whether movement of the Z stage has completed the short-distance autofocus range; if not, the process returns to step S2704 and the above process is repeated.

[0103] On the other hand, if movement within the predetermined autofocus range has been completed, proceed to step S2707, where the Z stage is moved to the height position where the focus value is maximized. Since this height position is the in-focus position, a live image is displayed in step S2708, and the short-distance autofocus process ends.

[0104] Here, the procedure for performing frame skipping in steps S2702 and S2705 of the flowchart in FIG. 27 and step S2602 of the flowchart in FIG. 26 will be described with reference to the flowchart in FIG. 28. First, in step S2801, an image is captured, and in step S2802, a focus value is calculated based on the captured image. At this time, the focus value is buffered. Next, in step S2803, it is determined whether the focus value is maximum. That is, it is determined whether the calculated focus value is greater than any of the focus values ​​obtained in the past. If it is greater, the process proceeds to step S2804, where the focus value is updated, and in step S2805, the displayed image is updated to the newly captured image, and the process proceeds to step S2808. On the other hand, if the focus value is not greater in step S2803, the process proceeds to step S2806, where the acquired focus value is discarded, and then to step S2807, where the displayed image is not updated, and the process proceeds to step S2808. Then, in step S2808, it is determined whether there has been a change in the height direction. If there has been a change, the process returns to step S2801 and repeats the above process. On the other hand, if there is no change in the height direction, the process ends. In this way, a frame skip function is realized that replaces only the image with the one that is more in focus. When short-distance autofocus ends, the buffered focus value is reset. Note that the focus value buffering may be such that each focus value obtained during short-distance autofocus is buffered, or such that only the maximum focus value at that time is buffered while being updated. (Search autofocus function)

[0105] Furthermore, the magnification observation device according to this embodiment is equipped with a search autofocus function as an example of autofocus. Figure 29 illustrates the change in the relative distance between the stage unit 30 and the objective lens unit 25, expressed as movement of the objective lens unit 25 in the Z direction. The left side of the figure shows the positional relationship between the stage unit 30, the substrate CB on which the chip resistors CR are mounted, and the objective lens unit 25, while the right side shows a time series of the Z direction movement of the objective lens unit 25 at positions 25-1 to 25-4. As shown in Figure 29, the search autofocus function combines a wide coarse search in the Z direction with a narrower fine search during autofocusing, thereby achieving both high speed and accuracy in autofocusing. The control of moving the objective lens unit 25 and the Z stage in the vertical direction in accordance with the height of the observation object shown in Figure 29 is explained in chronological order based on the timing chart of search autofocus execution shown in Figure 30. Here, the vertical control in Figure 29 is illustrated, showing the vertical change in the focus value and the screen display switching between live display and frame skip display. In FIG. 30, the search operation first involves a rough search in a wide Z range, and then a fine search in a narrower Z range.

[0106] First, as shown in the upper part of Figure 30, a coarse search is performed as a search operation. Here, the Z stage is moved to the lower limit of the coarse search by user input, and then moved from this position toward the upper limit of the coarse search. During this time, the focus value continues to rise until halfway through, so the display becomes live and is updated sequentially. After that, the focus value decreases and does not exceed the peak, so the display stops at the image acquired at the peak position. In other words, the image is frame skipped and not updated sequentially. When the coarse search ends, the Z stage is lowered again and moved to the lower limit of the fine search. The upper and lower limit positions of the fine search are determined based on the peak position of the coarse search. Next, a fine search is started, and the Z stage is raised toward the upper limit of the fine search. Since the fine search searches at a finer pitch than the coarse search, the peak position can be found more precisely. If the focus value acquired during the fine search is greater than the maximum focus value acquired during the coarse search, the display frame is updated. After the peak position is passed, frame skipping is similarly performed and a still image is displayed. Once the peak position of the focus value has been determined through this rough search and fine search, the Z stage is moved to a position corresponding to the peak position and stopped, at which point frame skipping ends and live display continues. (Focus tracking mode procedure)

[0107] Here, the procedure for executing the focus tracking mode with focus tracking turned on will be described with reference to the flowchart in Fig. 31. First, in step S3101, the system transitions to focus tracking mode in accordance with a user instruction. Next, in step S3102, it is determined whether or not to perform plane estimation. If yes, the system proceeds to step S3112. On the other hand, if no plane estimation is to be performed, the system proceeds to step S3103.

[0108] In step S3103, the stage unit 30 is moved to the observation position in accordance with the user's instructions. Then, in step S3104, it is determined whether or not 3D data exists. If not, the process proceeds to step S3107, where a predetermined focus tracking process is performed. Details are as described above with reference to FIG.

[0109] On the other hand, if 3D data exists, in step S3105, it is determined whether there is a large difference between the current height position and the height of the 3D data at the XY position of the field of view movement destination. For example, it is determined whether the difference between the current height position and the height of the 3D data at the XY position of the field of view movement destination is larger than a predetermined threshold, and if it is not, proceed to step S3107 and perform a predetermined focus tracking process. On the other hand, if the difference is larger than the threshold, in step S3106, the Z movement of the Z stage is controlled based on the observation position and the 3D data.

[0110] Then, the process proceeds from step S3106 or step S3107 to step S3108, where it is determined whether or not to stop the movement of the stage unit 30. If not, the process returns to step S3103, and the above processing is repeated. On the other hand, if the movement of the stage unit 30 is to be stopped, the process proceeds to step S3109, and processing to stop the stage unit 30 is performed. Here, short-distance autofocus is performed, and Z coordinate measurement is performed (details will be described later). Furthermore, in step S3110, 3D data is corrected based on the measured Z coordinate. In other words, the three-dimensional reference information used for focus tracking is updated. Then, in step S3111, it is determined whether or not to end the focus tracking mode. If not, the process returns to step S3103 and repeats, and if to end, the focus tracking mode is ended. (plane estimation)

[0111] On the other hand, if a plane is to be estimated in step S3102, the process proceeds to step S3112, where plane estimation is performed. Specifically, first, in step S3112, normal autofocus and coordinate measurement at the measurement reference point are performed. Next, in step S3113, a plane is estimated from the coordinates of the measurement reference point. Furthermore, in step S3114, the stage unit 30 is moved to the observation position in accordance with a user's instruction. Then, in step S3115, the Z movement of the Z stage is controlled based on the observation position and the estimated plane. Next, in step S3116, it is determined whether or not to stop the movement of the stage unit 30. If not, the process returns to step S3114 and repeats the process. On the other hand, if the movement of the stage unit 30 is to be stopped, the process proceeds to step S3117, where coordinate measurement is performed. Here, short-distance autofocus (details will be described later) is also performed. Then, in step S3118, the plane is re-estimated based on the measured coordinates. Finally, in step S3119, it is determined whether or not to end the focus follow-up mode. If not, the process returns to step S3114 to repeat the above processing; if to end the focus follow-up mode, the process ends. (One-shot composite)

[0112] The magnification observation device according to this embodiment can also perform depth stacking, which generates 3D image data by combining images captured in the height direction. When performing depth stacking, the user can specify upper and lower height limits as a range in the height direction. Alternatively, the device may be equipped with a one-shot synthesis function that automatically sets the height range without requiring user specification. This type of one-shot synthesis can be performed by the processor unit 80 in FIG. 2. In the example of FIG. 2, the processor unit 80 implements the function of the one-shot synthesis unit 98.

[0113] The one-shot synthesis procedure will now be described with reference to the flowchart in FIG. 32 and the schematic diagram in FIG. 36. First, in step S3201, a user instruction to execute one-shot synthesis is accepted. For example, the user shown in FIG. 16 clicks the "One-shot Synthesis" button 249 on the user interface screen of the magnified image observation program. Next, in step S3202, a height movement range is determined based on 3D data corresponding to the field of view range. Then, in step S3203, the upper Z stage and / or lower Z stage are controlled so that the lower limit height Zmin of the height movement range is reached. Then, in step S3204, the Z stage is moved in a direction that increases the distance between the objective lens unit 25 and the stage unit 30. Next, in step S3205, an image is captured, and a focus value for each pixel is calculated based on the captured image. The focus value is calculated by the focus degree evaluation unit 91. Next, in step S3206, it is determined whether the movement range, in this case the upper limit height Zmax, has been reached. If not, the process returns to step S3204, and the above process is repeated. On the other hand, if the movement range is reached, the process proceeds to step S3207, where a depth stacking image, which is 3D image data, is generated based on the brightness value corresponding to the height position where the focus value is maximized for each pixel. In this way, 3D image data having height information is synthesized with appropriate upper and lower limit settings, even without the user explicitly specifying the upper and lower limits. One-shot synthesis may be triggered by clicking the "One-shot synthesis" button 249 shown in FIG. 16, etc., or may be performed when transitioning to a stop sequence. The relative distance movement of the short-distance autofocus when transitioning to a stop sequence may also serve as the relative distance movement for one-shot synthesis. Since the depth stacking image is displayed and the relative distance is controlled to the in-focus position, a live image in focus can be displayed immediately when transitioning to a moving sequence. (Stop sequence)

[0114] Although the stop sequence includes short-distance autofocus and one-shot composition at the stop position, the stop sequence is not limited to this example. When transitioning to the stop sequence, the image processing unit 84 may generate a composite image and display the composite image on the display unit 70. Examples of composite images generated by the image processing unit 84 include depth composite images, 3D composite images, pixel-shifted images, super-resolution images, HDR images, and HDR video. If a recording instruction is received from the user during the stop sequence, the displayed composite image is saved in the storage unit 53. When transitioning back to the moving sequence, the composite image is cleared, and a live image is displayed on the display unit 70. When transitioning to the stop sequence, short-distance autofocus and generation of one or more types of composite images may be performed. For example, when generating an HDR image, short-distance autofocus is performed at the stop position, followed by image capture for generating the HDR image, and the image processing unit 84 generates the HDR image based on the obtained image data. [Embodiment 3]

[0115] The magnification observation device can also be equipped with an inclined observation function. Such an example is shown in Fig. 33 as a magnification observation device according to embodiment 3. The magnification observation device 300 shown in this figure is a front view showing the imaging system 1. In the magnification observation device shown in Fig. 33, the same members as those explained in the above-mentioned embodiment 1 etc. are given the same reference numerals and detailed explanations thereof will be omitted as appropriate.

[0116] The imaging system 1 comprises a stage unit 30 on which an object to be observed is placed, and a support base 40 that supports the head unit 4. The support base 40 comprises a stage fixing mechanism 42 that holds the stage unit 30 in a state where it can move in a horizontal plane or up and down, and a head tilting mechanism 44 that tilts the head unit 4 while holding the stage unit 30. The stage fixing mechanism 42 and head tilting mechanism 44 are fixed to a base unit 41. The base unit 41 is flat, allowing the support base 40 to stand independently in a stable manner.

[0117] By tilting the head unit 4 relative to the stage unit 30 using the head tilt mechanism 44, tilted observation is possible, in which the object being observed is observed from an oblique direction. In particular, by swinging the head unit 4 from a vertical position left or right around the swing axis 45 as a rotation axis, observation is possible from either the left or right direction, thereby increasing the degree of freedom in observation by observing from different viewpoints. Furthermore, such tilted observation requires eucentric observation, in which the observation field does not change even when the head unit 4 is tilted. For this reason, it is desirable to adjust the height of the stage unit 30 in advance so that the observation surface of the object being observed coincides with the center of the swing axis 45 during tilted observation. The magnification observation device is also configured to be able to execute a focus tracking mode during tilted observation. If the head unit 4 is tilted by more than a predetermined angle relative to the stage unit 30 during 3D shape tracking mode execution, the 3D shape tracking mode is canceled and the device automatically switches to a general-purpose mode using the focus value. Even in such a case, the device automatically switches to the 3D shape tracking mode when the tilt angle of the head unit 4 relative to the stage unit 30 is readjusted to within the predetermined angle. Furthermore, the general-purpose mode can always be executed regardless of the tilt angle of the head unit 4 relative to the stage unit 30. An angle sensor may be used as a means for detecting tilt of a predetermined angle or more. In this case, for example, when the angle sensor detects that the head unit 4 is tilted 15 degrees or more relative to the stage unit 30, the 3D shape tracking mode may be canceled and the general-purpose mode may be automatically switched to, and when the tilt angle is adjusted back to 15 degrees or less, the 3D shape tracking mode may be automatically switched back on and off. Alternatively, the 3D shape tracking mode may be automatically switched on and off using a physical locking means for fixing the tilt angle at 0 degrees, i.e., for fixing the head unit 4 in an upright position relative to the stage unit 30. For example, the 3D shape tracking mode can be executed when the tilt angle is locked at 0 degrees, but the 3D shape tracking mode may be automatically switched back to the general-purpose mode when it detects that the locked state has been released. [Embodiment 4]

[0118] The magnification observation device can further include a side camera unit 18. The side camera unit 18 captures images of the stage unit 30 from a side view. This allows the positional relationship between the observation object placed on the mounting surface of the stage unit 30 and the objective lens unit 25 to be grasped, and the positional relationship between the two can be confirmed in real time during observation, preventing unintended collisions. The side camera image captured by the side camera unit 18 is positioned so that the lower end of the objective lens unit 25 and the upper end of the stage unit 30 are visible. Furthermore, by saving the side camera image when capturing a magnification observation image with the objective lens unit 25, it becomes easier to visually understand at a later date what observation object was photographed and in what manner. A magnification observation device equipped with such a side camera unit 18 is shown in a side view in FIG. 34 as a fourth embodiment. In the magnification observation device 400 shown in this figure, the same components as those described in the first embodiment and the like are designated by the same reference numerals, and detailed descriptions thereof will be omitted as appropriate. As shown in these figures, the magnification observation device 400 includes, as the imaging system 1, a support base 40 that supports a stage unit 30 and a head unit 4. The head unit 4 also includes a side camera unit 18. (Side camera image)

[0119] The side camera unit 18 is positioned so that it can capture images of the top surface of the observation object and the tip of the objective lens unit 25. An example of a side camera image captured by the side camera unit 18 is shown in FIG. 35. As shown in this figure, the magnification and observation field of view of the side camera image SI are set so that the positional relationship between the tip of the objective lens unit 25, the observation object, and the stage unit 30 can be grasped on a single screen. The virtual focal position of the objective lens unit 25 may also be displayed. Furthermore, an edge image in which an edge ED is extracted from the image may be displayed, as shown in bold in FIG. 35. For example, the image processing unit 84 detects the top edge of the observation object and the tip of the objective lens unit 25 by edge extraction from the image captured by the side camera unit 18. This allows calculations to be performed to avoid collisions when photographing or measuring the observation object. Based on the calculation results, a warning can also be issued if there is a risk of a collision. As one form of warning, a warning message such as "Be careful of collisions" is displayed on the side camera image SI on the screen of the display unit 70 shown in FIG. 35. The warning message can be highlighted by highlighting, underlining, changing the display color, or other enhancements to make it more noticeable to the user. Voice guidance or a warning may be emitted from a speaker, or a warning lamp may be turned on, such as by flashing a light. It goes without saying that these may also be used in combination as appropriate.

[0120] Furthermore, in the side camera image SI, the objective lens unit 25 and the object of observation may be distinguished from the background image to make them easier to see. For example, the depth of field of the side camera unit 18 may be adjusted so that only the depth range of the stage unit 30 is clearly captured. Also, a background determination unit may be provided to automatically determine and extract the background area from the side camera image SI based on stationary portions of the image or fluctuations in brightness. In this case, the brightness of the illumination light may be changed. Alternatively, a background influence reduction unit may be provided to mask the background portion. Such a background determination unit and background influence reduction unit may be provided as separate components, or these functions may be assigned to the image processing unit 84. [Industrial Applicability]

[0121] The magnification observation device, magnified image observation method, magnified image observation program, computer-readable recording medium, and device storing the same of the present invention can be suitably used in microscopes, reflective and transmissive digital microscopes, and the like. [Explanation of symbols]

[0122] 100, 200, 300, 400...Magnifying observation device 1. Imaging system 2...Control system 3...Cable section 4...Head section 5...Field of view shift mechanism 10...Camera section 11...Imaging optical system 12...imaging element; 13...imaging element control circuit 15...Autofocus sensor 16...Upper Z elevator 18...Side camera section 20...Microscope lens part 25...objective lens unit; 25-1 to 25-9...relative movement locus of the objective lens unit 26, 26', 26"...Relative movement trajectory 27, 27', 27", 27''... relative distance changes 28, 28-1, 28-2...Direction of change in relative distance 30...Stage section 35...Lower stage elevator 36...Motor control circuit 37...Stepping motor 40...Support stand 41...Base 42...Stage fixing mechanism 43...Camera mounting part 44...Head tilt mechanism 45...Oscillating shaft 50...main body section; 52...display control section 53...Storage section; 54...Interface; 55...operation unit; 55a...movement direction indicator; 55b...joystick 55d...Route guide release section 56...Memory section; 57...Buffer memory 60...Lighting section 61...Beam splitter 62…Coaxial epi-illumination section 63...Ring lighting unit 66...Lighting control unit 70...Display section 80...Processor section 83...Movement control unit 84...Image processing unit 89...Height information acquisition unit 90...Focus control section 91...Focus degree evaluation unit 92...Frame skip section 93...Focus sequence execution unit 94...Movement stop detection unit 95...Motion detection unit 96...Stop detection unit 97...judgment department 98...One-shot synthesis section 230...Navigation image registration screen 231...Image display area 232…Operation area 233...Navigation area 240...3D navigation image registration screen 241...Image display area 242…Operation area 243... "3D Navigation Registration" button 244...Low magnification imaging window 245...Lens magnification adjustment column 246...Focus adjustment field 247...Execute button 248..."Focus Tracking" button 249... "One-shot composite" button 250…Navigation area 252..."Plane Fit" button 254..."Use registered 3D data" checkbox WK...observation object CR1 to CR3: Chip resistors CB…board SD…Land AX…Optical axis NI...Navigation image FR…Rectangular shape RI…area DR1: Relative visual field movement direction DR2...Relative movement trajectory DS: Judgment threshold SQ1...Multi-point height acquisition (autofocus operation) SQ2...Sequence on the move SQ3...Stop sequence (autofocus operation) SQ4...Sequence on the move SQ5...Stop sequence (autofocus operation) TH1, TH3...Lower limit height TH2, TH4...Maximum height OA: Observation field FF1, FF2, FF3...Focus frame SI: Side camera image ED...Edge

Claims

1. a stage portion for placing an object to be observed; an objective lens unit arranged to face an observation object placed on the stage unit; a camera unit that captures an image of an observation object formed through the objective lens unit and generates image data representing the image; a field of view moving mechanism that changes the relative position of the optical axis of the objective lens unit on the stage unit to move the observation field of view of the camera unit; a display control unit that displays an image of the observation object on a display unit based on image data generated by the camera unit; a focus degree evaluation unit that calculates a focus degree feature amount that indicates a focus degree of image data that indicates an image of an observation object formed through the objective lens unit; a focus adjustment mechanism that adjusts the focus of image data by moving at least one of the stage unit and the objective lens unit along the optical axis of the objective lens unit, thereby sequentially changing the relative distance between the focal position of the objective lens unit and the observation object in directions toward and away from each other along the optical axis of the objective lens unit; a focus-degree feature amount calculated by the focus-degree evaluation unit based on image data showing images of the object to be observed that are sequentially formed through the objective lens unit while the focus adjustment mechanism is changing the relative distance in either the near or far direction along the optical axis direction, and a maximum buffered focus-degree feature amount among the focus-degree feature amounts calculated by the focus-degree evaluation unit based on image data showing images of the object to be observed that are sequentially formed through the objective lens unit before the focus adjustment mechanism changes the relative distance in that direction; if the sequentially calculated focus-degree feature amount is smaller than the buffered maximum focus-degree feature amount, skipping updating of the live display on the display unit by the display control unit and discarding the sequentially calculated focus-degree feature amount; a frame skip unit that, when the sequentially calculated focus-degree feature amount is greater than the buffered maximum focus-degree feature amount, switches the image of the observation object displayed on the display unit to an image showing an image of the observation object newly formed through the objective lens unit while changing the relative distance in either the near or far direction along the optical axis direction by the focus adjustment mechanism, thereby updating the live display, and updates the buffered maximum focus-degree feature amount with the focus-degree feature amount calculated based on the image showing the image of the observation object newly formed through the objective lens unit; A magnification observation device comprising:

2. The magnification observation device according to claim 1, the focus level evaluation unit uses a focus value calculated based on image data as the focus level feature amount, The frame skip unit is a magnification observation device configured to update the display content of the display unit by the display control unit based on a comparison of the focus value of the image data after movement by the focus adjustment mechanism, which is sequentially calculated by the focus degree evaluation unit, with the focus value of the image data before movement displayed on the display unit, when adjusting the focus of the image by the focus adjustment mechanism.

3. The magnification observation device according to claim 1, further comprising: an autofocus sensor for performing autofocus between the objective lens unit and an observation object; the focus level evaluation unit uses a difference between a focus position and a current position as the focus level feature amount based on a measurement value measured by the autofocus sensor, The frame skip unit is a magnification observation device configured to update the display content of the display unit by the display control unit when the measurement value after movement by the focus adjustment mechanism, which is measured sequentially by the autofocus sensor when adjusting the focus of the image with the focus adjustment mechanism, is improved in comparison with the in-focus position compared with the measurement value before movement displayed on the display unit.

4. The magnification observation device according to any one of claims 1 to 3, further comprising: a height information acquisition unit that acquires height information at different positions of the observation object as three-dimensional reference information; a magnification observation device that, when the field of view movement mechanism moves the observation field of view displayed on the display unit by the display control unit, adjusts the relative distance between the focal position of the objective lens unit and the object to be observed by the focus adjustment mechanism based on height information of a position corresponding to the field of view movement position of the destination, among the three-dimensional reference information acquired by the height information acquisition unit.

5. The magnification observation device according to claim 4, The focus adjustment mechanism is configured to perform focus adjustment based on height information estimated based on height information around a part that does not include height information acquired by the height information acquisition unit when the part is displayed.

6. The magnification observation device according to claim 4 or 5, A magnification observation device, wherein the height image is a low-magnification image captured at a magnification of the objective lens unit lower than the magnification of the image displayed on the display unit by the display control unit.

7. The magnification observation device according to any one of claims 1 to 6, The focus adjustment mechanism is configured to determine the direction of movement according to changes in the focus degree feature amount based on image data obtained when the field of view movement mechanism is moving the observation field of view, and move the focus adjustment mechanism.

8. The magnification observation device according to any one of claims 1 to 6, further comprising: a focus sequence execution unit that causes the display control unit to display a live image of the observation object on the display unit based on image data obtained by executing a focus sequence during field of view movement in which the focus adjustment mechanism adjusts a relative distance between a focal position of the objective lens unit and the observation object based on focus degree feature amounts of image data sequentially calculated by the focus degree evaluation unit while the observation field of view is moved by the field of view movement mechanism, The focus sequence execution unit estimates the movement plane of the focus position of the objective lens unit by fitting a predetermined geometric shape to height information at different positions of the observation object during a focus sequence while the field of view is moving, and determines and moves a target height value based on the field of view movement position and information on the estimated movement plane.

9. The magnification observation device according to any one of claims 1 to 8, The focus adjustment mechanism is configured to perform a first autofocus control that is executed in response to a user's instruction, and a second autofocus control that is executed when the movement of the observation field by the field of view movement mechanism is stopped.

10. The magnification observation device according to any one of claims 1 to 8, The magnification observation device is configured such that the focus adjustment mechanism immediately stops measuring the focus value when an instruction to stop autofocusing is received while measuring the focus value for autofocusing is being performed.

11. The magnification observation device according to any one of claims 1 to 10, A magnification observation device in which the display of a target area to be focused by autofocus control is changed according to the focus control when autofocus is performed by the focus adjustment mechanism.

12. a magnified image observation method comprising the steps of: capturing an image of an observation object placed on a stage unit by a camera unit via an objective lens unit and displaying the image on a display unit; adjusting the focus of image data by sequentially changing the relative distance between the focal position of the objective lens unit and the observation object in a direction toward and away from each other along the optical axis of the objective lens unit using a focus adjustment mechanism; and relatively moving the optical axis of the objective lens unit and the stage unit by a field movement mechanism so that the position of the optical axis of the objective lens unit on the stage unit changes and the observation field output to the display unit moves, a step of displaying on the display unit an image of an observation field including an observation object based on image data generated by the camera unit, at a relative distance between the stage unit and the objective lens unit that is changed by the focus adjustment mechanism; a step of calculating a focus degree feature amount indicating a focus degree of image data indicating an image of an observation object formed through the objective lens unit by a focus degree evaluation unit, and adjusting the focus of the image by the focus adjustment mechanism; a focus-degree feature amount calculated by the focus-degree evaluation unit based on image data showing images of the object of observation formed sequentially through the objective lens unit while the relative distance between the focal position of the objective lens unit and the object of observation is changed along the optical axis of the objective lens unit by the focus adjustment mechanism is sequentially compared with a buffered maximum focus-degree feature amount among focus-degree feature amounts calculated by the focus-degree evaluation unit based on image data showing images of the object of observation formed sequentially through the objective lens unit before the relative distance between the focal position of the objective lens unit and the object of observation is changed along the optical axis of the objective lens unit by the focus adjustment mechanism; and if the sequentially calculated focus-degree feature amount is smaller than the buffered maximum focus-degree feature amount, a display control unit skips updating of the live display on the display unit and discards the sequentially calculated focus-degree feature amount. a step of updating the live display by the display control unit by switching the image of the observation object displayed on the display unit to an image showing an image of the observation object newly formed through the objective lens unit while changing the relative distance in either the near or far direction along the optical axis direction by the focus adjustment mechanism, when the sequentially calculated focus degree feature amount is greater than the buffered maximum focus degree feature amount, and updating the buffered maximum focus degree feature amount with the focus degree feature amount calculated based on the image showing the image of the observation object newly formed through the objective lens unit; A method for observing a magnified image, comprising:

13. a stage portion for placing an object to be observed; an objective lens unit arranged to face an observation object on the stage unit; a camera unit that captures an image of an observation object formed through the objective lens unit and generates image data representing the image; a display unit that displays an image of an observation field including an observation object based on image data generated by the camera unit; a field of view moving mechanism that moves the optical axis of the objective lens unit and the stage unit relatively to each other so that the position of the optical axis of the objective lens unit on the stage unit changes and the observation field of view output to the display unit moves; a focus adjustment mechanism that adjusts the focus of image data by sequentially changing the relative distance between the focal position of the objective lens unit and an observation object along the optical axis of the objective lens unit in a direction toward and away from each other; A magnified image observation program for operating a magnification observation device comprising: a function of displaying on the display unit an image of an observation field including an observation object based on image data generated by the camera unit at a relative distance between the stage unit and the objective lens unit that is changed by the focus adjustment mechanism; and a function of calculating a focus degree feature amount indicating a focus degree of image data indicating an image of an observation object formed through the objective lens unit by a focus degree evaluation unit, and adjusting the focus of the image by the focus adjustment mechanism; a focus-degree feature amount calculated by the focus-degree evaluation unit based on image data showing images of the object of observation formed sequentially through the objective lens unit while the relative distance between the focal position of the objective lens unit and the object of observation is changed along the optical axis of the objective lens unit by the focus adjustment mechanism is sequentially compared with a buffered maximum focus-degree feature amount among focus-degree feature amounts calculated by the focus-degree evaluation unit based on image data showing images of the object of observation formed sequentially through the objective lens unit before the relative distance between the focal position of the objective lens unit and the object of observation is changed along the optical axis of the objective lens unit by the focus adjustment mechanism, and if the sequentially calculated focus-degree feature amount is smaller than the buffered maximum focus-degree feature amount, a display control unit skips updating of the live display on the display unit and discards the sequentially calculated focus-degree feature amount; a function of updating the live display by switching, by the display control unit, the image of the observation object displayed on the display unit to an image showing an image of the observation object newly formed through the objective lens unit while changing the relative distance in either the near or far direction along the optical axis direction by the focus adjustment mechanism, when the sequentially calculated focus degree feature amount is greater than the buffered maximum focus degree feature amount, and updating the buffered maximum focus degree feature amount with the focus degree feature amount calculated based on the image showing the image of the observation object newly formed through the objective lens unit; A magnified image observation program that enables a computer to do this.

14. A computer-readable recording medium or storage device storing the program according to claim 13.

Images