Enlarging observation device, enlarged image observation method, enlarged image observation program, and computer-readable recording medium, and apparatus storing program

Abstract

To facilitate observation of a state of unevenness of and damage to an observation object while changing an illumination direction in real time.SOLUTION: An enlarging observation device 100 comprises: an illumination unit 60 that performs illumination in a second illumination pattern illuminating in one of a plurality of illumination directions constituting a first illumination pattern; a feature quantity calculation unit 88 that, for pieces of image data of an observation object illuminated from the illumination directions of the first illumination pattern, calculates a feature quantity for every image data; and an illumination control unit 66 that can switch the illumination unit 60 between a first sequence for actuating the illumination unit in the first illumination pattern and causing a display control unit 52 to display, on a display unit 70, a live image of the observation object illuminated in the first illumination pattern, and a second sequence for actuating the illumination unit in the second illumination pattern in one of the illumination directions selected on the basis of, the feature quantities of the pieces of image data calculated by the feature quantity calculation unit 88 and causing the display control unit 52 to display, on the display unit 70, an image of the observation object illuminated in the second illumination pattern.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a magnification observation device, a magnified image observation method, a magnified image observation program, a computer-readable recording medium, and a device storing the program. [Background technology]

[0002] Optical microscopes using optical lenses and digital microscopes are used as magnification observation devices that enlarge and display subjects such as microscopic specimens and workpieces. Digital microscopes receive light reflected from or transmitted through an object to be observed placed on an XY stage via an optical system such as an objective lens, using an image sensor such as a CCD or CMOS that electrically reads each pixel arranged two-dimensionally, and displays the electrically read image of the observation field at that position on a display.

[0003] Some magnification observation devices are equipped with an illumination unit that irradiates an object on a stage with illumination light (see, for example, Patent Document 1). This illumination unit can switch the direction of light incident on the object, i.e., the illumination direction, by switching on and off each block that makes up the ring illumination. This allows still image data of the object corresponding to each illumination direction to be acquired. The user can easily set the illumination direction by comparing and selecting multiple still image data with different illumination directions displayed on the display unit.

[0004] When using a magnifying observation device, one may wish to focus on minute scratches or defects on the surface of a subject. The appearance of scratches or defects is greatly influenced not only by the lighting direction but also by their position in the observation field. Furthermore, depending on the location of scratches or defects in the observation field, they can be difficult to detect in a still image. Patent Document 1 discloses a system in which even a user unfamiliar with lighting settings, such as lighting direction, can acquire still image data of the subject as an observation image without having to consider lighting settings, such as lighting direction. After acquiring the observation image, the user can easily display the observation image with a different lighting direction via a user interface on the display. For example, even for scratches or defects that are clearly visible only under a specific lighting direction, the appropriate lighting direction can be easily selected after acquiring the observation image, making it easy for even an unfamiliar user to display an observation image with the optimal lighting direction and observe the scratches or defects.

[0005] However, with this method, if the position of a scratch or defect in the observation field is not appropriate, it is necessary to move to a different position, acquire multiple still image data with different lighting directions, and then change the lighting direction via the user interface on the display, which can be a cumbersome process, for example, if the scratch or defect is only clearly visible from a specific lighting direction or at a specific position in the observation field. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2018-013732 Summary of the Invention [Problem to be solved by the invention]

[0007] An object of the present invention is to provide a magnification observation device, a magnified image observation method, a magnified image observation program, a computer-readable recording medium, and a device storing the program, which can observe an object by changing the illumination direction in real time, making it easier to observe the surface irregularities, scratches, dents, etc.

[0008] a first illumination pattern that is capable of switching the illumination direction of irradiating illumination light toward the observation object and sequentially switches between a plurality of different illumination directions, and a second illumination pattern that illuminates in one of a plurality of illumination directions constituting the first illumination pattern; a camera unit that captures an image of the observation object formed through the objective lens unit and generates image data representing the image; a display control unit that causes the image of the observation object to be displayed on a display unit based on the image data generated by the camera unit; an area designation unit that accepts designation of a boundary shape surrounding the area or a line segment corresponding to the area as a desired area on the image data displayed on the display unit by the display control unit; a feature amount calculation unit that calculates feature amounts within the desired area designated by the area designation unit for each image data of the observation object illuminated in each illumination direction of the first illumination pattern; and a lighting control unit that can switch between a first lighting pattern in which illumination light is sequentially applied to the object of observation in a plurality of different lighting directions and a second sequence in which illumination light is sequentially applied to the object of observation in a plurality of different lighting directions, in the first sequence, the lighting control unit operates the lighting unit in the first lighting pattern in which illumination light is sequentially applied to the object of observation in a plurality of different lighting directions, the camera unit sequentially generates image data showing an image of the object of observation corresponding to the plurality of different lighting directions of the first lighting pattern, the display control unit sequentially displays the image data sequentially generated by the camera unit on the display unit, the area designation unit accepts designation of a desired area on the image data displayed on the display unit by the display control unit, the feature calculation unit calculates feature amounts within the area accepted by the area designation unit for each piece of image data showing an image of the object of observation corresponding to the plurality of different lighting directions of the first lighting pattern, and the lighting control unit further selects an illumination direction by the lighting unit based on the desired feature amount of each piece of image data calculated by the feature calculation unit, and switches the operation of the lighting unit from the first lighting pattern, which is sequentially switched to a plurality of different lighting directions, toThe selected illumination direction is switched to the second illumination pattern fixed thereto, and the process proceeds to the second sequence, in which the camera unit generates image data showing an image of the object of observation illuminated with the second illumination pattern, and the display control unit causes the display unit to display the image data showing the image of the object of observation illuminated with the second illumination pattern. With the above configuration, it is possible to automatically select and display an illumination direction that makes it easy to check the surface condition of the object of observation, thereby realizing observation that makes it easy to avoid overlooking, for example, the presence or absence of scratches or unevenness.

[0009] Furthermore, according to another aspect of the present invention, there is provided a magnified image observation method for observing an object to be observed, illuminated by an illumination unit capable of switching illumination directions, by moving an observation field of view, which is imaged by a camera unit via an objective lens unit and displayed on a display unit, the method including the steps of: operating the illumination unit with a first illumination pattern in which illumination is sequentially switched between different illumination directions; causing the camera unit to sequentially generate image data representing images of the object to be observed corresponding to a plurality of different illumination directions of the first illumination pattern; causing the display unit to sequentially display live images of the object to be observed sequentially generated by the camera unit; and executing a first sequence for accepting designation of a desired region on the image data displayed on the display unit; and for each image data of the object to be observed illuminated in each illumination direction of the first illumination pattern, calculating a feature amount of the image data within the region, selecting an illumination direction based on the feature amount of the image data, switching the operation of the illumination unit from the first illumination pattern in which a plurality of different illumination directions are sequentially switched to the second illumination pattern in which the selected illumination direction is fixed; and executing a second sequence for displaying the object to be observed illuminated with the second illumination pattern on the display unit. This makes it easier to observe the unevenness, scratches, dents, etc. on the surface of the object being observed by changing the illumination direction in real time.

[0010] Furthermore, according to a magnified image observation program according to another aspect of the present invention, there is provided a magnified image observation program for operating a magnifying observation device comprising: a stage section for placing an observation object thereon; an objective lens section arranged facing the observation object placed on the stage section; an illumination section for irradiating illumination light toward the observation object; a camera section for capturing an image of the observation object illuminated by the illumination section through the objective lens section and generating image data; and a display section for displaying an image of an observation field including the observation object based on the image data generated by the camera section, wherein the magnified image observation program operates the illumination section with a first illumination pattern that sequentially switches between different illumination directions and displays image data on the camera section showing images of the observation object corresponding to a plurality of different illumination directions of the first illumination pattern. and a second sequence of executing a second sequence of executing a first sequence of sequentially generating image data of the object to be observed illuminated in each illumination direction of the first illumination pattern, selecting an illumination direction based on the image data feature amount, switching the operation of the illumination unit from the first illumination pattern, which sequentially switches among a plurality of different illumination directions, to the second illumination pattern, which is fixed to the selected illumination direction, and displaying the object to be observed illuminated with the second illumination pattern on the display. The above configuration makes it possible to easily observe unevenness, scratches, dents, etc. on the surface of the object to be observed by changing the illumination direction in real time.

[0011] Furthermore, according to another aspect of the present invention, a computer-readable recording medium or device on which the program is recorded stores the program. Recording media include magnetic disks such as CD-ROMs, CD-Rs, CD-RWs, flexible disks, magnetic tapes, MOs, DVD-ROMs, DVD-RAMs, DVD±Rs, DVD±RWs, HD DVDs (AODs), Blu-rays (product names), UHD BDs (product names), USB memory sticks, and SSD memory sticks, as well as optical disks, magneto-optical disks, semiconductor memories, and other media capable of storing programs. Programs distributed by download over a network such as the Internet, as well as those stored on the recording media, are also included. Recording media also include devices capable of recording programs, such as general-purpose or dedicated devices on which the program is implemented in an executable form, such as software or firmware. Each process and function included in the program may be executed by computer-executable program software, or each process may be implemented by hardware such as a predetermined gate array (FPGA, ASIC), or by a combination of program software and partial hardware modules that implement some of the hardware elements. In this specification, a computer-readable medium also includes a non-transitory tangible medium or a transitory propagating signal. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a perspective view of the appearance of a magnification observation device according to Embodiment 1. FIG. [Figure 2] FIG. 2 is a block diagram of the magnification observation device of FIG. [Figure 3] FIG. 2 is a schematic diagram illustrating a configuration of an illumination unit. [Figure 4] FIG. 1 is a schematic diagram of a ring illumination. [Figure 5] FIG. 10 is a schematic diagram showing a user interface screen of a magnified image observation program including a navigation area. [Figure 6] 2 is a schematic diagram showing the relationship between a stage unit, an observation object, a navigation image, and an observation field of view. FIG. [Figure 7] FIG. 7A is a schematic side view showing the sequence of the head unit during movement, and FIG. 7B is a schematic side view showing the stopping sequence. [Figure 8] FIG. 10 is a schematic side view showing a sequence during travel when a 3D navigation image is present. [Figure 9] FIG. 10 is a schematic side view showing focus tracking with an offset function. [Figure 10] FIG. 10A is a schematic perspective view showing the operation of the head unit in the plane tracking mode, and FIG. 10B is a graph showing the estimated tilt of the plane. [Figure 11] 10 is a user interface screen showing a plane fit function. [Figure 12] 12A to 12D are schematic diagrams showing how the lighting direction changes with the search lighting function. [Figure 13] 13A to 13D are schematic diagrams showing how the lighting positions of the lighting blocks are switched. [Figure 14] FIG. 10 is a schematic diagram showing a search lighting screen. [Figure 15] FIG. 10 is a schematic diagram showing a search lighting screen. [Figure 16] FIG. 10 is a schematic diagram showing a search lighting screen. [Figure 17] 1 is a flowchart showing a search writing procedure. [Figure 18] 18 is a flowchart showing the display process in step S1704 of FIG. 17. [Figure 19] 10 is a flowchart showing a display process in a focus follow-up mode. [Figure 20] 10 is a timing chart showing the taper control of search lighting. [Figure 21] 10 is a timing chart showing search lighting during field of view movement. [Figure 22] FIG. 2 is a schematic diagram showing a multi-lighting screen. [Figure 23] FIG. 2 is a schematic diagram showing an illumination light source display screen. [Figure 24]FIG. 10 is a block diagram of a magnification observation device according to a second embodiment. [Figure 25] 10 is a schematic diagram showing the direction of movement of a joystick when moving the observation field along the edge of a cylindrical workpiece. FIG. [Figure 26] FIG. 10 is a schematic diagram showing how the direction of movement of the stage is instructed with a joystick. [Figure 27] 10A and 10B are schematic diagrams showing the tilt direction of the joystick and the movement direction of the observation field in the route tracing mode. [Figure 28] FIG. 10 is a schematic diagram showing an example in which a movement direction indicator is realized by button operation. [Figure 29] FIG. 10 is a schematic diagram showing an example in which a movement direction indicator is implemented by mouse operation. [Figure 30] 30A and 30B are schematic diagrams showing the relationship between a visual field movement trajectory and a movement direction instruction. [Figure 31] 10 is a flowchart showing a method for observing an enlarged image that realizes a route guide function. [Figure 32] 32A to 32F are perspective views showing examples of geometric shapes that define the route. [Figure 33] FIG. 10 is a schematic diagram showing how a field of view movement locus is set for chips mounted at a distance from each other on a substrate. [Figure 34] FIG. 34 is a schematic diagram showing a state in which the observation object in FIG. 33 is moved along a field of view movement trajectory set thereon. [Figure 35] FIG. 35A is a wide-area image of an object to be observed, and FIG. 35B is a schematic diagram showing a state in which the contours of FIG. 35A are extracted. [Figure 36] FIG. 36A is a schematic diagram showing the observation field while the XY stage is being moved, and FIG. 36B is a schematic diagram showing the observation field while the XY stage is being further moved. [Figure 37] Figure 37A is an image showing the observation field before switching the objective lens unit, Figure 37B is the observation field after switching the objective lens unit, and Figure 37C is the observation field after correcting the deviation of the field center from the state of Figure 37B. [Figure 38]10A and 10B are schematic diagrams showing how a field of view movement trajectory is offset by a field of view deviation correction offset function. [Figure 39] FIG. 39 is a schematic diagram showing how the field of view movement trajectory is offset in accordance with the position corresponding to the observation field of view. [Figure 40] Figure 40A is a schematic diagram showing a field of view movement trajectory and reference point specified in three-dimensional space, Figure 40B is a schematic diagram showing the reference point projected onto the XY plane, and Figure 40C is a schematic diagram showing a field of view movement trajectory defined by the projected reference point. [Figure 41] FIG. 41A is a plane defined in three-dimensional space, and FIG. 41B is a schematic diagram showing the state in which the visual field movement trajectory within this plane is projected onto the XY plane. [Figure 42] FIG. 42A is a schematic diagram showing a polygonal view movement trajectory defined in three-dimensional space, and FIG. 42B is a schematic diagram showing this polygonal view movement trajectory projected onto the XY plane. [Figure 43] FIG. 10 is a schematic diagram showing how the visual field movement direction is determined when visual field movement trajectories intersect. [Figure 44] FIG. 10 is a schematic diagram showing how a route guide function is executed on visual field movement trajectories that intersect at a twist position. [Figure 45] 10 is a schematic diagram showing how a route guide function is executed for a visual field movement trajectory on which adjacent registration points are set. FIG. [Figure 46] 10 is a schematic diagram showing an example of an operation when moving in the Z direction while the route guide function is being executed. FIG. [Figure 47] 10A and 10B are schematic diagrams showing another example of the operation when moving in the Z direction while the route guide function is being executed. [Figure 48] FIG. 10 is a front view showing a magnification observation device according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Embodiments of the present invention will be described below with reference to the drawings. However, the embodiments described below exemplify a magnification observation device, a magnification image observation method, a magnification image observation program, a computer-readable recording medium, and a device storing the same, which embody the technical concepts of the present invention. The present invention does not specify the magnification observation device, the magnification image observation method, the magnification image observation program, the computer-readable recording medium, and the device storing the same. Furthermore, this specification does not specify the components described in the claims as those of the embodiments. The dimensions, materials, shapes, and relative positions of the components described in the embodiments, unless otherwise specified, are not intended to limit the scope of the present invention and are merely illustrative examples. The size and positional relationships of components shown in the drawings may be exaggerated for clarity. Furthermore, in the following description, identical names and symbols indicate identical or similar components, and detailed descriptions will be omitted as appropriate. Furthermore, the elements constituting the present invention may be configured with the same components, with one component serving multiple functions, or the functions of one component may be shared among multiple components.

[0014] The magnification observation device used in the embodiments of the present invention is connected to a computer, printer, external storage device, or other peripheral device connected thereto for operation, control, display, and other processing, for example, via a serial connection such as IEEE1394, RS-232x, RS-422, or USB, a parallel connection, or a network such as 10BASE-T, 100BASE-TX, or 1000BASE-T, and the connection is electrically, magnetically, or optically connected for communication. The connection is not limited to a physical connection using a wire, but may also be a wireless connection using radio waves, infrared rays, optical communications, or the like, such as a wireless LAN such as IEEE802.x or Bluetooth (registered trademark). Furthermore, recording media for exchanging data and saving settings may include memory cards, magnetic disks, optical disks, magneto-optical disks, semiconductor memories, and the like. Note that in this specification, the terms "magnification observation device" and "magnified image observation method" refer not only to the magnification observation device itself, but also to a magnification observation system that combines it with peripheral devices such as a computer and external storage device.

[0015] Furthermore, in this specification, the term "magnification observation device" is not limited to the system itself that performs magnification observation, nor to devices or methods that perform input / output, display, calculation, communication, and other processing related to imaging using hardware. Devices and methods that implement processing using software are also within the scope of the present invention. For example, devices and systems that incorporate software, programs, plug-ins, objects, libraries, applets, compilers, modules, macros that run on specific programs, etc. into general-purpose circuits or computers to enable imaging itself or related processing also fall under the category of the magnification observation device of the present invention. Furthermore, in this specification, the term "computer" includes not only general-purpose and dedicated electronic computers, but also workstations, terminals, and other electronic devices. Furthermore, in this specification, the term "program" is not limited to a standalone program, but can also be used in the following ways: as part of a specific computer program, software, service, etc.; ... [Embodiment 1]

[0016] A magnification observation device 100 according to a first embodiment of the present invention will be described below with reference to FIGS. 1 and 2. As shown in FIG. 1, the magnification observation device 100 is broadly divided into an imaging system 1 and a control system 2. The imaging system 1 includes an illumination unit 60 for illuminating an observation object WK, such as a specimen, sample, workpiece, or other subject, and a head unit 4 for capturing an image of the observation object WK illuminated by the illumination unit 60. As shown in FIG. 2, the head unit 4 includes a camera unit 10 including an image sensor 12 and a microscope lens unit 20 detachably attached to the tip of the camera unit 10. The microscope lens unit 20 constitutes an imaging optical system (lens optical system) 11 made up of multiple optical lenses. Here, the microscope lens unit 20 includes an objective lens unit 25 as shown in FIG. 3. The head unit 4 also functions as an imaging means for receiving reflected or transmitted light of the illumination light.

[0017] The magnification observation device also includes a focus adjustment mechanism. The focus adjustment mechanism adjusts the focus of image data by moving the relative distance between the focal position of an optical imaging system, such as the objective lens unit 25, and the object of observation either toward or away from each other along the optical axis of the optical imaging system. The focus adjustment mechanism is composed of, for example, an upper Z elevator 16, such as an upper Z stage, or a lower stage elevator 35, such as a lower Z stage. The focus adjustment mechanism may move the head unit 4 along the optical axis of the optical imaging system, or may move the objective lens unit 25 along the optical axis of the optical imaging system relative to the head unit 4. Alternatively, the focus adjustment mechanism may move a lens in the objective lens unit 25 along the optical axis of the optical imaging system. Furthermore, if the objective lens unit 25 includes a variable-focus lens, such as a liquid lens, the focus of the liquid lens may be controlled by a control circuit to move the relative distance between the focal position of the optical imaging system, such as the objective lens unit 25, and the object of observation either toward or away from each other along the optical axis of the optical imaging system. The focus adjustment mechanism includes a liquid lens or the like whose focus is controlled by a focus adjustment mechanism or a control circuit. (Camera section 10)

[0018] As shown in Fig. 3, the camera unit 10 includes an image sensor 12 that electrically reads reflected light incident via an imaging optical system 11 from an observation object WK illuminated by an illumination unit 60. In this example, the image sensor 12 uses a CMOS, but other light-receiving elements such as a CCD can also be used. The objective lens unit 25 is disposed facing the observation object on the stage unit 30. The camera unit 10 also captures an image of the observation object formed via the objective lens unit 25 and generates image data representing this image.

[0019] The imaging system 1 also includes a stage unit 30 on which the observation object WK is placed, an upper Z stage as a first focus adjustment unit that adjusts the focus by changing the relative distance in the optical axis direction between this stage unit 30 and the head unit 4, and an upper Z elevator 16 for driving this upper Z stage. Light is incident on the observation object WK placed on this stage unit 30 via an imaging optical system 11, and reflected light reflected by the observation object WK or transmitted light irradiated from the bottom side of the observation object WK is electrically read by an imaging element 12 of the camera unit 10.

[0020] The control system 2 further includes a main body 50 having a display 70 that displays an enlarged image captured by the camera 10. The camera 10 is connected to the main body 50 via a cable 3. The display 70 displays an image of the observation field including the object to be observed based on image data generated by the camera 10. In the example of FIG. 1, the display 70 is provided integrally with the main body 50, but the display may also be a separate member from the main body 50. For example, the main body 50 may include a display control unit 52 that generates the display content to be displayed on the display 70, as well as a display connection interface that connects the display 70 to the main body 50.

[0021] The main body 50 also includes a processor 80. As shown in the block diagram of FIG. 2, the processor 80 realizes multiple functions (details will be described later). The main body 50 can be a general-purpose computer with a dedicated program installed, or a specially designed device. In this example, a general-purpose computer with a magnified image observation program for operating the magnification observation device installed is used as the main body. The main body 50 includes the processor 80, a display control unit 52, a storage unit 53, an interface 54, an operation unit 55, and a memory unit 56.

[0022] The cable unit 3 is an electrical cable for transmitting image information obtained by the image sensor 12 of the camera unit 10 to the body unit 50. In addition to the electrical cable, the cable unit 3 may also include an optical cable for transmitting illumination light from the body unit 50 to the head unit 4. In this case, the cable unit 3 may be an integrated unit of an electrical cable and an optical cable, or these may be provided separately. (Display unit 70)

[0023] The display unit 70 can be a monitor such as an LCD display, organic EL display, or CRT. The main body 50 is also connected to an operation unit 55 that enables the user to perform various operations. The operation unit 55 is an input device such as a console or a mouse. In this example, the display unit 70 and operation unit 55 can be integrated into the main body 50 or can be external components. Furthermore, if the display unit 70 is configured as a touch panel, the display unit 70 and operation unit 55 can be configured as an integrated unit.

[0024] The operation unit 55 is connected to the main unit 50 or a computer via a wired or wireless connection, or is fixed to the computer. Typical operation units 55 include various pointing devices, such as a mouse, keyboard, slide pad, TrackPoint, tablet, joystick, console, jog dial, digitizer, light pen, numeric keypad, touchpad, and AccuPoint. These operation units 55 can be used not only to operate the magnification observation operation program, but also to operate the magnification observation device itself and its peripheral devices. Furthermore, a touch screen or touch panel can be used for the display itself that displays the interface screen, allowing the user to input and operate the screen by directly touching it with their hands, or voice input or other existing input means can be used, or both. In the example of Figure 1, the operation unit 55 is composed of a mouse, keyboard, and joystick 55b. (Lighting section 60)

[0025] The illumination unit 60 generates illumination light that illuminates the observation object WK that is imaged on the imaging element 12. A schematic configuration of the illumination unit 60 is shown in FIG. 3. The illumination unit 60 includes an illumination control unit 66. The illumination control unit 66 controls the illumination light according to set illumination conditions. The illumination conditions adjust the brightness of the illumination light emitted from the illumination unit 60. For example, the brightness of the illumination light can be adjusted by adjusting the irradiation time or irradiation intensity of the illumination unit 60. The illumination unit 60 may include an LED as a light source. The light emission of the light source may be controlled by the illumination control unit 66, or the brightness of the irradiated light may be controlled by a light-blocking member such as a shutter.

[0026] For illumination, epi-illumination such as ring illumination (dark-field illumination) or coaxial illumination (bright-field illumination) can be used. Furthermore, illumination methods such as transmitted illumination and differential interference illumination can be appropriately used for the illumination unit 60. Epi-illumination is an illumination method in which illumination light falls on the observation object from above, and includes ring illumination, coaxial illumination, etc. Coaxial illumination and ring illumination have different angles of incidence on the observation object, resulting in different shadows. These can be used depending on the observation object. Furthermore, the microscope lens unit 20 detachably attached to the tip of the camera unit 10 and the objective lens unit 25 detachably attached to the microscope lens unit 20 may be compatible with both coaxial illumination and ring illumination, only coaxial illumination, or only ring illumination. Trans-illumination is an illumination method in which illumination light is irradiated onto the observation object from below. In the example of FIG. 3, the illumination unit 60 includes a coaxial epi-illumination unit 62 and a ring illumination unit 63. The coaxial epi-illumination unit 62 is bent via a beam splitter 61 so as to be parallel to the optical axis AX of the objective lens unit 25. The illumination unit 60 can be either built into the head unit 4 or configured as a separate unit that can be detached from the head unit 4. The light source of the illumination unit 60 may be an LED or an LD, etc., provided in each of the coaxial epi-illumination unit 62 and the ring illumination unit 63, or a common illumination light source may be built into the main body unit 50 so that illumination light is transmitted to the illumination unit in the head unit via an optical cable.

[0027] Coaxial lighting and ring lighting can also be equipped with oblique lighting. An example of a coaxial lighting and ring lighting equipped with such oblique lighting functionality is shown in the schematic diagram of Figure 4. The lighting unit 60 shown in this figure comprises a circular coaxial epi-illumination unit 62 located on the inner ring side and a similarly circular ring lighting unit 63 located on the outer ring side. Both the coaxial epi-illumination unit 62 and the ring lighting unit 63 are divided into sections along the circumference, and by switching the lighting of each divided block, it is possible to provide oblique lighting with different lighting directions.

[0028] In this way, the illumination unit 60 is capable of switching the illumination direction in which illumination light is irradiated toward the observation object. Furthermore, the illumination unit 60 is capable of irradiating with either a first illumination pattern in which the illumination direction in which illumination light is irradiated is sequentially switched, or a second illumination pattern in which illumination is irradiated in one of the illumination directions constituting the first illumination pattern. The illumination pattern switching is performed by the illumination control unit 66.

[0029] In the example of FIG. 4 , the ring illumination unit 63 is divided into four illumination blocks 63a, 63b, 63c, and 63d, and each of the illumination blocks 63a to 63d can be individually lit. By turning on one of the illumination blocks 63a to 63d and turning off the others, the illumination direction in which the illumination light is irradiated toward the observation object can be changed. The number of divisions of the ring illumination unit is not limited to four, and may be three or less, five or more. This can be changed as appropriate depending on the observation purpose, etc. The lighting sequence of the illumination blocks 63a to 63d in the first illumination pattern is predetermined, and may be, for example, clockwise lighting or figure-eight lighting. The lighting sequence of the illumination blocks 63a to 63d in the first illumination pattern may be stored in the memory unit 56 as lighting sequence information. Furthermore, the first illumination pattern may be selected by sequentially lighting up each illumination block of the coaxial epi-illumination unit 62, sequentially lighting up each illumination block of the ring illumination unit 63, or sequentially lighting up each illumination block of the coaxial epi-illumination unit 62 and each illumination block of the ring illumination unit 63, depending on the microscope lens unit 20 and the objective lens unit 25. For example, ring illumination may be used preferentially. In this case, when an objective lens unit 25 compatible with only coaxial illumination is used, the coaxial epi-illumination unit 62 may be selected as illumination for the first illumination pattern, and when an objective lens unit 25 compatible with both coaxial illumination and ring illumination is used, the ring illumination unit 63 may be selected as illumination for the first illumination pattern. (Lighting control unit 66)

[0030] The illumination control unit 66 can switch the illumination light control between a first sequence and a second sequence. In the first sequence, the illumination unit 60 operates in a first illumination pattern. A live image of the observation object illuminated with the first illumination pattern is displayed on the display unit 70. This first sequence is also referred to as search lighting (details will be described later). In the second sequence, the illumination unit 60 operates in a second illumination pattern using one of the illumination directions selected based on the feature values ​​of each image data calculated by the feature calculation unit 88 (details will be described later). An image of the observation object illuminated with the second illumination pattern is displayed on the display unit 70. This makes it possible to automatically select and display an illumination direction that makes it easier to check the surface condition of the observation object. For example, by changing the illumination direction in real time, an observation environment is realized in which unevenness, scratches, dents, etc. on the surface of the observation object are less likely to be overlooked.

[0031] When the observation field of view is moved along the field of view movement trajectory (described in detail below), the illumination control unit 66 controls the illumination unit 60 to emit light with a first illumination pattern. This allows the illumination direction to be automatically changed while the observation field of view is moving, making it possible to observe the surface condition of the object under observation with different illumination, making it easier to find scratches, chips, etc. Furthermore, when the illumination control unit 66 stops the movement of the observation field of view by the field of view movement mechanism 5, it analyzes multiple image data with different illumination directions stored in the buffer memory 57, selects an image with an illumination direction in which the scratches are most clearly visible, and controls the illumination unit 60 to emit light with a second illumination pattern fixed to that illumination direction. As a result, when the movement of the observation field of view is stopped, image data with an illumination direction in which the scratches are most clearly visible is displayed, realizing observation that is ideal for searching for scratches.

[0032] The illumination unit 60 shown in FIG. 1 includes a coaxial incident illumination unit 62 (see FIG. 3) for irradiating the observation object WK with coaxial incident light, and a ring illumination unit 63 for irradiating ring-shaped illumination light from a ring-shaped light source. These illumination units are connected to the main body 50 via cables. The main body 50 is equipped with a connector for connecting the cables. The ring illumination unit 63 can switch between all-around illumination and side illumination. To achieve this, the ring illumination unit 63 can be configured to have multiple LEDs arranged in a ring shape and some of the LEDs can be turned on and off, or to have a turret-type mask that cuts off part of the illumination light. The illumination control unit 66 controls the lighting and switching of these illumination lights. (Field of view movement mechanism 5)

[0033] The magnification observation device 100 also includes a field-of-view moving mechanism 5 that moves the observation field displayed on the display unit 70. The field-of-view moving mechanism 5 changes the relative position between the objective lens unit 25 and the stage unit 30 to change the position of the optical axis AX of the objective lens unit 25 on the mounting surface of the stage unit 30. When the field-of-view moving mechanism 5 moves the relative position between the objective lens unit 25 and the stage unit 30, an updated image can be displayed on the display unit 70 in the observation field after the movement. In the example of FIG. 1, an XY stage that moves the stage unit 30 is used as the field-of-view moving mechanism 5. However, the present invention may also use a field-of-view moving mechanism that moves the objective lens unit instead of or in addition to this. It is sufficient for the field-of-view moving mechanism to be able to move the relative position of the objective lens unit and the stage unit as viewed from the optical axis side, i.e., the observation field within the XY plane.

[0034] The XY stage is an electrically driven stage that can move the mounting surface of the stage unit 30 in the X-axis direction and the Y-axis direction. The field of view moving mechanism 5 can also include a θ stage that can rotate the stage unit 30.

[0035] Furthermore, in addition to being able to move the stage unit 30 in the XY plane by the field of view moving mechanism 5, the stage unit 30 can also be moved in the height direction, that is, in the Z direction, by the lower stage elevator 35.

[0036] The operation of the lower stage elevator 35 will now be described. The main body 50 changes the relative distance in the optical axis direction between the stage 30 and the head 4, which has the imaging optical system 11 and the imaging element 12, in this case, the height in the z direction, by inputting control data related to the control of the stepping motor 37 to the motor control circuit 36. Specifically, the main body 50 controls the rotation of the stepping motor 37 by inputting control data necessary for controlling the lower stage elevator 35 to the motor control circuit 36, thereby raising and lowering the height z (position in the z direction) of the stage 30. The stepping motor 37 generates a rotation signal corresponding to the rotation. Based on the rotation signal input via the motor control circuit 36, the main body 50 stores the height z of the stage 30 as information related to the relative distance in the optical axis direction between the stage 30 and the imaging optical system 11. The stage 30 functions as an observation positioning unit that positions the observation position with respect to the observation object WK.

[0037] Furthermore, in this embodiment, changing the height of the stage unit 30 not only changes the relative distance between the stage unit 30 and the imaging optical system 11 in the optical axis direction, but also changes the height of the imaging optical system, i.e., the height of the head unit 4. The head unit 4 is connected to the main body unit 50 by the cable unit 3. As a result, data acquired by the head unit 4 is sent to the main body unit 50 via the cable unit 3, and the necessary processing can be performed on the main body unit 50 side. Note that the stage unit may be provided in the microscope main body, or it may be provided in the head unit that is a separate member from the main body, or an imaging unit without a stage may be provided in the head unit. An imaging unit without a stage may be attached to a mounting stand or may be handheld by the user.

[0038] The image sensor 12 can electrically read the amount of received light for each pixel arranged two-dimensionally in the x and y directions. The image of the observation object WK formed on the image sensor 12 is converted into an electrical signal according to the amount of received light at each pixel of the image sensor 12, and the signal is further converted into digital data by the image sensor control circuit 13. The main body 50 stores the digital data converted by the image sensor control circuit 13 as received light data D in the storage unit 53 together with pixel arrangement information (x, y) as two-dimensional position information of the observation object WK in a plane (x, y directions in FIG. 2) approximately perpendicular to the optical axis direction (z direction in FIG. 2). Here, the plane approximately perpendicular to the optical axis direction does not necessarily have to be a plane strictly at 90° with respect to the optical axis; it may be an observation plane within a range of inclination that allows the shape of the observation object WK to be recognized with the imaging optical system and the resolution of the image sensor 12.

[0039] In the above explanation, an example of the stage unit 30 has been given in which the observation object WK is placed on the stage unit 30, but it is also possible to use a configuration in which, for example, an arm is attached instead of the stage unit and the observation object WK is fixed to the tip of the arm. Furthermore, the head unit 4 is not only used by being attached to the camera attachment unit 43, but can also be detachable and positioned at a desired position and angle by hand or other methods. (Movement direction instruction section 55a)

[0040] The control system 2 is provided with an operation unit 55. The operation unit 55 is an input device connected to the main body 50. The operation unit 55 functions as a movement direction indicator 55a that accepts user input indicating the movement direction of the observation field on the display unit 70. The movement direction of the field of view movement mechanism 5 is indicated according to the direction input from this operation unit 55. The operation unit 55 can be a joystick, a touchpad, a mouse, a keyboard (arrow keys or specific keys), or the like. In particular, by using a joystick 55b as the operation unit 55, the user can easily intuitively indicate the movement direction of the observation field by the direction in which he tilts the joystick 55b. The movement speed can also be specified by the angle at which the joystick 55b is tilted from the vertical position.

[0041] Furthermore, the stage unit 30 can be moved in the height direction, i.e., the Z direction, by the lower stage elevator 35, and also in a plane. Specifically, it is provided with an XY stage that can move in the X-axis direction and the Y-axis direction. It can also be provided with a rotatable stage (θ stage) that rotates the stage unit 30.

[0042] In this example, both the upper Z elevator 16 and the lower stage elevator 35 are electrically driven. However, in the present invention, it is sufficient to obtain height information for the objective lens unit 25 and the stage unit 30, and it is not essential that both the upper Z elevator and the lower stage elevator are electrically driven. For example, either the upper Z elevator or the lower stage elevator may be configured to be driven manually.

[0043] 2, the main body 50 includes a processor 80, a display control unit 52, a storage unit 53, an interface 54, an operation unit 55, and a memory unit 56. This magnification observation device 100 captures an observation image using an image sensor 12 that electrically reads reflected light or transmitted light from an observation object WK fixed to a stage unit 30 that is incident via an imaging optical system 11, and displays the image on a display unit 70.

[0044] The storage unit 53 functions as a memory unit, for example, by saving image data displayed on the display unit 70 by the display control unit 52 as a moving image. The storage unit 53 also saves image data of composite images generated by the image processing unit 84 for different observation objects, in association with condition data including one or more pieces of information relating to the conditions when the composite images were captured.

[0045] The interface 54 is a connection section that allows the main body section 50 to communicate data with the head section 4, the lower stage elevator 35, etc. The memory section 56 is composed of RAM, ROM, etc. This memory section 56 includes a buffer memory 57 that sequentially stores image data captured by the camera section 10 in different illumination directions while the field of view movement mechanism 5 is moving. The operation section 55 is also a member that sets the imaging conditions for setting the conditions for capturing an image with the camera section 10, as well as other necessary settings and operations.

[0046] The buffer memory 57 temporarily stores multiple pieces of image data for different lighting directions to be displayed on the display unit 70 in the first sequence. The image data buffered in the buffer memory 57 is used when determining the lighting direction, and is therefore stored in correspondence with or associated with each lighting direction. The size of the buffer memory 57 is sufficient to store image data for at least one cycle, and the image data may be overwritten for the next cycle. Here, one cycle refers to a period covering one cycle of each of the different types of lighting included in the first lighting pattern, and each of the different types of lighting included in the first lighting pattern is turned on at least once during one cycle. In other words, by buffering image data in the buffer memory 57 for a period of one cycle or more, it is possible to buffer sufficient image data for determining the lighting direction.

[0047] If multiple image data sets with different illumination directions are stored in the buffer memory 57 when the region is specified by the region designation unit, the feature calculation unit 88 calculates the feature values ​​of the image data within the specified region for each of the stored image data, selects one of the illumination directions, and transitions from the first sequence to the second sequence. Features of image data can be variances of brightness, contrast values, etc. This allows image data to be stored in the buffer memory 57 while sequentially switching illumination directions, and when the user specifies the desired region, the illumination direction can be selected using the already stored image data. This allows for rapid switching to an appropriate illumination direction without waiting for new image data to be captured. If, when the region is specified by the region designation unit, image data for a period less than one cycle is stored in the buffer memory 57 or no image data is stored, the first sequence is maintained until image data corresponding to one cycle can be buffered in the buffer memory 57. Then, the feature calculation unit 88 calculates the feature values ​​of the image data within the specified region for each of the image data sets stored in the buffer memory 57, selects one of the illumination directions, and transitions from the first sequence to the second sequence. (Display control unit 52)

[0048] The display control unit 52 outputs image data generated by the camera unit 10 to the display unit 70. The display unit 70 displays image data of the observation field output from the display control unit 52. This display control unit 52 can be configured with a GPU or the like. In the example of FIG. 2, an example is described in which the display control unit 52 is configured as a separate member from the processor unit. Such a display control unit 52 is configured with, for example, a GPU. However, the present invention is not limited to this configuration, and the display control unit 52 may be incorporated into the processor unit. For example, the display control unit 52 may be integrated into a CPU or MPU that constitutes the processor. (Processor unit 80)

[0049] The processor unit 80 realizes functions such as a movement control unit 83, an image processing unit 84, a focus control unit 90, a height information acquisition unit 89, and a feature calculation unit 88. The movement control unit 83 controls the movement of the field of view movement mechanism 5 according to the movement direction instruction from the movement direction instruction unit 55a. The focus control unit 90 controls the focus adjustment mechanism to adjust the focus of the image. For example, it performs autofocus to automatically achieve focus. The focus control unit 90 realizes functions such as a focus degree evaluation unit 91, a frame skip unit 92, a movement stop detection unit 94, and a focus sequence execution unit 93. The focus degree evaluation unit 91 calculates a focus degree feature that indicates the focus degree of the image data displayed on the display unit 70 by the display control unit 52.

[0050] When moving in either the near or far direction along the optical axis to perform autofocus control with the focus adjustment mechanism, the frame skip unit 92 causes the display control unit 52 to update the display content on the display unit 70 if the focus degree feature amount of the image data after movement by the focus adjustment mechanism, which is sequentially calculated by the focus degree evaluation unit 91, is improved compared to the focus degree feature amount of the image data before movement displayed on the display unit 70. As a result, when adjusting focus with the focus adjustment mechanism, the display content on the display unit 70 is updated only if an image with an improved focus degree feature amount is obtained.

[0051] Furthermore, when adjusting the focus of an image with the focus adjustment mechanism, if the focus degree feature amount of the image data after movement by the focus adjustment mechanism, which is sequentially calculated by the focus degree evaluation unit 91, is worse than the focus degree feature amount of the image data before movement displayed on the display unit 70, the frame skip unit 92 does not cause the display control unit 52 to update the display content on the display unit 70. This achieves comfortable focus adjustment without the stress of switching to a display that is temporarily out of focus when conventional autofocus is performed.

[0052] Here, the focus degree feature is an index indicating the degree of focus of an image, and known parameters such as focus value and phase difference can be used. For example, a focus value such as contrast calculated based on image data can be used. In this case, when adjusting the focus of an image using the focus adjustment mechanism, the frame skip unit 92 causes the display control unit 52 to update the display content on the display unit 70 if the focus value of the image data after movement by the movement mechanism, which is sequentially calculated by the focus degree evaluation unit 91, is improved compared to the focus value of the image data before movement, which is displayed on the display unit 70. This makes it possible to perform autofocus based on focus values ​​such as contrast obtained from image data without using a dedicated sensor or the like.

[0053] Furthermore, when a phase difference is used as the focus degree feature, an autofocus sensor is provided to detect the difference between the in-focus position and the current position. In this case, when adjusting the focus of an image with the focus adjustment mechanism, the frame skip unit 92 causes the display control unit 52 to update the display content on the display unit 70 if the measurement values ​​measured sequentially by the autofocus sensor after movement by the focus adjustment mechanism are improved compared to the measurement values ​​displayed on the display unit 70 before the movement. This method does not require movement in the Z direction, thereby achieving high-speed autofocus.

[0054] While the observation field is being moved by the field of view moving mechanism 5, the focus sequence execution unit 93 executes a focus sequence during field of view movement, which adjusts the relative distance between the focal position of the optical imaging system, such as the objective lens unit 25, and the observation object WK using the focus adjustment mechanism based on information regarding the XY position of the observation field, and causes the display control unit 52 to display a live image of the observation object on the display unit 70 based on image data obtained.

[0055] The movement stop detector 94 is a component that detects movement or stop of the observation field. When the movement stop detector 94 detects movement of the observation field, a focus sequence during field movement is executed, in which the focus adjustment mechanism adjusts the relative distance between the focal position of the optical imaging system, such as the objective lens unit 25, and the observation object WK based on information about the XY position of the observation field. Then, based on image data captured by the camera unit 10 during movement of the observation field, the display controller 52 displays a live image of the observation object on the display unit 70. On the other hand, when the movement stop detector 94 detects stoppage of movement of the observation field, the focus sequence during field movement is terminated, and a stop sequence is executed, in which the focus adjustment mechanism stops adjusting the relative distance between the focal position of the optical imaging system, such as the objective lens unit 25, and the observation object WK. In this way, focus adjustment, which was previously not possible during movement of the observation field, can be achieved. The movement stop detection unit 94 may detect both the movement state and the stop state of the observation field using a single component, or may be provided with a movement detection unit that detects the movement state of the observation field and a stop detection unit that detects the stop state of the movement of the observation field.

[0056] When the focus sequence during field of view movement is executed while the illumination control unit 66 is executing the first sequence, the illumination unit 60 is operated in the first illumination pattern, and when the movement stop detection unit 94 detects that the movement of the observation field of view has stopped, the focus sequence during field of view movement is terminated and a stop sequence is executed. In the stop sequence, for example, when autofocus is to be executed, the illumination unit 60 is temporarily switched to omnidirectional illumination in which the coaxial epi-illumination unit 62, ring illumination unit 63, etc. are all turned on, and after autofocus is completed, the illumination unit 60 is again operated in the first illumination pattern. When the movement stop detection unit 94 detects that the observation field of view is moving, the focus sequence execution unit 93 executes the focus sequence during field of view movement while the illumination control unit 66 continues to operate the illumination unit 60 in the first illumination pattern.

[0057] The image processing unit 84 realizes a function for synthesizing an image from multiple images (synthetic image mode). For example, for an image with a shallow depth of focus, it is possible to synthesize only the in-focus portions of multiple images taken while changing the focal position based on the focus information to obtain an image with a deep depth of focus (depth synthetic image). It is also possible to obtain images with enhanced resolution or an expanded dynamic range using so-called super-resolution technology. Examples of synthetic images generated by the image processing unit 84 in this way include depth synthetic images, 3D synthetic images, pixel-shifted images, super-resolution images, and HDR images. Furthermore, the image processing unit 84 may be provided with a function for measuring the object being observed. For example, a measurement tool with an array of buttons for performing several measurements can be displayed on the image, and desired measurements can be performed by operating the buttons on the displayed image.

[0058] During execution of the first sequence by the illumination control unit 66, the illumination unit 60 is operated in a first illumination pattern to display a live image, and during the second sequence, the illumination unit 60 is operated in a second illumination pattern in one of the illumination directions selected based on the feature amounts of each image data calculated by the feature amount calculation unit 88, and the image processing unit 84 realizes a function of synthesizing an image from a plurality of images (synthetic image mode).The display unit 70 also displays a composite image such as a depth composite image, a 3D composite image, a pixel shift image, a super-resolution image, or an HDR image of the object to be observed illuminated with the second illumination pattern.

[0059] The transition from the first sequence to the second sequence may be initiated by issuing an instruction to execute the composite image mode, or by specifying an ROI using the region specifying unit after the image composite mode has been set in advance.

[0060] The height information acquisition unit 89 acquires height information at different XY positions of the observation target WK as three-dimensional reference information, e.g., height image data of the observation target WK. When the field of view movement mechanism 5 moves the observation field displayed on the display unit 70 via the display control unit 52, the focus adjustment mechanism can be adjusted to adjust the focal position based on the height information of the XY position corresponding to the destination field of view movement position among the three-dimensional reference information acquired by the height information acquisition unit 89. Here, the height information indicates the position in the z direction from the mounting surface of the stage unit 30. With this configuration, by acquiring height image data, which is three-dimensional information of the observation target WK, in advance, focus adjustment can be performed based on the height information of this height image data, and when the observation field of view is moved, focused image data can be displayed on the display unit 70. For example, the height information acquisition unit 89 acquires height information corresponding to the XY position as three-dimensional reference information from wide-area image data with three-dimensional information captured by the low-magnification objective lens unit 25. The height information acquisition unit 89 can acquire height information corresponding to XY positions as three-dimensional reference information for wide-area image data with three-dimensional information that has been captured in advance, such as a navigation image described below. Note that when displaying an area that does not include height information acquired by the height information acquisition unit 89, the focus adjustment mechanism may be configured to perform focus adjustment based on height information estimated based on height information around the area. If the movement of the mounting surface of the stage unit 30 is not taken into consideration, such as when the mounting surface of the stage unit 30 is fixed, the height information does not necessarily need to be the position in the z direction from the mounting surface of the stage unit 30, and may be, for example, the position in the z direction in the device coordinate system of the magnification observation device.

[0061] The focus adjustment mechanism may also be configured to estimate a predetermined geometric shape as three-dimensional reference information from the measured height information, and determine and move the target height value based on the field of view movement position and the estimated information. The predetermined geometric shape can be a plane, sphere, cylinder, cone, parabola, etc.

[0062] When the illumination control unit 66 executes a focus sequence during field of view movement using a predetermined geometric shape via the focus adjustment mechanism while the illumination unit 60 is operating in a first illumination pattern, and the focal position is adjusted based on the height information of the observation object WK at each XY position measured in advance and the geometric shape, such as the planar shape. When the movement stop detection unit 94 detects a stop state of the movement of the observation field, the focus sequence during field of view movement is terminated and a stop sequence is executed. In the stop sequence, when measuring the height information of the observation object WK and performing autofocus, the illumination unit 60 is temporarily switched to omnidirectional illumination, with the coaxial epi-illumination unit 62, ring illumination unit 63, etc. fully lit. After the height information measurement and autofocus are completed, the illumination unit 60 is again operated in the first illumination pattern. The three-dimensional reference information is updated based on the height information measured and acquired by the height information acquisition unit 89. When the movement stop detection unit 94 detects the movement state of the observation field, the illumination control unit 66 continues operating the illumination unit 60 in the first illumination pattern, and the focus sequence execution unit 93 executes the focus sequence during field of view movement.

[0063] Furthermore, the focus adjustment mechanism may be configured to automatically perform autofocus control when movement of the observation field of view by the field of view movement mechanism 5 is stopped. Also, the focus adjustment mechanism may be configured to immediately stop autofocus control when an instruction to stop autofocus is received. (Feature calculation unit 88)

[0064] The feature calculation unit 88 calculates feature values ​​for each image data of the object illuminated in each illumination direction of the first illumination pattern. The feature value can be, for example, the variance of the image data. This makes it possible to select the illumination direction that best reveals the image features corresponding to scratches or defects as the display target for the second sequence.

[0065] The processor unit 80 can be configured with a general-purpose CPU, MPU, SoC, or a gate array such as an ASIC or FPGA customized for a specific application. While this example illustrates a configuration in which a single CPU serves as the processor unit and realizes multiple functions, the present invention is not limited to this configuration, and the processor unit may also be configured with multiple CPUs. The multiple CPUs may be multiple physical CPUs or a so-called multi-core MPU incorporating multiple CPU cores in a single package. In this case, each function may be realized by multiple CPUs or CPU cores, or different functions may be assigned to each CPU or CPU core. Furthermore, the processor unit may be configured with a combination of a CPU and a GPU. In this case, the GPU may perform the functions of the display control unit 52 described above and may also be configured to execute some or all of the functions assigned to the processor unit. (area specification part)

[0066] The region designation unit accepts designation of a desired region on the image data displayed on the display unit 70. The region designation unit can utilize the operation unit 55, such as the mouse. In the first sequence for switching the illumination direction of the illumination unit 60, the user designates a desired region of interest (ROI) on the image data using the region designation unit. The feature calculation unit 88 calculates the feature values ​​of the image data within the region designated by the region designation unit for each of the multiple pieces of image data illuminated in each illumination direction of the first illumination pattern. Meanwhile, the illumination control unit 66 selects one of the illumination directions based on the feature values ​​of each piece of image data calculated by the feature calculation unit 88 when the region is designated by the region designation unit, and executes the second sequence. In this way, by the user designating a desired region on a live image whose illumination direction is sequentially switched, the illumination direction appropriate for the designated region is automatically selected, making it possible to observe an image illuminated in a way that makes it easy to see flaws and other defects.

[0067] The designation of an ROI by the region designation unit also functions as a trigger for switching the illumination pattern of the illumination unit 60 from the first illumination pattern to the second illumination pattern. Furthermore, by redesignating the ROI, the illumination direction is determined again based on the feature values ​​of image data calculated within the newly designated ROI. If one or more cycles of image data are stored in the buffer memory 57, the illumination direction is determined again for each piece of image data stored in the buffer memory 57 based on the feature values ​​of the image data within the newly designated ROI. On the other hand, if the buffer memory 57 stores image data for less than one cycle or no image data, the system temporarily switches to the first sequence until image data corresponding to one cycle can be buffered in the buffer memory 57. Then, the feature value calculation unit 88 determines the illumination direction again for each piece of image data stored in the buffer memory 57 based on the feature values ​​of the image data within the newly designated ROI, and returns from the first sequence to the second sequence.

[0068] The ROI is specified by accepting a boundary shape, such as a rectangular shape, on the image data displayed on the display unit 70, by clicking the mouse or the like, thereby specifying the region surrounded by the boundary shape. In addition to specifying the vertices of the boundary shape by clicking the mouse or the like, the region may be enclosed by a polygonal boundary whose vertices are specified, or by a geometric shape such as a circle or ellipse. The boundary shape may also be specified by a free curve on the image data displayed on the display unit 70. The ROI may also be specified by accepting a free line, which is a line segment having a predetermined thickness, on the image data displayed on the display unit 70 by clicking the mouse or the like, thereby specifying the region corresponding to the line segment. The ROI may be specified by selectively accepting a boundary shape surrounding the region or a line segment corresponding to the region. (Navigation image registration function)

[0069] The magnification observation device according to this embodiment also includes a navigation image registration function for registering a wide-area image to facilitate searching for the observation area of ​​the observation object. The navigation image registration function acquires a wide-area image of the observation object in advance, captured using a low-magnification objective lens unit 25, and displays this wide-area image on a screen separate from the current observation field, such as a navigation window. Clicking a desired position on the navigation window moves the XY stage to that position. Details of the navigation image registration function will be explained based on the user interface screen of the magnification image observation program shown in FIG. 5. The navigation image registration screen 230 shown in this figure includes an image display area 231, an operation area 232, and a navigation area 233. The image display area 231 is an area for displaying an image. The operation area 232 is an area for displaying various operation buttons, explanations, and the like. The navigation area 233 is an area for displaying the navigation image NI, which is a wide-area image.

[0070] A position corresponding to the observation field displayed in the image display area 231 is displayed as a rectangle FR on the navigation image NI displayed in the navigation area 233. The display of the position corresponding to the observation field displayed in the image display area 231 is not limited to a rectangle FR, but may be displayed as cross lines intersecting each other on the navigation image NI displayed in the navigation area 233. In this case, the cross lines are displayed so that the position on the navigation image NI where the cross lines intersect corresponds to the center of the observation field displayed in the image display area 231. This allows the user to relatively grasp the positional relationship of the currently observed position and the location of the object being observed. In FIG. 5, chip resistors CR1 to CR3 soldered on the circuit board are displayed on the navigation image NI, and the center of the chip resistor CR1 mounted on the solder SD is located at the center of the observation field. Furthermore, when the XY stage is moved to change the observation field displayed in the image display area 231, the position of the rectangle FR on the navigation area 233 is updated accordingly. Furthermore, when a desired position is designated in the navigation area 233 with a mouse cursor or the like, the XY stage is moved to the corresponding position. It is preferable that such a navigation image NI captures a wide range of the object to be observed, but it does not need to be a full view of the object to be observed.The relationship between the stage unit 30, the object to be observed WK, the navigation image NI, and the observation field of view is shown in Figure 6. (3D navigation image registration function)

[0071] Furthermore, the magnification observation device according to this embodiment can also have a 3D navigation image registration function for registering a wide-area image having height information. The 3D navigation image registration function may be integrated with the navigation image registration function. That is, when registering a navigation image as a wide-area image, it can also be registered as a 3D navigation image having height information. (Autofocus adjustment mechanism)

[0072] This magnification observation device 100 has an autofocus function. Specifically, autofocus is performed by a focus adjustment mechanism that adjusts the focus of the image by changing the relative distance between the camera unit 10 and the objective lens unit 25 along the optical axis of the objective lens unit 25. Known methods such as a contrast method and a phase difference method can be used as appropriate to achieve autofocus. The magnification observation device according to the above-described first embodiment uses a contrast method that utilizes the brightness values ​​of pixels that make up the image data.

[0073] There are two types of autofocus: normal autofocus, which searches for a focal position within a normal height range, and short-distance autofocus, which searches for a focal position within a range narrower than the height range of normal autofocus.

[0074] Typically, autofocusing is performed while the XY stage is stopped. That is, autofocusing is not performed while the head unit 4 or the objective lens unit 25 is moving, but is performed while the XY stage is stopped. In other words, autofocusing is not performed during the observation field movement sequence, but is performed during the stop sequence. Such autofocusing can be performed by an explicit command from the user, such as pressing an autofocus execution button, or it can be performed automatically when the magnification observation device detects that the XY stage movement has stopped and the movement of the observation field is stopped. This allows a focused image to be automatically displayed on the display unit 70 when the movement of the observation field is stopped, without the user having to issue an autofocus command each time, improving usability. The movement stop detection unit 94 can be used to move or stop the observation field. (Focus tracking function)

[0075] On the other hand, the magnification observation device according to this embodiment can perform autofocus not only when the XY stage is stationary, but also when the XY stage is moving, i.e., when the observation field is moving. In this specification, the function of adjusting the focus while the observation field is moving within the XY plane is called focus tracking. Focus tracking tracks the movement of the head unit 4 in the direction in which the focus is achieved.

[0076] This focus control will be described with reference to Figures 7A and 7B. Focus tracking includes a moving sequence, which is a focus control performed while the observation field of view is moving, and a stop sequence, which is performed when the movement of the observation field of view is stopped. Figure 7A shows the moving sequence, and Figure 7B shows the stop sequence. For ease of explanation, these figures illustrate an example in which the observation field of view is moved by moving the objective lens unit 25. However, the movement of the observation field of view is not limited to this, and it goes without saying that the XY stage may also be moved. As shown in Figure 7A, during movement of the observation field of view, the objective lens unit 25 is moved vertically to follow the general shape of the observation object, thereby changing the relative distance between the focal length of the objective lens unit 25 and the observation object, thereby adjusting the focus position. Arrow 26 indicates the control direction of the objective lens unit 25 during the moving sequence. In other words, arrow 26 is a combination of the control direction of the field of view movement mechanism and the control direction that changes the relative distance between the focal length of the objective lens unit 25 and the observation object. Furthermore, as shown in FIG. 7B, when the movement of the observation field of view is stopped, autofocus is executed as a stop sequence to accurately adjust the focus position. In this case, more accurate autofocus results can be obtained than with the moving sequence. In FIG. 7B, arrow 27 indicates the direction of control that changes the relative distance between the focal length of the objective lens unit and the observation object in the stop sequence. Note that execution of autofocus in the stop sequence is not necessarily required, and it is also possible to set it not to execute autofocus when the observation field of view is stopped. (3D shape following mode)

[0077] In the 3D shape tracking mode, height information of the object being observed is acquired in advance as three-dimensional reference information, and the focus control unit 90 uses this height information to adjust the focus. The height information is acquired by simultaneously acquiring a height image when capturing a 3D navigation image as a wide-area image capturing the object being observed at low magnification. In this way, in the 3D shape tracking mode, autofocus is performed using a 3D navigation image with height information captured in advance.

[0078] The 3D shape tracking mode is shown in Figure 8. In this figure, the area 31 where 3D data 32 exists is indicated by a thick dashed line for the observation target mounted with chip resistors CR4 and CR5. In the range 31 where height information, i.e., 3D data, exists, the focal position is adjusted according to this height information. In Figure 8, arrow 28 indicates the control direction of the objective lens unit in the section where 3D data exists. On the other hand, when the observation field of view moves into a range where 3D data does not exist, control switches to control using the focus value. In this range, control similar to the general-purpose mode of normal autofocus may be used. Even in the 3D shape tracking mode, when movement of the observation field of view stops, a stop sequence is initiated. Furthermore, the color of the ROI may change depending on the presence or absence of 3D data.

[0079] When the field of view movement mechanism moves the observation field of the camera unit, the processor unit 80 controls the relative distance between the focal position of the objective lens unit and the observation object so that the focus is achieved based on the three-dimensional reference information. When the field of view movement mechanism moves the observation field of the camera unit, the focus control unit 90 controls the focus adjustment mechanism based on the three-dimensional reference information acquired by the height information acquisition unit 89, and controls the relative distance between the focal position of the objective lens unit and the observation object. In addition, the height information acquisition unit 89 updates the three-dimensional reference information based on height information of the observation object measured when autofocus is performed. (Offset function)

[0080] Furthermore, the 3D shape tracking mode can also add a function to offset the height of the 3D data. This offset function will be described with reference to FIG. 9. Here, it is assumed that the 3D data is obtained as indicated by the thick dashed line, as in FIG. 8. When the movement of the observation field of view is stopped at a position where 3D data exists, the movement stop detection unit 94 detects the stop of the movement of the observation field of view. The autofocus operation in the stop sequence is performed at this position, and the height information acquisition unit 89 acquires height information. The height information acquisition unit 89 calculates an offset amount for the 3D data so that the height of the 3D data at this position matches the calculated height information. The height information acquisition unit 89 then offsets the 3D data based on the offset amount. By constructing the height information of the 3D data in this manner, a more accurate surface height of the observation object can be acquired as updated three-dimensional reference information, enabling accurate focusing control.

[0081] When the offset function is executed while the first sequence is being executed by the illumination control unit 66, the illumination unit 60 is temporarily switched to full illumination in the stop sequence, an autofocus operation is performed, and height information of the observation object WK is acquired. After the autofocus operation is completed, the illumination unit 60 is again operated in the first illumination pattern. (Plane following mode)

[0082] Furthermore, the magnification observation device according to this embodiment is equipped with a plane tracking mode for focus control. In the plane tracking mode, as shown in FIG. 10A, a plane is estimated as three-dimensional reference information from multiple focus positions, and the Z coordinate is moved along the estimated plane while the XY stage is moving. After the XY stage stops moving, normal autofocusing is performed. In the example of FIG. 10A, SQ1 indicates multi-point height acquisition (autofocus operation), SQ3 indicates height acquisition (autofocus operation), and SQ5 indicates height acquisition (autofocus operation). SQ2 and SQ4 indicate the movement sequence, and SQ3 and SQ5 indicate the stop sequence. To execute this plane tracking mode, for example, press the "Plane Fit" button 252 in the operation area 242 on the user interface screen shown in FIG. 11. This executes the plane tracking mode. Specifically, heights of multiple points are acquired within the observation field of view indicated by the dotted line in SQ1 in FIG. 10A, and plane A is estimated as shown in FIG. 10B. Next, the objective lens unit 25 moves through the section SQ2, following the plane A estimated in SQ1. In FIG. 10A, I indicates the control direction of the objective lens unit 25 in the section SQ2. That is, it indicates the combination of the control direction of the field of view movement mechanism in the section SQ2 and the control direction for changing the relative distance between the focal length of the objective lens unit 25 and the observed object. After moving through the section SQ2, the objective lens unit 25 is stopped at SQ3. Autofocus is performed at the stopped position, and height information is acquired. In FIG. 10A, III indicates the control direction for changing the focal length of the objective lens unit 25 and the relative distance between the observed object. In addition, plane B is estimated as updated three-dimensional reference information, as shown in FIG. 10B, by combining it with the heights of the multiple points acquired in SQ1. Next, the objective lens unit 25 moves through the section SQ4, following the plane B estimated in SQ3. Finally, after moving through the section SQ4, the objective lens unit 25 is stopped at SQ5. Autofocus is performed at the stopped position, and height information is acquired. Then, by combining this with the heights of the multiple points obtained in SQ1 and SQ3, plane C is estimated as updated three-dimensional reference information, as shown in Fig. 10B. In this way, in plane tracking mode, autofocus is performed when the movement of the observation field of view stops, and the height is obtained, and the plane is re-estimated each time.Furthermore, while the observation field of view is moving, it is controlled vertically so as to follow the estimated plane. In this way, focus adjustment, which was previously not possible while the observation field of view was moving, can be achieved. Note that SQ3 and SQ5 may also be configured to acquire heights at multiple points, just like SQ1.

[0083] When the plane tracking mode is executed while the illumination control unit 66 is executing the first sequence, the illumination unit 60 is temporarily switched to full illumination in the stop sequence, autofocus is performed, and the height of the observation object WK is acquired. After autofocus is completed, the illumination unit 60 is again operated in the first illumination pattern. If the "Perform depth stacking when XY stage stops" checkbox 255 is selected in FIG. 11, the illumination unit 60 is controlled to operate in the first illumination pattern while the observation field is moving, and when the stop of movement of the observation field is detected, the illumination unit 60 is controlled to operate in the second illumination pattern according to the illumination direction selected in the first sequence. After the illumination unit 60 is controlled to operate in the second illumination pattern, depth stacking is executed. (Frame skip function)

[0084] Furthermore, the magnification observation device according to this embodiment is provided with a frame skip function that, when performing autofocus, updates the image display only if an image that is more in focus than the currently displayed image is obtained, and does not update the image display if an image that is out of focus than the currently displayed image is obtained. This allows the user, when performing autofocus on an image displayed on the display unit 70, to switch to a more in-focus image without being subjected to the stress of having an out-of-focus image displayed when they want to observe an in-focus image. (Search Writing)

[0085] The magnification observation device according to this embodiment can execute a searchlighting function that automatically changes the illumination direction. Searchlighting allows the observation field to be moved while switching the illumination light direction between different directions. This prevents the overlooking of minute scratches or defects on the object of observation. This searchlighting function includes a first sequence in which the object of observation is moved within the observation field of view while switching the illumination direction, and a second sequence in which the illumination direction is determined to make it easier to see any detected scratches or defects. In the first sequence, a live image of the object of observation is displayed on the display unit 70 while the illumination direction is switched as needed. The user searches for the position of the object of observation within the observation field of view while viewing this live image. During this search, simultaneously changing both the illumination direction and the observation position of the object of observation within the observation field of view enables the user to thoroughly search for minute scratches or defects on the object of observation. In addition, in the second sequence, the illumination direction optimal for observing the detected scratches or defects is determined. After the illumination direction is determined, the switching of the illumination direction is stopped, and a live image is displayed with continuous illumination in the determined illumination direction.

[0086] For example, the illumination unit 60 shown in FIG. 4 switches the illumination of the illumination blocks in a figure-eight pattern or counterclockwise. As a result, the illumination direction is switched in the image of the observation field, as shown in FIGS. 12A to 12D, making the surface condition of the object being observed, such as scratches, easier to see. Surface conditions such as scratches and irregularities appear differently and shade differently depending on the illumination direction. Furthermore, the illumination direction from which the surface is easily visible also varies depending on the surface condition. For example, in the examples of FIGS. 12A to 12D, scratches that are difficult to see in FIG. 12A are clearly visible in FIG. 12D. Therefore, periodically changing the illumination direction in various directions makes it easier to observe surface conditions such as scratches and irregularities.

[0087] In the first lighting pattern in which the lighting direction is switched sequentially, the lighting blocks 63a to 63d shown in FIG. 4 are not necessarily limited to being lit one by one. As an example, a pattern in which multiple lighting blocks are lit simultaneously may be included. For example, two lighting blocks may be lit simultaneously. In the example of FIG. 4, a first lighting pattern may be configured in which lighting blocks 63a and 63b are lit simultaneously, and lighting blocks 63c and 63d are lit simultaneously, repeatedly. Furthermore, the first lighting pattern may include full lighting in which all lighting blocks are lit after the lighting blocks are switched on and off.

[0088] In the case of ring illumination, it is preferable to change the illumination direction in a fixed direction along the ring, for example, counterclockwise or clockwise. This makes the appearance of the object to be observed change continuously, making scratches and other imperfections more easily visible.

[0089] Furthermore, when switching the lighting direction, it is preferable to gradually change the illuminance rather than instantly switching each lighting block on and off, i.e., changing the illuminance in a rectangular wave pattern. This type of lighting control is called "decreasing control." For example, when turning off the lights, the illuminance is gradually decreased, and when turning on the lights, the illuminance is gradually increased. This enables imaging without the occurrence of so-called flash bands, which are stripes that appear in image data when the image sensor in the camera unit 10 uses a rolling shutter CMOS instead of a global shutter. The effect of flash bands is particularly pronounced when using high-power LEDs as the light source for the lighting unit 60, so gradually suppressing brightness changes is an effective countermeasure. Alternatively, the display timing can be shifted from the transitional timing of lighting ON / OFF to a period when stable illuminance is obtained.

[0090] Furthermore, when switching the illumination ON / OFF, it is preferable to switch adjacent illumination blocks so that they overlap in time rather than exclusively ON / OFF. This can alleviate the annoying flicker caused by the ON / OFF changes of the illumination. Furthermore, when overlapping in time, it is preferable to gradually reduce the illumination intensity rather than changing it in a rectangular wave pattern as described above. For example, as shown in FIGS. 13A to 13D, when switching the illumination blocks counterclockwise, instead of switching from a state in which only illumination block 63a is lit as shown in FIG. 13A to a state in which only illumination block 63b is lit as shown in FIG. 13D, the illumination intensity of illumination block 63a is first reduced from the state shown in FIG. 13A to a state in which illumination block 63b is gradually turned on as shown in FIG. 13B. Furthermore, by increasing the illuminance of illumination block 63b as shown in FIG. 13C while further decreasing the illuminance of illumination block 63a, and switching the lighting positions of the illumination blocks as shown in FIG. 13D, a smooth change in the illumination direction is achieved, reducing the strain on the user's eyes, and making it possible to acquire image data while avoiding the occurrence of flash bands even with a camera unit 10 that uses a rolling shutter type CMOS. (Search lighting while the field of view is moving)

[0091] Furthermore, such searchlighting can be performed not only when the observation field is stationary, but also while the field is moving. This makes it easier to perform searchlighting when searching for the field of view and to find areas to be observed, such as scratches or irregularities. (Searchlighting screen 260)

[0092] An example of a search lighting screen 260 for setting search lighting is shown in the user interface screens of Figures 14 to 16. These search lighting screens 260 have an image display area 261 on the left and an operation area 262 on the right. To the left of the operation area 262 is arranged an item tab 263. The item tab 263 has multiple tabs for switching the display content of the operation area 262, and the search lighting screens 260 of Figures 14 to 16 show a state in which the "Illuminance / Brightness" tab 263a is selected.

[0093] The operation area 262 is provided with a "Back" button 264, a "Search Lighting" button 265, a detailed specification field 266, a light intensity adjustment slider 267, a "Multi-Lighting" button 268, a status display field 269, etc. The "Back" button 264 provided in the upper right of the operation area 262 is a member for returning to the top screen.

[0094] The "search lighting" button 265 is a button for switching to the lighting switching mode, which is the first lighting pattern. Pressing the "search lighting" button 265 starts sequential switching of the lighting state. Pressing the "search lighting" button 265 again returns to the normal lighting mode. When the search lighting function is ON, the "search lighting" button 265 is displayed as active, as shown in FIG. 14, etc.

[0095] The detail specification field 266 is provided with an ROI specification section 270, a color selection button 271, a lighting condition display button 272, a "re-preview" button 273, and the like.

[0096] The ROI designation section 270 is one aspect of the region designation section, and is a button for selecting a method for designating an ROI. Here, either a "Rectangle" button 270a or a "Free Line" button 270b can be selected. When the "Rectangle" button 270a is selected, the "Rectangle" button 270a is displayed as active. At this time, when two points are selected on the image display area 261, an ROI having a rectangular boundary with the two points as diagonal corners is designated. Furthermore, when the "Free Line" button 270b is selected, the "Free Line" button 270b is displayed as active, and an arbitrary region can be designated on the image display area 261 with a line segment having a predetermined thickness. Furthermore, the thickness of the free line can be selected by operating a slider 270c provided next to the "Free Line" button 270b.

[0097] The color selection button 271 is a button for selecting the color of the ROI to be specified. The illumination status display button 272 displays the illumination status. In this example, it displays which of the four illumination directions is active. In the example of FIG. 14, the lower illumination status display button 272 is active, indicating that an image of an illumination pattern in which the lower illumination block is lit is displayed in the image display area 261. In particular, when switching to the illumination status mode, which is the second illumination pattern, the user can be visually notified of which illumination direction has been selected.

[0098] Furthermore, the "Re-preview" button 273 is a button for switching the lighting conditions sequentially again after the lighting conditions have been determined by specifying the ROI. The light intensity adjustment slider 267 is a member for adjusting the intensity of the illumination light from the lighting unit 60. The "Multi-lighting" button 268 is a button for performing multi-lighting photography. Multi-lighting photography acquires images based on each lighting condition and creates a composite image. It also displays a recombined image in the lighting direction specified by the user. A status display field 269 provided below the operation area 262 displays the tilt angle of the stand, the rotation angle of the θ stage, the magnification of the objective lens unit 25, etc. (Search writing procedure)

[0099] Here, specific steps of the search lighting will be described based on the flowchart in FIG. 17 and the search lighting screen 260 in FIGS. 14 to 16. In step S1701, an instruction to switch to lighting switching mode is accepted. For example, the "search lighting" button 265 is pressed on the search lighting screen 260 in FIG. 14. In response to this, in step S1702, the mode is switched to lighting switching mode, which is the first lighting pattern. Then, in step S1703, the lighting state is switched sequentially. Specifically, the lighting positions of the lighting blocks are switched sequentially. Then, in step S1704, display processing is executed to display a live image (details will be described later with reference to FIG. 18).

[0100] Next, in step S1705, it is determined whether or not a ROI has been specified. The ROI is specified through the ROI specification unit 270 on the search / lighting screen 260 shown in FIG. 14, etc. If no ROI has been specified, the process returns to step S1703 and repeats the above process. On the other hand, if an ROI has been specified, the process proceeds to step S1706, where feature quantities of image data corresponding to the specified ROI for the image for each lighting condition are calculated. Here, the feature calculation unit 88 calculates feature quantities of image data within the region specified by the ROI in the image data illuminated in the lighting direction of each lighting block. For example, if an ROI is specified by selecting the "Rectangle" button 270a on the search / lighting screen 260 shown in FIG. 14, a rectangular ROI is displayed in the image display area 261 in the color selected by the color selection button 271, as shown in FIG. 15. The variance within this ROI region is calculated for the image data for each lighting direction, and the lighting direction is determined.

[0101] Further, in step S1707, the lighting conditions are determined based on the feature amounts of the image data, and the process transitions to the determined lighting condition mode, i.e., the second lighting pattern. That is, the lighting direction corresponding to the image data with the greatest number of feature amounts is determined, and the process switches from the first sequence to the second sequence. Then, in step S1708, the lighting conditions are controlled to the determined lighting conditions. That is, the second sequence is executed, lighting is performed in the determined lighting direction, and the image is displayed in step S1709. In the example of the searchlighting screen 260 in FIG. 16, the lighting condition display button 272 on the right is activated based on the feature amounts of the image data, and an image of the lighting pattern in which the lighting blocks on the right are lit is displayed in the image display area 261. That is, the lighting direction from the right is selected, and as a result, an image in FIG. 16 in which the flaw is more visible than in FIGS. 14 and 15 is displayed in the image display area 261.

[0102] Furthermore, in step S1710, it is determined whether or not re-designation of an ROI has been accepted. If it is detected that the user has re-designated an ROI, the process returns to step S1706, where feature quantities of image data are calculated for the re-designated ROI, and the above processing is repeated. Re-designation of an ROI can be performed, for example, by re-designating a new ROI in the image display area 261 on the searchlighting screen 260 of FIG. 16. On the other hand, if no re-designation of an ROI has been made, the process proceeds to step S1711, where the observation field is moved based on an instruction to move the observation field, and a live image is displayed. In this way, when an ROI is designated while changing the illumination direction with searchlighting, the illumination direction is automatically determined, and an appropriate image can be displayed. (Display processing)

[0103] An example of the display processing of a live image during searchlighting (step S1704 in FIG. 17) will now be described with reference to the flowchart in FIG. 18. First, in step S1801, a live image is displayed. Next, in step S1802, images are buffered for each lighting condition. Here, image data is temporarily stored in buffer memory 57. Then, in step S1803, it is determined whether or not an instruction to move the observation field of view has been issued. If an instruction to move the observation field of view has not been detected, the display processing ends. On the other hand, if an instruction to move the observation field of view, for example, movement of the stage unit, has been detected, the processing proceeds to step S1804, where the buffered image is cleared. Then, in step S1805, the observation field of view is moved based on the movement instruction, and in step S1806, image data is buffered for each lighting condition. In this manner, the display processing of a live image is executed. (Focus tracking mode display processing)

[0104] Furthermore, when a live image is displayed in the display process, the focus can be automatically adjusted. As an example, a procedure for performing the display process with the focus tracking mode set to ON will be described with reference to the flowchart in FIG. 19. First, in step S1901, a live image is displayed. Next, in step S1902, it is determined whether an instruction to move the observation field of view has been received. If an instruction to move the observation field of view has not been detected, the process jumps to step S1911. On the other hand, if an instruction to move the observation field of view has been detected, the process proceeds to step S1903, where the observation field of view is moved based on the movement instruction. Then, in step S1904, focus tracking is performed. Here, focus control is performed according to the type of focus tracking mode (general mode, 3D shape tracking mode, plane tracking mode, etc.). Furthermore, in step S1905, it is determined whether the movement of the observation field of view has stopped. If the stop of the field of view movement has not been detected, the process returns to step S1904, where focus tracking continues. On the other hand, if it is detected that the movement of the observation field has stopped, the process proceeds to step S1906, where the sequential switching of the illumination conditions is temporarily stopped, and in step S1907, short-distance autofocus is executed. Here, the height (Z coordinate) of the object to be observed is measured. Furthermore, in step S1908, the 3D data is corrected based on the measured coordinates. Next, in step S1909, the sequential switching of the illumination conditions is resumed. Then, in step S1910, it is determined whether or not an instruction to move the observation field has been received. If an instruction to move the field has been detected, the process returns to step S1903, where the observation field is moved, and the above processing is repeated. On the other hand, if no instruction to move has been received, the process proceeds to step S1911, where image data is buffered for each illumination condition. In this manner, focus tracking mode display processing is executed. (decreasing control)

[0105] Here, specific procedures for achieving fade-out control, in which the switching of illumination blocks is overlapped in time and the light intensity is gradually changed, will be described with reference to the timing chart in Figure 20. This explains the process of switching from coaxial illumination, in which all illumination blocks are turned on, to fade-out control searchlighting. As shown in this figure, in the first sequence, the illumination is turned on sequentially, live images are displayed, and image data is buffered. In the second sequence, the illumination direction is determined based on the buffered images, allowing observation. After the second sequence, the ROI is re-specified, and the illumination direction is determined based on the buffered image data. If a command to switch the illumination direction is issued again, the process returns to the first sequence.

[0106] In addition, to prevent strain on the user's eyes, the lighting pattern of the lighting block is gradually switched by using a taper control that gradually changes the lighting intensity and an overlap control that temporally overlaps two lighting directions.

[0107] The lighting direction is switched using a first lighting pattern in which the lighting blocks are lit clockwise until all the lighting blocks are finally lit. That is, one cycle is composed of lighting block 63d → lighting block 63c → lighting block 63b → lighting block 63a → lighting blocks 63a to 63d. The period in which each lighting pattern goes around once in this way is defined as one cycle. Note that it is not necessary to provide a period in which all the lighting is on. Alternatively, a period in which all the lighting is off may be provided.

[0108] Specifically, when search lighting is started by pressing the "search lighting" button 265 on the user interface of Figure 14, the coaxial lighting is switched to sequential lighting, and the lighting direction is switched periodically, repeating the cycle until it is instructed to stop.

[0109] In the step-down control, the illumination intensity of each lighting block is gradually changed as the illumination light rises and falls. Furthermore, the switching points of the illumination light are configured to overlap each other. Here, the rate of decrease in the light intensity of the lighting blocks that are turned off corresponds to the rate of increase in the light intensity of the lighting blocks that are turned on, thereby suppressing changes in the total light intensity.

[0110] If the gradual decrease control is not performed, the change in light intensity may be a rectangular wave. Also, the lighting control may be performed at a timing that does not provide a section where the lighting of the lighting blocks overlaps.

[0111] While the illumination blocks are sequentially illuminated to search for flaws, a live image is displayed on the display unit 70. Meanwhile, image data is buffered in the buffer memory 57 during steady-state periods in which the illumination light maintains a constant brightness within the illumination period of each illumination block. In other words, buffering is not performed during periods in which the brightness is decreased or increased. This is because, during these periods, the illumination light from two illumination blocks overlaps, making it inappropriate to uniquely determine the appropriate illumination direction. Since the image data stored in the buffer memory 57 is used to determine the appropriate illumination direction, it is essential that the image data be perfectly associated with the illumination direction.

[0112] Once the ROI is specified, the process moves to the second sequence. Here, the lighting direction is determined based on image data that has already been buffered, so sequential lighting is stopped. Buffering of image data is also not performed. After the lighting direction is determined, the display unit 70 may display either a live image or a still image. Buffering of images is not performed in the second sequence.

[0113] Furthermore, even when the ROI is re-specified, sequential lighting is not performed because the lighting pattern is determined based on the image that has already been buffered. On the other hand, when a re-preview is specified, the same processing as the first sequence is performed again. A re-preview is performed by pressing the "re-preview" button 273 on the searchlighting screen 260 in FIG. 14, etc.

[0114] In the example of Figure 20, the change in light intensity of each illumination block is not a square wave but a trapezoidal wave. By using a trapezoidal wave, a steady-state section where the illumination intensity is constant is created, making it easier to buffer image data during this section. However, if decay control is not required, the change in light intensity may be a square wave. For example, if a low-intensity light-emitting diode is used as the illumination light source for the illumination unit 60 or if a global shutter CMOS is used for the image sensor, a flash band will not occur even if the change is a square wave. Furthermore, if the shutter speed is high, the change in light intensity may be a triangular wave.

[0115] 20, an overlapping section is provided when switching between illumination blocks, and there is a section where the two illumination directions overlap. However, the overlapping section is not necessarily required in the present invention, and an illumination pattern that does not create an overlapping section can be appropriately adopted depending on the purpose or use of the observation. (Searchlighting while moving the field of view)

[0116] In the above example, searchlighting is performed with the observation field fixed. In the magnification observation device according to this embodiment, searchlighting can also be performed while the observation field is moving. The buffering of image data during this process, and the relationship between ROI designation and buffered image data, will be explained based on the timing chart in FIG. 21. Here, one cycle is the same as the one defined in FIG. 20.

[0117] When the movement of the observation field is detected to have stopped, autofocus is executed and the lights are turned on sequentially once the focus is achieved. Here, since it is not possible to achieve focus when the lighting direction is switched, autofocus is executed once all the lighting blocks are turned on after the movement of the observation field has stopped. Once the focus is achieved by autofocus, image data buffering begins.

[0118] Furthermore, when an ROI is specified, the illumination pattern is determined based on the buffered image data. Furthermore, if the ROI is re-specified, the illumination pattern can be determined without sequentially turning on the lights again because the buffered image data is retained. On the other hand, if the observation field is moved after the illumination pattern has been determined, autofocus is performed after the observation field stops, but the illumination direction is not switched sequentially. The interrelationships between the operations of each part are described in detail below.

[0119] The movement and stop of the observation field of view is determined by the movement stop detection unit 94. Note that here, an example is shown in which the observation field of view is moved by moving the XY stage. In Fig. 21, if the period in which a stage movement command is not detected is shorter than the stage stop determination period, it is determined that the observation field of view (XY stage) is moving, and autofocus and other operations when stopped are not performed.

[0120] When movement of the observation field of view begins, the image data buffered in the buffer memory 57 at that time is discarded. This is because image data is successively updated as the observation field of view moves, and image data captured in the past cannot be used. To discard the image data, for example, all buffered image data is discarded immediately after the movement begins, and no further image data is buffered. Alternatively, image data may be buffered and discarded repeatedly.

[0121] On the other hand, if it is determined that the movement of the observation field has stopped, autofocus is first performed to bring the image into focus. During this time, autofocus is not possible if the illumination pattern is switched, so switching of the illumination direction is stopped. Here, it is preferable to use omnidirectional illumination, which lights up all illumination blocks. This increases the brightness of the image, making it easier to perform autofocus with high precision. Once the image is in focus through autofocus, sequential lighting and image buffering begin. In this way, image data buffering is performed while the observation field is stopped.

[0122] On the other hand, when an ROI is specified, this triggers a switch from the first sequence to the second sequence, i.e., the illumination direction is fixed, and thereafter the buffer memory 57 is not updated, and the image data previously held is maintained.

[0123] Furthermore, after the ROI is re-designated, the illumination pattern is determined based on the buffered image data, so sequential lighting is not performed.

[0124] Furthermore, if the observation field of view is moved during the second sequence, autofocus is performed after stopping, but the illumination pattern remains fixed. Image data is not buffered. Therefore, if the observation field of view is moved during the second sequence, there will be no buffered image data. If the ROI is re-specified during this time, the illumination pattern will be switched and image data will be acquired for each illumination direction before the illumination pattern is determined. Alternatively, the user may be prompted to specify a re-preview without being asked to re-specify the ROI.

[0125] The steady-state section refers to a section in which the illumination light intensity remains constant. Image data acquired during this section is buffered. The duration of this steady-state section is not limited, and it may not even be necessary to have a steady-state section. In this case, the waveform of the illumination light intensity will be a triangular wave or a square wave (if no overlapping section is provided). The overlapping section is a section that is provided to avoid straining the user's eyes. The length of this section is also not limited, and as in Figure 20, it does not need to exist. Furthermore, the stop determination section is the section from when the movement of the observation field stops until it is determined that it has stopped. Further possible triggers include specifying an ROI, moving the observation field, or an instruction to determine the illumination direction independent of the observation object. When an instruction to determine the illumination direction is issued, the feature values ​​for each image data are calculated and the illumination direction is determined. (Exclusive control of the first lighting pattern)

[0126] Furthermore, in the first sequence, in which the illumination direction is periodically changed using the first illumination pattern, autofocusing becomes difficult. As described above, autofocusing is performed based on image brightness information. Therefore, if the amount of light changes during autofocusing, for example by changing the illumination direction, accurate brightness information cannot be obtained. Therefore, a configuration may be adopted in which the change in illumination direction is stopped during autofocusing. For example, if autofocusing is performed automatically when the movement of the observation field of view is stopped, the illumination direction is switched while the observation field of view is moving, but when the movement of the observation field of view is stopped, the switching of the illumination direction is stopped, the second sequence is switched to, and autofocusing is performed. The movement or stop of the observation field of view can be detected by the movement stop detection unit 94. Furthermore, switching from the first sequence to the second sequence can be performed by the illumination control unit 66.

[0127] Furthermore, the exclusive control of stopping the switching of the lighting direction can be applied not only during the execution of stopped autofocus, but also during other processes. For example, searchlighting may be turned off during execution of a focus tracking mode in which autofocus is performed while the observation field of view is moving. Furthermore, the switching of the lighting direction of the first lighting pattern is stopped during image processing by the image processing unit 84, such as image stitching that stitches together multiple images with different observation fields of view, depth stacking that combines multiple images at different heights, and HDR synthesis. This is because it is desirable to maintain a constant amount of lighting light in these image synthesis processes. (Flaw detection algorithm)

[0128] The above describes a procedure for automatically determining the illumination direction according to an ROI specified by a user that includes a flaw or other defect. However, the present invention is not limited to this configuration, and the illumination direction can be determined using other methods. For example, the spatial frequency components for which the feature quantities of the image data are calculated are specified. This makes it possible to distinguish between high-spatial-frequency hairlines on the surface of the object being observed and low-spatial-frequency dents. If a hairline is extracted using the specified spatial frequency components, the illumination direction that best reveals the extracted hairline is determined. Similarly, if a dent is extracted using the specified spatial frequency components, the illumination direction that best reveals the extracted dent is determined. In addition to specifying the spatial frequency components, feature size, flaw size, etc. may also be specified. Features in the image data corresponding to the specified feature size or flaw size are extracted, and the illumination direction that best reveals the extracted feature is determined. (Flaw detection algorithm)

[0129] As described above, the feature quantities of the image data corresponding to the illumination light are used to determine which of multiple illumination directions is most suitable for observation. The procedure for determining the illumination direction based on the feature quantities of the image data is called the flaw detection algorithm here. One example of the feature quantity is the variance of the brightness of the image data. This is because it can be assumed that an image with a large variance of brightness is one in which flaws on the object of observation are most noticeable.

[0130] Another feature is the sum of brightness differences. For example, an image with a large sum of brightness differences, which is the sum of the differences between the brightness values ​​of each pixel constituting the image data and the average value, can be assumed to be an image in which scratches on the object of observation are most noticeable.

[0131] Another feature is the spatial frequency component of an image. For example, if there is a mixture of small and large scratches within the ROI specified in the region specification section, they will affect each other. The following can be considered as a way to avoid this. First, set an ROI for the original image data and calculate the total brightness difference within this region. This makes it possible to detect small scratches (corresponding to detecting high-frequency components). Meanwhile, reduce the original image data by one level and calculate the total brightness difference in the same way. This makes it possible to detect scratches of medium size. Meanwhile, reduce the original image data by two levels and calculate the total brightness difference in the same way. This makes it possible to detect large scratches (corresponding to detecting low-frequency components). This procedure is equivalent to detecting scratches by changing the strength of a smoothing filter. In this way, image data containing scratches can be identified.

[0132] Furthermore, a configuration may be adopted in which a detection target included in the observation target is designated, and an illumination direction suitable for the designated detection target is determined. Examples of detection targets include irregularities such as scratches and dust, and the user designates these types. For example, based on image data of the observation target WK illuminated from each illumination direction in the first sequence, irregularities such as scratches and dust may be automatically recognized from the image data by calculating irregularity information from the difference between each image data.

[0133] Furthermore, when an area such as an ROI is specified, irregularities such as scratches and dust may be automatically recognized from the image data within the specified area. When an area such as an ROI is not specified, the illumination direction may be automatically determined from the entire image data, or a predetermined area near the center of the image data may be set in advance as the ROI. (Multi-lighting function)

[0134] The multi-lighting function can be executed by pressing a "multi-lighting" button 268 provided in the lower section of the operation area 262 on the search lighting screen 260 of, for example, FIG. 14. An example of the multi-lighting screen is shown in FIG. 22. The multi-lighting screen 430 shown in this figure comprises a function display area 431 provided in the upper section of the screen, a main display area 432 provided on the left side of the screen, and a secondary display area 433 provided on the right side. The secondary display area 433 has an emission direction specification field 433a in the upper section and an emission direction display field 433b in the lower section.

[0135] On the multi-lighting screen 430, still images illuminated in each lighting direction are acquired and the user is allowed to select from these images the image with the optimal lighting direction. This allows the user to visually and intuitively select an image that achieves the desired visibility without having to worry about adjusting parameters such as which lighting direction is appropriate.

[0136] A function display area 431 is set at the top of the multi-lighting screen 430. In the function display area 431, a focus stacking button b1, a DR adjustment button b2, and a save button b7 are displayed.

[0137] A main display area 432 and a sub-display area 433 are set to be aligned horizontally below the function display area 431. The main display area 432 has a larger area than the function display area 431 and the sub-display area 433. In the initial state, one of a plurality of images SI based on a plurality of pieces of original image data generated by the immediately preceding multiple illumination imaging is displayed over almost the entire main display area 432. In this example, an image SI of the object to be observed when illuminated with the first directional illumination is displayed in the main display area 432.

[0138] The secondary display area 433 displays an emission direction designation field 433a and an emission direction display field 433b. The emission direction designation field 433a displays an object position image ss0 indicating the position of the object to be observed on the placement surface. In addition, a light icon ss1 indicating the emission position of light with respect to the object to be observed when the object to be observed is viewed from a position above the illumination unit 60 is superimposed on the object position image ss0. In this case, the relative positional relationship between the target partial image sp of the object to be observed on the object position image ss0 and the light icon ss1 corresponds to the emission direction of light to be irradiated onto the object to be observed in order to obtain the image SI displayed in the primary display area 432 (hereinafter referred to as the virtual emission direction of light).

[0139] 1 etc., to move the light icon ss1 in Fig. 22 relatively to the target partial image sp of the observation object on the object position image ss0, thereby easily specifying the virtual emission direction of the light while grasping the virtual emission position of the light. When the user specifies the virtual emission direction of the light, the image SI of the observation object displayed in the main display area 432 is updated to the image SI of the observation object that should be obtained when light in the specified emission direction is irradiated onto the observation object.

[0140] In the emission direction display field 433b, an image indicating a reference point on the placement surface is displayed as a reference point image ss2, and an image of a virtual hemisphere covering the reference point on the stage unit 30 is displayed three-dimensionally as a hemispherical image ss3. On the hemispherical image ss3, an image indicating the light emission position corresponding to the virtual emission direction of light specified by the light icon ss1 is displayed as an emission position image ss4. Furthermore, a straight line is displayed connecting the emission position image ss4 on the hemispherical image ss3 to the reference point image ss2. In this case, the direction on the straight line from the emission position image ss4 to the reference point image ss2 indicates the emission direction of light specified by the light icon ss1. By visually checking the reference point image ss2, hemispherical image ss3, and emission position image ss4 displayed in the emission direction display field 433b, the user can easily and accurately recognize the virtual emission direction of light specified by the light icon ss1. In the magnification observation device 1, a predetermined planar coordinate system is defined for the object position image ss0 displayed in the sub-display area 433 in FIG.

[0141] As described above, the image data of the image SI displayed in the main display area 432 is generated after the completion of multiple illumination imaging based on the generated multiple original image data and the virtual light emission direction specified by the user. Therefore, even if the virtual light emission direction specified by the user continuously changes, multiple image data corresponding to the specified emission direction are generated substantially continuously at a speed according to the processing capacity of the control unit. Furthermore, the image SI based on the generated multiple image data is continuously displayed. Therefore, an image (video) substantially identical to an image obtained by continuously capturing images while changing the position of the illumination is reproduced in the main display area 432. As a result, by viewing the image SI on the main display area 432 while specifying the virtual light emission direction, the user feels as if the light from the specified emission direction is being irradiated on the observed object in real time. In the multi-lighting screen 430 of FIG. 22, a light icon ss1 is superimposed on the object position image ss0 displayed in the sub-display area 433. In this embodiment, in addition to the light icon ss1 on the object position image ss0, the light icon ss1 may also be superimposed on the image SI displayed in the main display area 432.

[0142] Alternatively, an illumination light source display screen 280 as shown in FIG. 23 may be separately prepared. In this example, the position of the illumination light source arranged above the observation object is displayed three-dimensionally as a light icon ss1. The user can move the position of the light icon ss1 to any position by clicking or dragging the mouse, and the actual illumination direction is adjusted according to the moved position. Such control is performed by the illumination control unit 66, which selects an illumination block and adjusts the light intensity so that the illumination direction corresponds to the position specified on the illumination light source display screen 280. In addition to dragging with the mouse, the illumination direction may also be specified using an illumination status display button 272 provided on the user interface screen as shown in FIGS. 14 to 16. [Embodiment 2]

[0143] Furthermore, the magnification observation device according to this embodiment can also be provided with a route guide function in addition to the searchlighting function. In particular, by using the route guide function in combination with the searchlighting function, the operation when moving the field of view can be simplified, further reducing the burden on the user. Such a magnification observation device will be described as embodiment 2 based on the block diagram of FIG. 24. In the magnification observation device 200 shown in this figure, the same components as those in the magnification observation device according to the above-mentioned embodiment 1 are assigned the same reference numerals, and detailed description thereof will be omitted.

[0144] The processor unit 80 implements functions such as a trajectory instruction unit 81, a trajectory calculation unit 82, a movement control unit 83, an image processing unit 84, and a feature calculation unit 88. The trajectory instruction unit 81 instructs trajectory information regarding the direction of movement of the observation field for the image data displayed on the display unit 70. The trajectory calculation unit 82 calculates a field of view movement trajectory for moving the observation field based on the trajectory information instructed by the trajectory instruction unit 81. The movement control unit 83 controls the movement of the field of view movement mechanism 5 along the field of view movement trajectory calculated by the trajectory calculation unit 82 in accordance with the movement direction instruction from the movement direction instruction unit 55a. The image processing unit 84 calculates the height of the observation object WK corresponding to the set region in the optical axis direction based on focal length information stored in the storage unit 53 for part or all of the observation object WK corresponding to the set region. This magnification observation device 100 can calculate the average height (depth) of the observation object WK corresponding to the set region in the optical axis direction using the image sensor 12. (Route guide function)

[0145] This magnification observation device 100 is equipped with a route guide function that makes it easy to move the observation field along a preset route. For example, consider an application in which the observation object is a cylindrical workpiece WK2 as shown in Figure 25, and burrs and scratches on the circumference of its end face are observed using the magnification observation device. In this case, it is necessary to move the stage unit 30 so that the observation field displayed on the display unit 70 moves along the arc of the edge of the observation object.

[0146] Because the magnification is high during observation with a digital microscope, moving the observation field to the intended position can be somewhat difficult. When moving the stage manually, it is common to move the X and Y axes separately. For example, separate knobs are provided for moving the stage in the X and Y directions, and the user operates each knob to move in the desired direction. This method requires separate operation for movement in the X and Y directions, making it difficult to move to the desired position. For example, it was necessary to alternately move the X and Y axes to follow the arc of the cylindrical workpiece WK2 shown in Figure 25 during observation.

[0147] However, advances in digital microscopes have led to an increasing use of motorized XY stages in recent years, which enable free movement on the XY plane by operating an on-screen mouse or a joystick as shown in Figure 26.

[0148] However, in the case of mouse operations such as dragging and double-clicking, the operation must be performed continuously when moving a long distance.

[0149] Furthermore, even when moving continuously in a specified direction using a joystick or the like, the movement does not strictly follow the object being observed, but instead winds along the desired route, as shown in Figure 25. While the observation field of view is winding along, the user must continue to concentrate on the operation so as not to deviate from the desired route, which can be very stressful.

[0150] On the other hand, you can also use the image stitching function to take pictures in advance and check them later. However, in this case, it takes a lot of time from setting the shooting range to completing the shooting, and for users who just want to check whether there are burrs, the effort of taking pictures and saving them each time can be a big burden.

[0151] Therefore, the magnification observation device 100 according to this embodiment includes a route tracing mode in which a route for moving the observation field is preset, and when the movement direction indicator 55a is used to actually specify the observation field, the field of view movement mechanism 5 is controlled so that the observation field changes along the preset route. By executing the route tracing mode, the user can move the observation field along a desired route, even if the path is complex, simply by specifying a general direction using the movement direction indicator 55a. This allows the user to concentrate on the observation without being bothered by moving the stage unit 30. In the example shown in FIG. 26 above, an arc-shaped route RT is set in advance to follow the end face of the cylindrical object, as shown in FIG. 27. Then, the observation field can be moved along the arc simply by tilting the joystick 55b, which serves as the movement direction indicator 55a, roughly to the right. In FIG. 27, the arrow DJ1 indicates the direction in which the joystick 55b is tilted. Normally, the observation field of view cannot be moved in an arc unless the tilt direction of the joystick 55b is gradually changed from right to upward along the tangent direction DS1 of the arc-shaped route. However, by executing the route tracing mode, it is possible to move the stage unit 30 in an arc along the route RT, i.e., in the direction of DS1, even while tilting the joystick 55b in a fixed direction DJ1, as shown in FIG. 27. In other words, it is possible to move the optical axis AX of the objective lens unit 25 on the mounting surface of the stage unit 30, bending along the circumference as shown in FIG. 27, without gradually changing the tilt direction DJ1 of the joystick 55b according to the position of the arc. In this way, the route tracing mode allows movement control in which the tilt direction DJ1 of the joystick 55b and the actual movement direction DS1 of the stage unit 30 are deviated from each other.

[0152] Such operations are not limited to the joystick 55b. For example, it is possible to prepare virtual or physical movement buttons 55c as shown in Fig. 28 and operate the buttons to instruct forward or backward, or to move right or down, up or down, or left or right. Furthermore, when a mouse with a wheel button is used as the movement direction instructing unit 55a, the movement direction of the observation field can be instructed in the same way as with a joystick by moving a mouse cursor MC, which indicates the movement direction of the field of view, around an icon IC that is displayed by clicking the mouse wheel, as shown in Fig. 29.

[0153] As described above, the field of view movement trajectory calculated by the trajectory calculation unit 82 can include a curve. This has the advantage of allowing curved movement of the observation field, which has been troublesome in the past, to be easily performed. Furthermore, the field of view movement trajectory calculated by the trajectory calculation unit 82 can also include a bent portion. This has the advantage of allowing non-linear, bent movement of the observation field, which has been troublesome in the past, to be easily performed. (Movement control unit 83)

[0154] The movement control unit 83 can switch between a route trace mode and a free mode as a control method for the field of view movement mechanism 5. In the route trace mode, the movement direction instruction unit 55a controls the field of view movement mechanism 5 so that it follows the field of view movement trajectory calculated by the trajectory calculation unit 82. In the free mode, the field of view movement mechanism 5 is controlled in the movement direction instructed by the movement direction instruction unit 55a, regardless of the field of view movement trajectory. This makes it possible to easily move the observation field of view along the field of view movement trajectory in the route trace mode, while also allowing the user to freely move the observation field of view in the free mode.

[0155] The movement control unit 83 realizes smooth movement of the field of view movement function of the stage unit 30, etc., by anticipating changes in the field of view movement trajectory in advance so that the observation field of view moves smoothly along the field of view movement trajectory in accordance with the movement direction indicated by the movement direction indication unit 55a, such as the joystick 55b. Specifically, as shown in FIG. 30A, the movement control unit 83 calculates a predicted arrival point from the movement direction DS1 and movement speed of the stage unit 30, etc., as well as the movement direction DJ1 indicated by the joystick 55b, etc., i.e., the sampling interval for monitoring the movement command. Then, as shown in FIG. 30B, the movement control unit 83 searches for a point on the field of view movement trajectory closest to the calculated predicted point, corrects the movement direction DS1 and movement speed of the stage unit 30, etc., and moves the stage unit 30, etc., toward the closest point. This achieves smooth movement even if the field of view movement trajectory is curved.

[0156] Here, a method for observing an enlarged image that realizes the route guide function will be described with reference to the flowchart in Fig. 31. First, in step S3101, the user is prompted to specify trajectory information related to the direction of movement of the observation field. For example, the trajectory specification unit 81 receives a reference point specification by the user as trajectory information.

[0157] Next, in step S3102, the trajectory calculation unit 82 calculates the field of view movement trajectory in accordance with the trajectory information. For example, an interpolated route is set based on a reference point specified by the user using the trajectory specification unit 81. Then, in step S3103, a user instruction for the movement direction of the stage unit 30 is accepted. Furthermore, in step S3104, the movement direction of the stage unit 30 is determined based on the specified movement direction and the interpolated route. Finally, in step S3105, the stage unit 30 is controlled based on the determined movement direction. (Trajectory instruction section 81, trajectory calculation section 82)

[0158] The trajectory instruction unit 81 specifies trajectory information relating to the direction of movement of the observation field. Based on the trajectory information specified by the trajectory instruction unit 81, the trajectory calculation unit 82 sets a trajectory, i.e., a route, along which the observation field will move. Here, the trajectory information may include, for example, multiple reference points. The trajectory calculation unit 82 calculates a route that passes through the multiple reference points specified by the trajectory instruction unit 81. For example, if the route is circular, three points on the circumference are specified as reference points. Furthermore, if the route is a combination of straight line segments, the start and end points of the line segments or bending points are specified as reference points. Note that the trajectory instruction unit 81 and the trajectory calculation unit 82 may be separate components or may be integrated.

[0159] The trajectory indication unit 81 and the trajectory calculation unit 82 can also approximate a route using a geometric shape prepared in advance. That is, after a geometric shape is selected in advance from the trajectory indication unit 81, a route can be calculated by the trajectory calculation unit 82 by specifying a point passing through this geometric shape as a reference point. Examples of geometric shapes include a circle, an ellipse, a rectangle, a polygon, a star, a straight line, a line segment, and a curve such as an arc. There are no particular restrictions on the geometric shape as long as it can be expressed on a two-dimensional plane. A circle, an ellipse, a Bezier curve, etc. can be specified by specifying three or more points. A polygon such as a rectangle can be specified by specifying vertices.

[0160] A geometric shape can be selected in the geometric shape selection section, and a reference point can be specified in the reference point specification section. For example, if a straight line or line segment is selected as the geometric shape GS1, by specifying two points as reference points on the display section 70 as shown in Fig. 32A, the straight line or line segment passing through these two points can be calculated as a route. For the sake of convenience, this example shows a state in which the head section 4 is moved relative to the stage section 30.

[0161] Furthermore, as shown in FIG. 32B, if a circle is selected as the geometric shape GS2, three reference points can be specified to calculate the circle passing through these points as the route. Similarly, as shown in FIG. 32C, if a rectangle is selected as the geometric shape GS3, four corner vertices can be specified as reference points to calculate the circle passing through these points as the route. Furthermore, the geometric shape is not limited to simple shapes such as a circle or a rectangle; complex shapes can also be specified. For example, any shape can be used, such as the union or intersection of multiple shapes, a series of line segments including bent portions, or a shape tracing a path indicated by a pointing device. For example, FIG. 32D shows an example of a partially broken circle as the geometric shape GS4, FIG. 32E shows an example of a star as the geometric shape GS5, and FIG. 32F shows an example of a shape composed of multiple continuous line segments as the geometric shape GS6.

[0162] Furthermore, the geometric shape does not need to be a closed figure such as a circle or a rectangle, and can also be a straight line, a curve, etc. For example, consider an example of observing each chip CP of an observation object in which multiple chip resistors CR are discretely mounted on a substrate CB, as shown in Figure 33. In this case, by setting a reference point at the position of each chip CP according to the arrangement of the chip CP, a polygonal route RT as shown in Figure 34 is set as the field of view movement trajectory. In Figure 33, each reference point is indicated by an "x", and the observation field of view at each reference point is indicated by a wavy frame. (Root completion)

[0163] The trajectory calculation unit 82 calculates the visual field movement trajectory based on the trajectory information input from the trajectory specification unit 81. The trajectory calculation unit 82 interpolates the coordinates between multiple reference points and completes the visual field movement trajectory as an interpolated route. Furthermore, the trajectory calculation unit 82 changes the interpolation algorithm based on the selected geometric shape. Note that the trajectory information is not limited to multiple reference points, and other specification methods can be used as appropriate. For example, trajectory information specifying a circular trajectory can be specified by the center coordinates and radius of the circle. When specifying a route RT on a broken line, if the reference points are specified in the order I to IV in Figure 33, the coordinates of the reference points are interpolated in the specified order. In other words, the route RT shown in Figure 34 is interpolated. In this figure, when the user instructs a movement direction, for example, from position II to position III, if the user's instruction at position II is included above the boundary area indicated by the dashed line, the movement will be in the direction of III. Furthermore, at the position indicated by a circle midway from II to III, if the user's instruction is included on the left side of the boundary area indicated by the dashed line, the object is moved toward position III.

[0164] Furthermore, the trajectory information for determining the direction of movement of the observation field of view is not limited to the method of specifying a reference point on the screen of the display unit 70 described above, but can also be obtained by, for example, directly inputting the coordinate position as a number or specifying the field of view movement trajectory using a mathematical formula.

[0165] In addition, when using an application in which an observation object is placed in a fixed position on the stage unit 30, for example, when the same observation object is always placed in the same position on the stage unit 30 using a positioning jig, once the field of view movement trajectory is set in accordance with the design values ​​of the jig, route guidance can be performed under the same conditions thereafter. In other words, there is no need to move the stage unit 30 each time, register the reference position, and set the field of view movement trajectory.

[0166] Furthermore, the view movement trajectory can be specified not only by a geometric shape but also automatically extracted from an image. For example, a wide-area image of the observation object WK3 shown in FIG. 35A is captured, and a contour PL is extracted by edge extraction as shown in FIG. 35B. If the movement is set to follow all or part of this contour PL, the trajectory specifying unit 81 or the trajectory calculating unit 82 can automatically obtain trajectory information and acquire the view movement trajectory, thereby reducing the user's effort of manually specifying trajectory information each time. Edge extraction can be performed, for example, by connecting extraction points to obtain a contour, or by approximating the line connecting the extraction points with a straight line or curve.

[0167] In addition to displaying the entire image of the observation object on one screen as shown in Figure 35A and extracting the shape of the observation object to set the field of view movement trajectory, it is also possible to set the field of view movement trajectory by sequentially extracting the contours while moving the observation field while partially displaying the observation object. For example, as shown in Figures 36A and 36B, it is also possible to update the observation field of view while moving the XY stage, obtain the contours from the image of the observation object WK4 in real time, and automatically detect the field of view movement trajectory. Such automatic acquisition of the field of view movement trajectory can be performed by the trajectory instruction unit 81 or the trajectory calculation unit 82.

[0168] After the reference point is set as described above, when the route trace mode is executed, the user can move the observation field of view along the field of view movement trajectory even with a rough instruction, without having to specify the movement direction of the field of view movement mechanism 5 in detail using the movement direction instruction unit 55a such as the joystick 55b. That is, in the route trace mode, the movement direction specified by the movement direction instruction unit 55a is compared with the direction of the field of view movement trajectory, and if it is within a predetermined range of the field of view movement trajectory, it is determined that a movement instruction along the field of view movement direction has been issued, and the field of view movement mechanism 5 is moved. As a result, even while the observation field of view is being moved, the center of the observation field of view is always moved to be on the field of view movement trajectory, so the user can always observe the object of observation from an easy-to-view position even while moving the observation field of view.

[0169] In this way, simply tilting the joystick 55b in a general direction allows the observation field of view to move along the specified route for easy viewing. The tilt direction of the joystick 55b is specified within a range of 0° to less than 360°. When using this joystick 55b to move along the route RT of the field of view movement trajectory of the linear sections I to IV as shown in FIG. 34, as described above, the movement in the field of view movement direction continues as long as the joystick 55b is tilted within a range of ±90° relative to the field of view movement direction, i.e., within the area on the traveling direction side defined by a line perpendicular to the field of view movement direction. On the other hand, if it is not within this range, i.e., if the joystick 55b is tilted in a direction generally opposite to the traveling direction or is not tilted, the movement of the observation field of view stops. Furthermore, if the field of view movement trajectory is a curve such as an arc, the tangent at the current position on the field of view movement trajectory is compared with the movement direction, and similarly, if it is within a predetermined angle range, route guidance continues.

[0170] On the other hand, when the movement direction indicated by the movement direction indicator 55a is at a position where the view field movement trajectory is bent, such as position II in Fig. 34, whether or not the observation view field can be moved is determined using the bisector that bisects the angle of the view field movement trajectory formed by the bent position. That is, in the example of Fig. 34, if the movement direction is indicated at an angle above the bisector indicated by the dashed line, the observation view field is moved in the direction from 2 to 3. On the other hand, if the movement direction is indicated at an angle below the bisector, the movement of the observation view field is stopped.

[0171] The movement speed of the observation field of view may also be changed depending on the tilt direction of the joystick 55b. For example, the movement speed of the observation field of view, for example, the movement speed of the stage unit 30, may be increased as the tilt direction of the joystick 55b is closer to the field of view movement trajectory, i.e., the smaller the angular difference between the two, and the movement speed may be decreased as the angular difference is greater. Furthermore, as described above, when the movement speed of the observation field of view, for example, the movement speed of the stage unit 30, is changed depending on the tilt angle of the joystick 55b in the vertical direction, the movement speed of the stage may be changed depending on the tilt direction of the joystick 55b in the plan view and the tilt angle in the vertical direction. (Field of view offset correction function)

[0172] It is also possible to change the display magnification in the image display area during observation. To ensure that the magnification can be changed and the device continues to operate according to the set route, the magnification observation device 100 is equipped with a field of view deviation correction offset function that automatically corrects field of view deviation when the magnification is changed. This will be described in detail below.

[0173] When switching the objective lens unit 25 or other components to change the magnification, the center position of the observation field may shift. For example, in the observation field shown in Figure 37A, if the display magnification is increased by mechanically switching the objective lens units 25 provided on a rotating revolver by rotating the revolver, the field center CS, indicated by the intersection of the cross-shaped grid lines, will be displayed shifted, as shown in Figure 37B. For this reason, a known technique is to calculate and store the shift amount of the field center CS in advance, and then, when the magnification is switched, move the XYZ stage by the shift amount to correct the field shift. For example, as shown in Figure 37C, the field shifting mechanism 5 is automatically moved from the state shown in Figure 37B so that the field center CS corresponds to the position of the field center CS shown in the previous Figure 37A.

[0174] However, when the route set in route tracing mode is managed using stage coordinates, if the XY stage is moved by such a correction, the field of view after switching will be on the route, but will be treated as being off the route in stage coordinates. Therefore, in the magnification observation device 100 according to this embodiment, when a shift is corrected, the route itself is offset by the amount of shift correction, as shown in FIG. 38, to match the route after the shift correction. This field of view shift correction offset function can be performed not only on the XY plane, but also in the Z direction, which is the height direction. Such a field of view shift correction offset function can be performed by the trajectory calculation unit 82.

[0175] Note that the field of view center shift when switching magnification is not limited to when switching the objective lens unit 25 using a revolver or the like or when physically replacing the objective lens unit 25, but can also occur when enlarging or reducing using a zoom optical system. The magnification observation device 100 according to this embodiment does not limit the use of the field of view shift correction offset function to when switching the objective lens unit 25, but can be used in any situation where field of view shift occurs.

[0176] The route offset function can also be used for other purposes. For example, if it is difficult to always place the observation target in the same position on a previously set route, the entire route can be offset by specifying that any point on the route corresponds to the current field of view. For example, by specifying a position on the field of view movement trajectory that corresponds to the observation field of view as shown by the dashed line in Figure 39, the entire field of view movement trajectory will be offset according to that specification, as shown by the solid line.

[0177] As described above, the magnification observation device 100 has a route tracing mode in which the route guide function is executed, and a free mode in which the route tracing mode is canceled. In the route tracing mode, as described above, the movement control unit 83 controls the movement direction instructing unit 55a to operate the field of view movement mechanism 5 so as to follow the field of view movement trajectory calculated by the trajectory calculating unit 82. On the other hand, in the free mode, the movement control unit 83 controls the field of view movement mechanism 5 to operate in the movement direction instructed by the movement direction instructing unit 55a, regardless of the field of view movement trajectory. With this configuration, the route tracing mode easily realizes movement of the observation field of view along the field of view movement trajectory, while also accommodating the user's free movement of the observation field of view.

[0178] Furthermore, when setting a route, it may be configured so that not only XY coordinates but also Z coordinates, i.e., the height direction, can be registered. This makes it possible to move along the desired observation position even when the observation target has an inclined surface. For example, when setting the circular field of view movement trajectory described above, by specifying three points as reference points and also acquiring height direction information, the inclination of the plane can be calculated from the height information of the three points. Therefore, by setting the field of view movement trajectory within the calculated plane, it becomes possible to realize the route guide function in a focused state along the inclined plane.

[0179] Routes in three-dimensional space can also be managed as a point cloud. They can also be managed as a figure projected onto the XY plane. For example, to set a circular view movement trajectory when viewed from directly above, three points specified in three-dimensional space as shown in Figure 40A are projected onto the XY plane as shown in Figure 40B, and a circle is defined on the XY plane based on the three projected points as shown in Figure 40C. This makes it possible to handle the view movement trajectory as data obtained by mapping three-dimensional data onto the XY plane, simplifying calculation processing.

[0180] The XY stage simply moves the stage unit 30 along a circle on the XY plane calculated as described above. The Z stage can move along a specified route in three-dimensional space as shown in Figure 41B by constantly calculating the Z coordinate corresponding to the XY coordinate of movement based on the plane calculated in three-dimensional space as shown in Figure 41A. By knowing the equation that defines the inclined plane calculated in three-dimensional space as shown in Figures 41A and 41B, the Z coordinate can be determined by projecting the XY plane onto this inclined plane.

[0181] In the case of a polygon, it can be projected onto the XY plane in the same way and the XY stage can be operated. In this case, by calculating the Z coordinate based on the slope between points as shown in Figures 42A and 42B, it becomes possible to move along the route intended by the user while moving between points.

[0182] Furthermore, even when routes intersect on the XY plane, the direction to proceed can be determined based on the direction of the joystick 55b, etc. For example, in the example shown in Fig. 43, if the joystick 55b is tilted within a range of ±90° with respect to the direction of travel, the robot will proceed straight, if it is in the range of +90° to +180°, the robot will turn left (upward in the figure), and if it is in the range of -90° to -180°, the robot will turn right (downward in the figure).

[0183] Alternatively, the direction of travel may be automatically determined based on the shape unit registered, such as a line segment or circle. Especially when the Z coordinate is also being tracked, it is preferable to determine the movement direction based on the shape unit. For example, if the Z coordinate intersects at a twisted position, as shown in Figure 44, an unintended Z-axis rise may occur, causing stress to the user. If the user operates the Z stage while the Z coordinate is being tracked, the route is offset to the position where the operation is completed. When the user moves the stage unit 30, it is likely that the object being observed is not in focus. For example, if the registration point is too close, as shown in Figure 45, the movement may cause the object to become out of focus. The process of offsetting the route to the Z coordinate operated by the user is also preferably performed when the Z stage is moved while the XY stage is stopped. For example, as shown in the lower left of Figure 46, if the XY movement is temporarily stopped and the Z movement is manually performed while the system is automatically tracking the route in the XYZ directions, when movement in the XY directions is resumed, the system will move along the offset route.

[0184] Furthermore, if managing Z stage operation while the XY stage is moving becomes cumbersome, you can either exclude Z stage operation or stop Z tracking when the stage is operated. For example, as shown in the lower left of Figure 47, if the stage is being automatically tracked in the XYZ directions along the route, and then manually moved in the Z direction during XY movement, Z tracking is stopped and the route guide function works only for movement in the XY directions.

[0185] It is also possible to configure the offset not to be performed even if movement is made in the Z direction during movement in the X and Y directions, and to perform the offset only when movement in the X and Y directions is stopped.

[0186] If the offset is set to the Z coordinate at the time the XY stage movement is started, even if the Z tracking stops working and blur becomes noticeable, the user can simply stop the XY stage, focus once, and then resume XY stage movement. [Embodiment 3]

[0187] The magnification observation device can also be equipped with an inclined observation function. Such an example is shown in Fig. 48 as a magnification observation device according to embodiment 3. The magnification observation device 300 shown in Fig. 48 is a front view showing the imaging system 1. Note that in the magnification observation device shown in the figure, the same members as those explained in the above-mentioned embodiment 1 etc. are given the same reference numerals and detailed explanations are omitted as appropriate.

[0188] The imaging system 1 comprises a stage unit 30 on which an object to be observed is placed, and a support base 40 that supports the head unit 4. The support base 40 comprises a stage fixing mechanism 42 that holds the stage unit 30 in a state where it can move in a horizontal plane or up and down, and a head tilting mechanism 44 that tilts the head unit 4 while holding the stage unit 30. The stage fixing mechanism 42 and head tilting mechanism 44 are fixed to a base unit 41. The base unit 41 is flat, allowing the support base 40 to stand independently in a stable manner.

[0189] By tilting the head unit 4 relative to the stage unit 30 using this head tilt mechanism 44, tilted observation, in which the observation object is observed from an oblique direction, becomes possible. In particular, by swinging the head unit 4 from a vertical position to the left or right about the swing axis 45 as a rotation axis, observation from either the left or right direction becomes possible, thereby increasing the degree of freedom in observation by observing from different viewpoints. Furthermore, such tilted observation requires eucentric observation, in which the observation field of view does not change even when the head unit 4 is tilted. For this reason, when performing tilted observation, it is desirable to adjust the height of the stage unit 30 in advance so that the observation surface of the observation object S coincides with the center of the swing axis 45.

[0190] Furthermore, the magnification observation device may be configured to be able to execute a searchlighting function even during tilted observation. When the searchlighting function is executed at one tilt angle and the sequence shifts from the first sequence to the second sequence, the magnification observation device may be configured to shift back to the first sequence when the user adjusts the tilt angle to the second tilt angle. In this case, images buffered for each lighting condition at one tilt angle are discarded when the user adjusts the tilt angle to the second tilt angle.

[0191] Furthermore, the method for achieving autofocus is not limited to the contrast method described above, and other methods, such as a phase-difference method, may also be used. For example, the magnification observation device 200 according to the second embodiment shown in FIG. 24 illustrates an example using the phase-difference method. The magnification observation device shown in this figure includes an autofocus sensor 15. The autofocus sensor 15 is composed of a phase-difference autofocus sensor. The phase-difference autofocus sensor splits and receives light incident from the objective lens unit 25, and the focus direction and amount are determined as focus-degree features from the distance between the two formed images by the control unit 80 (focus-degree evaluation unit). Compared to the contrast method described above, this method allows for faster focusing because it does not require moving the objective lens unit 25 to search for the focus. The autofocus sensor is not limited to a phase-difference autofocus sensor, and may be, for example, a distance measurement sensor that measures the distance between the objective lens unit and the object being observed. In this case, autofocusing can be achieved by using a distance sensor to measure the distance between the objective lens unit and the object being observed, calculating the difference between this and the previously measured focal length of the objective lens unit, and then using the autofocus adjustment mechanism to move the Z stage or the like by the corresponding distance. [Industrial Applicability]

[0192] The magnification observation device, magnified image observation method, magnified image observation program, computer-readable recording medium, and device storing the same of the present invention can be suitably used in microscopes, reflective and transmissive digital microscopes, and the like. [Explanation of symbols]

[0193] 100, 200, 300...Magnifying observation device 1. Imaging system 2...Control system 3...Cable section 4...Head section 5...Field of view shift mechanism 10...Camera section 11...Imaging optical system 12...imaging element; 13...imaging element control circuit 15...Autofocus sensor 16...Upper Z elevator 20...Microscope lens part 25...objective lens unit; 25-1 to 25-4...relative movement locus of the objective lens unit 26, 27, 28...Control direction 30...Stage section 31...Area with height information 32...3D data 35...Lower stage elevator 36...Motor control circuit 37...Stepping motor 40...Support stand 41...Base 43...Camera mounting part 42...Stage fixing mechanism 44...Head tilt mechanism 45...Oscillating shaft 50...main body section; 52...display control section 53...Storage section; 54...Interface; 55...operation unit; 55a...movement direction indicator; 55b...joystick 55c...Movement button 56...Memory section; 57...Buffer memory 60...Lighting section 61...Beam splitter 62…Coaxial epi-illumination section 63... ring lighting unit; 63a, 63b, 63c, 63d... lighting blocks 66...Lighting control unit 70...Display section 80...Processor section 81...trajectory instruction section; 82...trajectory calculation section; 83...Movement control unit 84...Image processing unit 88...Feature calculation unit 89...Height information acquisition unit 90...Focus control section 91...Focus degree evaluation unit 92...Frame skip section 93...Focus sequence execution unit 94...Movement stop detection unit 230...Navigation image registration screen 231...Image display area 232…Operation area 233...Navigation area 241...Image display area 242…Operation area 252..."Plane Fit" button 255..."Perform depth stacking when XY stage stops" checkbox 260...Searchlighting screen 261...Image display area 262…Operation area 263...Item tab; 263a..."Illuminance / Brightness" tab 264... "Back" button 265..."Searchlighting" button 266…Details specification field 267...Light intensity adjustment slider 268... "Multi-lighting" button 269...Status display column 270...ROI designation section; 270a..."Rectangle" button; 270b..."Free line" button; 270c...Slider 271...Color selection button 272...Lighting status display button 273... "Re-preview" button 280…Lighting light source display screen 430...Multi-lighting screen 431...Function display area 432…Main display area 433...Sub display area 433a…Emission direction specification field 433b…Emission direction display field SI…Image b1...Depth stacking button b2...Adjustment button b7...Save button sp...target part image ss0...Object position image ss1...Light icon ss2...Base point image ss3...hemispherical image ss4...Exit position image WK, WK2 to WK4...observation object CR, CR1 to CR5... Chip resistors SD…Solder CB…board SQ1...Multi-point height acquisition (autofocus operation) SQ2...Sequence on the move SQ3...Stop sequence (autofocus operation) SQ4...Sequence on the move SQ5...Stop sequence (autofocus operation) GS1~GS6…geometric shape AX…Optical axis NI...Navigation image FR…Rectangular shape OA: Observation field CS…center of vision PL…Contour RT…Route DS1: Route tangential direction DJ1: Joystick tilt direction IC…Icon MC...Mouse cursor

Claims

1. an objective lens unit arranged to face an observation object; The illumination direction in which illumination light is irradiated toward the observation object can be switched, and a first illumination pattern that sequentially switches between a plurality of different illumination directions; a second illumination pattern that illuminates in one of a plurality of illumination directions that constitute the first illumination pattern; a lighting unit that illuminates with either of the above; a camera unit that captures an image of the object to be observed formed through the objective lens unit and generates image data representing the image; a display control unit for displaying an image of the observation object on a display unit based on image data generated by the camera unit; an area designation unit for receiving designation of a boundary shape surrounding a desired area or a line segment corresponding to the desired area on the image data displayed on the display unit by the display control unit; a feature amount calculation unit that calculates a feature amount within a desired region designated by the region designation unit for each image data of the observation target illuminated in each illumination direction of the first illumination pattern; an illumination control unit capable of switching between a first sequence in which the illumination unit operates in the first illumination pattern and a second sequence in which the illumination unit operates in the second illumination pattern; Equipped with In the first sequence, the illumination control unit operates the illumination unit in the first illumination pattern in which illumination light is sequentially irradiated onto the observation object in a plurality of different illumination directions; the camera unit sequentially generates image data representing images of the observation object corresponding to a plurality of different illumination directions of the first illumination pattern; the display control unit causes the display unit to sequentially display image data sequentially generated by the camera unit; the area designation unit accepts designation of a desired area on the image data displayed on the display unit by the display control unit; the feature amount calculation unit calculates feature amounts within the region accepted by the region designation unit for each piece of image data showing an image of the object of observation corresponding to a plurality of different illumination directions of the first illumination pattern sequentially generated by the camera unit; the illumination control unit further selects an illumination direction by the illumination unit based on the desired internal feature of each image data calculated by the feature calculation unit, and switches the operation of the illumination unit from the first illumination pattern, which is a sequential switching among a plurality of different illumination directions, to the second illumination pattern, which is fixed to the selected illumination direction, to transition to the second sequence; In the second sequence, the camera unit generates image data representing an image of the object to be observed illuminated with the second illumination pattern; The display control unit displays image data showing an image of the object to be observed illuminated with the second illumination pattern on the display unit.

2. The magnification observation device according to claim 1, the area designation unit accepts designation of a desired area on live image data sequentially displayed on the display unit by the display control unit; the feature calculation unit calculates feature amounts within the region accepted by the region designation unit for a plurality of image data representing images of the object to be observed corresponding to a plurality of different illumination directions of the first illumination pattern sequentially generated by the camera unit in the first sequence, The illumination control unit selects the illumination direction of the illumination unit based on the feature amount of each image data calculated by the feature amount calculation unit, and is configured to transition from the first sequence to the second sequence.

3. The magnification observation device according to claim 2, further comprising: a buffer memory for temporarily storing each image data set with different illumination directions, which is displayed on the display unit by the display control unit in the first sequence; When a plurality of image data sets with different illumination directions for one cycle, which is a period for going around the different illumination directions included in the first illumination pattern, are stored in the buffer memory at the time when the region is specified by the region specifying unit, the feature amount calculation unit calculates a feature amount of the image data within the specified region for each of the stored image data sets, The illumination control unit is configured to select one of the illumination directions based on the feature amount within the specified region of one cycle of image data calculated by the feature amount calculation unit, and to transition from the first sequence to the second sequence.

4. The magnification observation device according to claim 3, when the buffer memory does not hold a plurality of image data corresponding to one cycle and having different illumination directions at the time when the region is specified by the region specifying unit, the illumination control unit maintains the first sequence until the image data corresponding to one cycle can be buffered in the buffer memory; the feature amount calculation unit calculates a feature amount of the image data within a designated area for each of the image data for one cycle held in the buffer memory; The illumination control unit is further configured to select one of the illumination directions based on the feature amount within the specified region of the one cycle of image data calculated by the feature amount calculation unit, and to transition from the first sequence to the second sequence.

5. The magnification observation device according to claim 3 or 4, In the second sequence, the region designation unit accepts re-designation of a desired region on the image data showing an image of the observation object illuminated with the second illumination pattern, which is displayed on the display unit; the feature amount calculation unit calculates feature amounts within the area re-designated by the area designation unit for each of the plurality of image data for one cycle having different illumination directions and stored in the buffer memory; The illumination control unit selects one of the illumination directions based on the feature amount in the specified region of the one cycle's worth of image data calculated by the feature amount calculation unit, and switches the illumination direction of the illumination unit to the selected illumination direction.

6. The magnification observation device according to any one of claims 1 to 5, The illumination control unit is configured to select the illumination direction corresponding to the image data having the largest feature amount calculated by the feature amount calculation unit as the illumination direction of the second sequence.

7. The magnification observation device according to any one of claims 1 to 6, A magnification observation device, wherein the feature amount of the image data is the variance of the luminance values ​​of each pixel of the image data.

8. The magnification observation device according to any one of claims 1 to 7, the illumination unit includes a plurality of illumination blocks arranged in a circular ring; A magnification observation device in which the illumination control unit is configured to switch the illumination direction in the first sequence by sequentially switching the illuminated illumination blocks among the plurality of illumination blocks based on predetermined sequence information.

9. The magnification observation device according to claim 8, The illumination unit includes a coaxial illumination unit having the plurality of illumination blocks.

10. The magnification observation device according to claim 9, the illumination unit includes a coaxial illumination unit having the plurality of illumination blocks and a selectively attached ring illumination unit; When the ring illumination is attached, the illumination control unit sequentially switches the illumination blocks included in the ring illumination as the plurality of illumination blocks, A magnification observation device configured such that when the ring illumination is not attached, the illumination control unit sequentially switches each illumination block included in the coaxial illumination as the plurality of illumination blocks.

11. The magnification observation device according to any one of claims 1 to 10, A magnification observation device in which the illumination unit controls switching from one illumination direction to a different illumination direction when irradiating with the first illumination pattern so as to provide a period in which these two illumination directions overlap in time.

12. The magnification observation device according to any one of claims 1 to 11, The illumination control unit, in the second sequence, operates the magnification observation device with the second illumination pattern using any one of the illumination directions selected based on the feature amounts of each image data calculated by the feature calculation unit, and the display control unit displays on the display unit a composite image of the observation object illuminated with the second illumination pattern.

13. The magnification observation device according to any one of claims 1 to 12, further comprising: a focus adjustment mechanism that adjusts the focus of image data by moving a relative distance between a focal position of the objective lens unit and the observation object along an optical axis of the objective lens unit in either a direction toward or a direction away from each other, the focus adjustment mechanism performs autofocusing in accordance with predetermined conditions; The magnification observation device is configured such that the illumination unit temporarily stops switching of the illumination direction during autofocusing by the focus adjustment mechanism.

14. The magnification observation device according to any one of claims 1 to 12, further comprising: a field of view moving mechanism that changes the relative position of the optical axis of the objective lens unit on the stage unit to move the observation field of view of the camera unit, The illumination control unit is configured to execute the first sequence while the field of view movement mechanism is moving the observation field of view.

15. The magnification observation device according to claim 14, further comprising: a focus adjustment mechanism that adjusts the focus of image data by moving a relative distance between a focal position of the objective lens unit and the observation object along an optical axis of the objective lens unit in either a direction toward or a direction away from each other; a movement stop detection unit that detects a movement stop state of the observation field of the camera unit; It is equipped with the focus adjustment mechanism performs autofocus in response to detection of a movement stop state by the movement stop detection unit, The magnification observation device is configured such that the illumination unit temporarily stops switching of the illumination direction during autofocusing by the focus adjustment mechanism.

16. The magnification observation device according to claim 15, further comprising: a focus control unit that controls the focus adjustment mechanism so as to achieve focus based on three-dimensional reference information when the field of view movement mechanism moves the observation field of view of the camera unit; a height information acquisition unit that acquires height information of the object to be observed when the autofocus is performed, and updates the three-dimensional reference information based on the acquired height information.

17. The magnification observation device according to claim 16, further comprising: a movement direction instructing unit that instructs a movement direction of the field of view moving mechanism in accordance with a user input indicating a movement direction of the observation field of view on the display unit; a trajectory instruction unit for instructing trajectory information regarding the definition of the moving direction of the observation field for the image data displayed on the display unit by the display control unit; a trajectory calculation unit that calculates a field of view movement trajectory for moving the observation field of view based on the trajectory information instructed by the trajectory instruction unit; a movement control unit that controls movement of the visual field movement mechanism in accordance with a movement direction instruction from the movement direction instruction unit along the visual field movement trajectory calculated by the trajectory calculation unit; A magnification observation device comprising:

18. A magnified image observation method for observing an object illuminated by an illumination unit capable of switching the irradiation direction, by moving an observation field of view in which an image is captured by a camera unit via an objective lens unit and displayed on a display unit, comprising: a step of operating the illumination unit with a first illumination pattern that sequentially switches between different illumination directions to illuminate the object, causing the camera unit to sequentially generate image data showing images of the object to be observed corresponding to a plurality of different illumination directions of the first illumination pattern, causing the display unit to sequentially display live images of the object to be observed sequentially generated by the camera unit, and executing a first sequence that accepts designation of a desired area on the image data displayed on the display unit; a step of executing a second sequence in which, for each image data of the observation object illuminated in each illumination direction of the first illumination pattern, a feature amount of the image data is calculated within the region, an illumination direction is selected based on the feature amount of the image data, an operation of the illumination unit is switched from the first illumination pattern, which is a sequential switching of a plurality of different illumination directions, to a second illumination pattern, which is fixed to the selected illumination direction, and the object of observation illuminated with the second illumination pattern is displayed on the display unit; A method for observing a magnified image, comprising:

19. a stage portion for placing an object to be observed; an objective lens unit disposed facing the observation object placed on the stage unit; an illumination unit that irradiates illumination light toward the observation object; a camera unit that captures an image of the observation object illuminated by the illumination unit through the objective lens unit and generates image data; a display unit that displays an image of an observation field including the object to be observed based on image data generated by the camera unit; A magnified image observation program for operating a magnification observation device comprising: a function of executing a first sequence in which the illumination unit is operated with a first illumination pattern in which illumination is irradiated by sequentially switching between different illumination directions, the camera unit is caused to sequentially generate image data showing images of the observation object corresponding to a plurality of different illumination directions of the first illumination pattern, the display unit is caused to sequentially display live images of the observation object generated by the camera unit, and a desired area is designated on the image data displayed on the display unit; a function of executing a second sequence of calculating, for each image data of the object of observation illuminated in each illumination direction of the first illumination pattern within the region, selecting an illumination direction based on the feature amount of the image data, switching the operation of the illumination unit from the first illumination pattern, which is a sequential switching of a plurality of different illumination directions, to a second illumination pattern, which is fixed to the selected illumination direction, and displaying the object of observation illuminated with the second illumination pattern on the display unit; A magnified image observation program that enables a computer to do this.

20. A computer-readable recording medium or storage device storing the program according to claim 19.

Images