Observation apparatus, observation method, and program

Abstract

To provide an observation device, observation method, and program which allow a user to easily confirm an observation object.SOLUTION: An observation device 10 comprises: a head 21 which has a digital camera 27 being one example of an imaging unit that images an observation object S and which can be inclined; a display device 40 being one example of a display unit which displays an image of the observation object S captured by the digital camera 27; an inclination angle detection unit (for example, Z-axis holding unit 242) which detects the inclination angle of the head 21; and a control device 30 being one example of a control unit which causes the display device 40 to display the plurality of images captured by the digital camera 27 in a plurality of states with the different inclination angles of the head 21 so as to be arrayed on the basis of the inclination angle of the head 21.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The disclosure of the present specification relates to an observation apparatus, an observation method, and a program. [Background Art]

[0002] Conventionally, a peripheral three-dimensional observation apparatus has been proposed that includes a mirror provided at the tip of an objective lens, rotates the mirror by rotating a rotating frame, and irradiates illumination onto the outer peripheral surface of a subject (see, for example, Patent Document 1).

[0003] Furthermore, a magnified observation apparatus is known in which a head unit attached to a head tilting mechanism can swing around a swing shaft, and a stage rotates an observation object (see, for example, Patent Document 2). [Prior Art Documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent No. 5944736 [Patent Document 2] Japanese Unexamined Patent Publication No. 2015-127777 [Summary of the Invention] [Problem to be Solved by the Invention]

[0005] Incidentally, when a head having an imaging unit that images an observation target can be tilted, the observation target can be observed from different directions. Furthermore, when a stage on which the observation target is placed is rotatable, the observation target can be observed from any direction.

[0006] However, when the image of the object being observed is displayed on the screen, the user will need to check the image multiple times when searching for defects in the object. Therefore, if the images are simply displayed in the order they were acquired when the object is photographed from multiple directions, it is not intuitively clear from which direction each image was taken. As a result, it is not possible to check the image while grasping the entire object.

[0007] Based on the circumstances described above, one aspect of the present invention is to provide an observation device, observation method, and program that facilitate the user's confirmation of the object being observed. [Means for solving the problem]

[0008] In one embodiment, the observation device includes an imaging unit for capturing images of an object to be observed, a tiltable head, a display unit for displaying images of the object to be observed captured by the imaging unit, a tilt angle detection unit for detecting the tilt angle of the head, and a control unit for arranging and displaying on the display unit multiple images captured by the imaging unit in multiple states with different tilt angles of the head, based on the tilt angle of the head.

[0009] In another embodiment, the observation method obtains the tilt angle of a tiltable head having an imaging unit that images an object to be observed, obtains a plurality of images captured by the imaging unit in a plurality of states with different tilt angles of the head, and displays the plurality of images on a display unit arranged according to the tilt angle of the head.

[0010] In another embodiment, the program causes the computer to perform the following functions: acquire the tilt angle of a tiltable head having an imager for capturing an object to be observed; acquire a plurality of images captured by the imager in a plurality of states with different tilt angles of the head; and arrange the plurality of images based on the tilt angle of the head and display them on a display unit. [Effects of the Invention]

[0011] According to the above aspect, it is possible to facilitate the user's confirmation of the outer circumferential surface of the observation target. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] [Figure 1] It is a right side view showing the configuration of an observation apparatus according to one embodiment. [Figure 2] It is a right side view showing a part of an observation apparatus according to one embodiment. [Figure 3] It is a front view showing a part of an observation apparatus according to one embodiment. [Figure 4] It is a front view showing an XY stage and a rotation stage according to one embodiment. [Figure 5] It is a diagram showing a display screen according to one embodiment. [Figure 6] It is a plan view for explaining rotation of a stage according to one embodiment. [Figure 7] It is a flowchart for explaining an observation method according to one embodiment. [Figure 8] It is an explanatory diagram for explaining the arrangement of captured images according to one embodiment. [Figure 9] It is an explanatory diagram for explaining rotation of a captured image according to one embodiment. [Figure 10] It is an explanatory diagram for explaining another example of the arrangement of captured images according to one embodiment. [Figure 11] It is a diagram showing an example of display content of a display screen according to one embodiment. [Figure 12] It is a diagram showing another example of display content of a display screen according to one embodiment. [Figure 13] It is a diagram showing the relationship between contrast and Z position for explaining determination of a focus position according to one embodiment. [Figure 14] It is a diagram illustrating an example of the hardware configuration of a computer. DETAILED DESCRIPTION OF EMBODIMENTS

[0013] Hereinafter, an observation apparatus, an observation method, and a program according to an embodiment of the present invention will be described with reference to the drawings.

[0014] FIG. 1 is a right side view showing a configuration of an observation apparatus 10 according to an embodiment.

[0015] Note that the X-axis direction, Y-axis direction, and Z-axis direction shown in FIG. 1 and FIGS. 2 to 4 and 6 described below are, for example, horizontal directions in which the X-axis direction and the Y-axis direction are orthogonal to each other, and the Z-axis direction is a vertical direction.

[0016] The observation apparatus 10 shown in FIG. 1 includes a microscope 20, a control device 30 that controls the microscope 20, a display device 40, and an input device 50. When the microscope 20 is a digital microscope or the like including a display unit (the display device 40) and a control unit (the control device 30), the microscope 20 itself can function as the observation apparatus.

[0017] The microscope 20 includes a head 21, a light source 22, an observation optical system 23, a tilt unit 24, a stage 25, a frame 26, a digital camera 27, and an operation unit 28. The light source 22 and the digital camera 27 are provided inside the head 21 together with an optical path splitting element (splitter) such as a half mirror, a dichroic mirror, or a polarizing beam splitter. The light source 22 is disposed on one of the optical paths branched by the splitter, and the digital camera 27 is disposed on the other.

[0018] The head 21 is held by the frame 26 via the tilt section 24. The head 21 is an example of a focusing unit that changes the distance between the observation target (sample) S, which is, for example, an electronic component on a substrate, and the observation optical system 23. While held by the tilt section 24, it is movable parallel to the optical axis of the objective lens 231 (observation optical axis A shown in Figure 3). The head 21, together with the tilt section 24 held by the frame 26, rotates (tilts) around the rotation axis 241 shown in Figure 2 (in the Y-axis direction) in inclination angles +T and -T, as shown in Figure 3. The center of the rotation axis 241 preferably extends through the upper surface of the stage 25 or the observation target S or its vicinity, enabling eucentric observation. The rotation axis 241 is preferably fixed to the frame 26 and held by the bearing of the tilt section 24. The rotation axis 241 may also have a scale head that reads the position information of a scale provided on the frame 26. This scale head detects the rotation angle of the rotating shaft 241 by reading the position information of the scale on the frame 26. Preferably, the scale head is a rotary encoder, for example. Thus, the microscope 20 is equipped with a tilt angle detection unit that detects the tilt angle of the head 21 (and tilt unit 24) by providing a scale head (rotary encoder).

[0019] The movement of the head 21 in the direction of the observation optical axis A (see Figure 3) can be performed manually, for example, but may also be controlled by the control device 30. When the head 21 moves in the direction of the observation optical axis A, the control device 30 detects the position of the head 21 in the direction of the observation optical axis A (Z-axis direction) by reading the position information of the linear scale of the Z-axis holding part 242 of the tilt part 24 using a scale head fixed to the head 21. Note that the direction of movement of the head 21, the direction of the observation optical axis A, is the Z-axis direction (vertical direction) when the head 21 is upright, but when the head 21 is tilted, it becomes a direction inclined with respect to the Z-axis direction.

[0020] The tilt of the head 21 is performed manually by the user. The tilt angle of the head 21 is, for example, in the range of plus or minus 90° from the upright position.

[0021] Light source 22 is a light source for incident illumination, and is, for example, a white LED. Light source 22 may be other light sources such as a xenon lamp or a halogen lamp. The illumination light emitted from light source 22 is collimated by an illumination lens. The collimated illumination light is reflected by a splitter and irradiated onto the object of observation S placed on the stage 25 via the objective lens 231.

[0022] The observation optical system 23 projects an optical image of the object being observed S, illuminated by illumination light, onto the digital camera 27. The observation optical system 23 includes, for example, an objective lens 231 and an imaging lens. The objective lens 231 is switchable via a revolving nosepiece. Multiple objective lenses of different magnifications may be mounted on the revolving nosepiece, and any objective lens from among the multiple objective lenses may be positioned on the observation optical axis A according to the user's selection.

[0023] The digital camera 27 is an example of an imaging unit that images the object of observation S. The digital camera 27 includes an image sensor 271. This image sensor 271 is, for example, an image sensor such as a CCD image sensor or a CMOS image sensor. The digital camera 27 images the object of observation and outputs the image of the object of observation S to the control device 30. As a result, the control device 30 functions as an acquisition unit that acquires the captured image.

[0024] The stage 25 on which the object of observation S is placed includes an XY stage 251, a rotation stage 252, and a Z stage 253.

[0025] As shown in Figure 4, the XY stage 251 includes an X-axis movable mechanism 251x and a Y-axis movable mechanism 251y. The X-axis movable mechanism 251x includes a feed mechanism such as a ball screw 251xa for moving the mounting plate of the object to be observed S in the X-axis direction, an X-coordinate detection mechanism such as a scale 251xb and a scale head 251xc, an actuator such as a motor 251xd, a handle 251xe for manual operation, and a connecting portion 251xf with a screw (not shown) for fixing the X-axis movable mechanism 251x. The Y-axis movable mechanism 251y also includes a feed mechanism such as a ball screw for moving the mounting plate of the object to be observed S in the Y-axis direction, a Y-coordinate detection mechanism such as a scale and a scale head, and an actuator such as a motor, although only a handle 251ya for manual operation is shown.

[0026] The XY stage 251 is, for example, an electric stage, and its position in the XY direction is controlled by the control device 30. The XY stage 251 is mounted on the rotary stage 252. In the example shown in Figure 4, a motor 251xd and handles 251xe, 251ya are provided for both the X-axis movable mechanism 251x and the Y-axis movable mechanism 251y, respectively, allowing for movement by both electric and manual means. However, one of the motor 251xd and the handles 251xe, 251ya may be omitted, allowing for movement by either electric or manual means only.

[0027] As shown in Figure 3, the rotating stage 252 rotates the XY stage 251 around a rotation center C that extends vertically, thereby rotating the object of observation S to the +R° and -R° sides of the rotation angle, as shown in Figure 6. The rotation direction of the rotating stage 252 may be either clockwise or counterclockwise. The rotation center C of the rotating stage 252 coincides with the observation optical axis A of the upright head 21. The rotating stage 252 functions as an example of a rotation angle detection unit that detects the rotation angle of the stage 25 by having, for example, a scale scale head (e.g., a rotary encoder) that detects the rotation angle of the stage 25. The rotating stage 252 also includes an actuator such as a motor. The rotation angle detection unit that detects the rotation angle of the stage 25 may be a member that directly detects the rotation angle, such as a rotary encoder or a magnetic sensor, or it may be the control device 30 itself, for example, when the control device 30 counts the number of steps of a stepping motor or detects the orientation of a mark (indicator: e.g., a single line) on the stage 25.

[0028] The Z-stage 253 is an example of a focusing unit that changes the distance between the observation target S and the observation optical system 23. The Z-stage 253 is, for example, an electric stage or a manual stage, and in the case of a manual stage, its vertical position (Z-direction) can be adjusted with a handle (not shown). Note that the XY stage 251, the rotation stage 252, and the Z-stage 253 are not limited to this example and may each be an electric stage or a manual stage.

[0029] The control unit 28 shown in Figure 1 receives various operational information from the user regarding the operation of the microscope 20, and sends this input information to the control device 30, which will be described later. For example, the control unit 28 may have at least one of the following for operating the stage 25 and head 21: a handle, switch, button, mouse, keyboard, joystick, dial, touch panel, etc.

[0030] The control device 30 is a device that controls the microscope 20 and is an example of a control unit of the observation device 10. The control device 30 may be composed of multiple devices. For example, the control device 30 may include a microscope controller that is mainly responsible for controlling the operation of the motorized part of the microscope 20 and a general-purpose computer that is mainly responsible for image processing of images acquired by the microscope 20.

[0031] The display device 40 is an example of the display unit of the observation device 10, and is any display such as a liquid crystal display or an organic EL display.

[0032] The display device 40 displays, for example, the display screen 41 shown in Figure 5. The content of this display screen 41 will be described later, but the display screen 41 may have multiple operation buttons 41a, an image display unit 41b, etc. The operation buttons 41a are provided in the display area of ​​the display screen 41 and are selected by operation of the touch panel or cursor operation using the input device 50, but they may also be provided in an area other than the display area and pressed. Alternatively, the operation buttons 41a may be omitted, and operations similar to those using the operation buttons 41a may be performed on the operation unit 28 of the microscope 20, the input device 50, etc.

[0033] The input device 50 is any input device such as a keyboard, mouse, touchpad, or joystick.

[0034] Next, the observation method according to this embodiment will be described with reference to Figure 7.

[0035] First, when the power to the microscope 20 (observation device 10) is turned on, the control device 30 determines whether the microscope 20 is in an upright position, that is, whether the inclination of the observation optical axis A of the head 21 with respect to the rotation center C of the stage 25 is 0°. If it is not in an upright position, the control device 30 notifies the user to manually set the microscope 20 to an upright position (step S1). The notification to the user may be made, for example, by display on the display device 40 or by outputting sound from an audio output unit (not shown). Furthermore, when the observation optical axis A and the rotation center C coincide, the control device 30 may identify the shape of the periphery of the observation target S or identify the height of the observation target S when creating a 3D model, based on the image captured by the digital camera 27.

[0036] Next, the control device 30 accepts the preview setting (step S2). As will be described later, this preview setting is a mode in which multiple images captured by the digital camera 27 in multiple states with different tilt angles of the head 21 (and rotation angles of the stage 25) are arranged based on the tilt angle of the head 21 and displayed on the display device 40 (a display mode that performs the display process S5, which will be described later).

[0037] Preview settings may be performed based on a manual display start operation by the user, such as selecting the preview setting button on the operation button 41a of the display screen 41. Alternatively, preview settings may be set automatically or the user may be notified to prompt the user to perform the setting, for example, when the head 21 is tilted (the observation optical axis A of the head 21 is tilted relative to the rotation center C of the stage 25), when the head 21 is tilted to a threshold tilt angle (e.g., 90°), when the stage 25 (rotating stage 252) is rotated, or when the stage 25 is rotated to a threshold rotation angle.

[0038] Next, the control device 30 informs the user of the tilt angle of the head 21 and the rotation angle of the stage 25 during imaging (step S3). If the control device 30 rotates the stage 25 electrically, it rotates the stage 25 to the informed angle. The control device 30 may determine multiple rotation angles of the stage 25 based on the user's specifications. In addition, the control device 30 informs the user, for example, to set the reference angle of the stage 25 to the initial angle of 0° during the first imaging, but the rotation angle of the stage 25 before rotation may be considered as the reference angle of 0°.

[0039] Subsequently, the control device 30 determines whether the head 21 is in the focused position. If the head 21 is not in the focused position, the control device 30 adjusts it to the focused position by moving the head 21 in the direction of the observation optical axis A relative to the tilt unit 24, or by moving the observation target S using the stage 25. For example, the control device 30 can look at the contrast value of a certain pixel (area) and determine the Z position at the peak value with the highest contrast value from the relationship between the Z position (position in the direction of the observation optical axis A) and the contrast value shown in Figure 13 as the focused position. By performing this for all pixels (areas) and moving the head 21 to the Z position, a fully focused image or a 3D image can be generated using a known method. When generating a fully focused image or a 3D image in this way, the roughness of the outer surface of the observation target S may also be measured.

[0040] Next, the control device 30 acquires the images captured by the digital camera 27 (step S4). The control device 30 also acquires and stores the tilt angle of the head 21 and the rotation angle and coordinates of the stage 25 for each of the captured images. The control device 30 may store multiple images as header information, with at least one of the tilt angle of the head 21, the rotation angle of the stage 25, and the coordinates of the stage 25 being used when multiple images are captured. Image capture by the digital camera 27 can be performed manually, semi-automatically, or automatically. When manual image capture is performed, the user performs the image capture operation by rotating the head 21 at an angle, for example, while looking at the rotation angle of the head 21 displayed on the display screen 41. When manual image capture is performed, for example, a preset angle may be announced, or when the stage 25 or head 21 reaches a preset angle, the frame of the corresponding display area among the nine display areas in Figure 8 may be highlighted as the RoI, or the angle at which the image is acquired may be determined by the user looking at the displayed rotation angle value and determining the angle. When imaging is performed semi-automatically, the control device 30 controls the digital camera 27 to automatically image the object of observation S when the head 21 is tilted to a predetermined inclination angle and the stage 25 is rotated to a predetermined rotation angle by the user's manual operation, or at predetermined intervals. When imaging is performed by the digital camera 27, it is preferable to temporarily lock the tilt of the head 21 and the rotation of the stage 25. When imaging is performed automatically, the control device 30 controls the head 21 to tilt to a predetermined inclination angle and the stage 25 to rotate to a predetermined rotation angle before imaging is performed. Here, the tilt of the head 21 may be performed manually or automatically. The order in which each image is captured may be multiple images taken by tilting the head 21 at each rotation angle of the stage 25, or multiple images taken by rotating the stage 25 at each inclination angle of the head 21.

[0041] The control device 30 arranges the acquired captured images P1 to P9 based on the tilt angle of the head 21 and the rotation angle of the stage 25 and displays them on the display screen 41 (image display unit 41b) of the display device 40 as shown in Figure 5 (step S5). For example, as shown in Figure 8, the display device 40 displays the captured image P1 of the object of observation S (for example, a rectangular parallelepiped shape that is long in the front-to-back direction with a circular mark on the upper surface on the front side) captured by the digital camera 27 in an upright state with the head 21 not tilted, on one side (right side) the captured image P2 captured by the digital camera 27 with the head 21 tilted to one side (angle T:+), and on the opposite side (left side) the captured image P3 captured by the digital camera 27 with the head 21 tilted to the other side (angle T:-), on the opposite side (left side). Although the arrangement direction of the captured images P2 and P3 is left-right, the control device 30 should arrange them so that the arrangement direction differs depending on the rotation angle of the stage 25 (for example, if the rotation angle of the stage is 90°, the captured images P8 and P9 are arranged in the up-down direction) and display them on the display device 40.

[0042] For example, the control device 30 may arrange multiple images (captured images P2, P3) taken before rotating the stage 25 from a reference angle to the left and right of the upright image (captured image P1) so that the arrangement direction is left to right, and then arrange multiple images (captured images P8, P9) taken after rotating the stage 25 by 90° (an example of R1°) above and below the upright image (captured image P1) so that the arrangement direction is up and down, and display them on the display device 40.

[0043] Furthermore, the control device 30 may arrange multiple images (captured images P4, P5) acquired after rotating the stage 25 to +45° (an example of +R2°, where the absolute value is smaller than the absolute value of R1) to the upper right and lower left of the upright image (captured image P1), and display them on the display device 40, while arranging multiple images (captured images P6, P7) acquired after rotating the stage 25 to -45° (an example of -R2°) to the lower right and upper left of the upright image (captured image P1).

[0044] Furthermore, the control device 30 may display the tilt angle of the head 21 and the rotation angle of the stage 25 within or around each captured image P1 to P9.

[0045] Returning to the flowchart in Figure 7, the control device 30 determines whether imaging has been completed at the predetermined tilt angle of the head 21 and the predetermined rotation angle of the stage 25 (step S6). If imaging is incomplete (step S6: NO), it returns to step S3. The end of imaging may be determined at any timing decided by the user, for example, based on the timing when the preview setting button of the operation button 41a is selected.

[0046] When the save button on operation button 41a is selected, the control device 30 checks the image P7 and P8 selected by touch panel operation or cursor operation and saves them. Alternatively, the control device 30 may save all image P1 to P9 at once when the save button on operation button 41a is selected.

[0047] Once imaging is complete (Step S6: YES), the control device 30 may, for example, automatically, or if the rotation button of the operation button 41a is selected, rotate the multiple captured images P4 to P9 taken after rotating the stage 25 by the rotation angle of the stage 25 in the opposite direction to the rotation of the stage 25, as shown in Figure 9, in order to align the orientation of each image, and display them on the display device 40 (Step S7). The control device 30 may also appropriately crop the rotated images into rectangles with four sides (vertical and horizontal) (Step S8). Then, the control device 30 terminates the process shown in Figure 7.

[0048] For example, to rotate the images, images P4 and P5, which are captured with a stage 25 rotation angle of 45°, can be changed to captured images P4r and P5r by rotating the image rotation angle r to -45°; images P6 and P7, which are captured with a stage 25 rotation angle of -45°, can be changed to captured images P6r and P7r by rotating the image rotation angle r to 45°; and images P8 and P9, which are captured with a stage 25 rotation angle of 90°, can be changed to captured images P8r and P9r by rotating the image rotation angle r to -90°.

[0049] Furthermore, if the reference angle of the stage 25 when multiple images (imaging images P1 to P3) were captured before the stage 25 was rotated deviates from a predetermined angle (for example, the initial angle of 0°), the control device 30 may rotate the multiple images captured before and after the stage 25 was rotated in the opposite direction by the rotation angle that deviated from the initial angle, and display them on the display device 40.

[0050] Here, the images displayed on the display screen 41 may be captured images P1, P2, P3, P11, and P12 arranged in only one direction, as shown in Figure 10. For example, the control device 30 may arrange captured images P2 and P11, captured by the digital camera 27 with the head 21 tilted at multiple angles (T: +45°, +67.5°) on one side, on the right side of the upright captured image P1, in order from the smallest tilt angle of the head 21 to the largest, so as to move away from the upright captured image P1. Then, captured images P3 and P12, captured by the digital camera 27 with the head 21 tilted at multiple angles (T: -45°, -67.5°) on the other side, on the opposite side of the upright captured image P1 (left side), in order from the smallest tilt angle (absolute value) of the head 21 to the largest, so as to move away from the upright captured image P1, and display them on the display device 40. Such arrangements of five or more images can also be applied, for example, when imaging is performed multiple times for each rotation angle of stage 25. That is, although Figure 8 illustrates an arrangement of nine images in a 3x3 grid, some or all of 17 images may be arranged in a 5x5 grid of 25 frames, with five images arranged in at least one direction (vertical, horizontal, or diagonal).

[0051] However, as shown in the example in Figure 8, if only images with a single tilt angle (e.g., T: +45°, -45°) on each side of the head 21 are displayed, the control device 30 may arrange some of the images captured by the digital camera 27 at multiple angles (e.g., T: +45°, +65°, or T: -45°, -67.5°) and display them on the display device 40, while storing the other images without displaying them.

[0052] As shown in Figure 11, the control device 30 may switch from a mode in which only multiple images are displayed on the image display section 41b of the display screen 41, to another mode, as shown in Figure 12, in which a preview image display section 41c that displays multiple images and a comment entry field 41d where comments can be entered, based on the selection of the toggle button (for example, the two triangular sections at the top) of the operation button 41a. In the example shown in Figure 12, it is preferable that the control device 30 be able to output data including the multiple images and comments displayed on the image display section 41b as a report.

[0053] The control device 30 may also display an enlarged image of a selected image from among the above-mentioned captured images P1 to P9, P11, and P12 on the display device 40.

[0054] Figure 14 is a diagram illustrating the hardware configuration of a computer 100 for realizing the control device 30 according to the above-described embodiment.

[0055] As shown in Figure 14, the computer 100 comprises a processor 101, memory 102, storage device 103, reader 104, communication interface 106, and input / output interface 107 as its hardware configuration. The processor 101, memory 102, storage device 103, reader 104, communication interface 106, and input / output interface 107 are connected to each other, for example, via a bus 108.

[0056] The processor 101 may be a single processor, a multi-processor, or a multi-core processor. The processor 101 operates as the control unit of the observation device 10 by reading and executing the program stored in the storage device 103.

[0057] Memory 102 is, for example, a semiconductor memory and may include a RAM area and a ROM area. Storage device 103 is, for example, a hard disk, a semiconductor memory such as flash memory, or an external storage device.

[0058] The reader 104 accesses the removable storage medium 105, for example, according to instructions from the processor 101. The removable storage medium 105 can be implemented by, for example, a semiconductor device, a medium through which information is input / output by magnetic action, or a medium through which information is input / output by optical action. A semiconductor device is, for example, a USB (Universal Serial Bus) memory. A medium through which information is input / output by magnetic action is, for example, a magnetic disk. A medium through which information is input / output by optical action is, for example, a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disk), a Blu-ray Disc, etc. (Blu-ray is a registered trademark).

[0059] The communication interface 106 communicates with other devices, for example, according to instructions from the processor 101. The input / output interface 107 is, for example, an interface between an input device and an output device. The input device is, for example, a device such as a keyboard, mouse, or touch panel that receives instructions from the user. The output device is, for example, a display device such as a display, and an audio device such as a speaker.

[0060] The program executed by the processor 101 is provided to the computer 100 in the following form, for example. (1) It is pre-installed on the storage device 103. (2) Provided by a removable storage medium 105. (3) Provided from a server such as a program server.

[0061] The hardware configuration of the computer 100 for realizing the control device 30, as described with reference to Figure 14, is illustrative and the embodiment is not limited thereto. For example, some of the above configuration may be deleted, or new configurations may be added. In another embodiment, for example, some or all of the functions of the calculation unit 42 described above may be implemented as hardware such as an FPGA (Field Programmable Gate Array), SoC (System-on-a-Chip), ASIC (Application Specific Integrated Circuit), and PLD (Programmable Logic Device). That is, any electrical circuit included in the control device 30 may perform the internal prediction processing described above.

[0062] In the embodiment described above, the observation device 10 includes a digital camera 27, which is an example of an imaging unit for capturing images of the object to be observed S; a tiltable head 21; a display device 40, which is an example of a display unit for displaying images of the object to be observed S captured by the digital camera 27; a tilt angle detection unit (for example, a rotating shaft unit 241) for detecting the tilt angle of the head 21; and a control device 30, which is an example of a control unit for arranging multiple images (for example, captured images P1 to P9) captured by the digital camera 27 in multiple states with different tilt angles of the head 21, based on the tilt angle of the head 21, and displaying them on the display device 40.

[0063] From another perspective, for example, the observation method performed by the observation device 10 involves acquiring the tilt angle of a tiltable head 21 having a digital camera 27, which is an example of an imaging unit that images the object of observation S, acquiring multiple images (e.g., captured images P1 to P9) captured by the digital camera 27 in multiple states with different tilt angles of the head 21, and arranging these multiple images based on the tilt angle of the head 21 and displaying them on a display device 40, which is an example of a display unit.

[0064] From another perspective, the program causes the computer 100 (e.g., control device 30) to perform the following functions: acquiring the tilt angle of a tiltable head 21 having a digital camera 27, which is an example of an imaging unit that images an object S being observed; acquiring multiple images (e.g., captured images P1 to P9) captured by the digital camera 27 in multiple states with different tilt angles of the head 21; and arranging these multiple images based on the tilt angle of the head 21 and displaying them on a display device 40, which is an example of a display unit.

[0065] As a result, multiple images with different tilt angles of the head 21 are arranged and displayed on the display device 40 based on the tilt angle of the head 21, allowing the user to grasp the entirety of the observation target S, especially in the low-magnification area, and to check for defects or malfunctions. Therefore, according to this embodiment, it is possible to easily check the observation target S by the user. Furthermore, in an observation device 10 having a configuration that tilts the head 21, multiple images can be arranged and displayed without adding any new configurations, thus preventing the observation device 10 from becoming complex or expensive.

[0066] Furthermore, in this embodiment, the control device 30 displays on the display device 40 an image (e.g., captured image P1) captured by the digital camera 27 when the head 21 is upright and not tilted, flanked by an captured image P2 (e.g., captured image P2) captured by the digital camera 27 when the head 21 is tilted to one side (e.g., the right side), and an captured image P3 (e.g., captured image P3) captured by the digital camera 27 when the head 21 is tilted to the other side (e.g., the left side), flanking the captured image P1 (e.g., captured image P1) captured by the digital camera 27 when the head 21 is tilted to one side (e.g., the right side). This makes it easier for the user to understand the positional relationship of which image was acquired from which direction of the observed object when the observed object is photographed from multiple directions.

[0067] Furthermore, in this embodiment, the observation device 10 further includes a rotatable stage 25 on which the object to be observed S is placed, and a rotating stage 252, which is an example of a rotation angle detection unit for detecting the rotation angle of the stage 25. The control device 30 displays on the display device 40 multiple images captured by the digital camera 27 in multiple states where the tilt angle of the head 21 is different for each of the multiple rotation angles of the stage 25, arranged based on the tilt angle of the head 21 and the rotation angle of the stage 25. This makes it easier for the user to intuitively grasp the entire object to be observed S.

[0068] Furthermore, in this embodiment, the control device 30 arranges the images on the display device 40 such that the arrangement direction differs for each rotation angle of the stage 25, with the image captured by the digital camera 27 in an upright position (e.g., captured image P1) flanked by the image captured by the digital camera 27 with the head 21 tilted to one side, and the image captured by the digital camera 27 with the head 21 tilted to the other side, with the image captured by the digital camera 27 on the opposite side. This makes it easier for the user to understand the positional relationship of each image.

[0069] Furthermore, in this embodiment, the control device 30 arranges multiple images (captured images P2, P3) captured when the stage 25 is rotated at a reference angle (initial angle) to the left and right of the upright image (captured image P1) so that the arrangement direction is left to right, and then arranges multiple images (captured images P8, P9) captured after rotating the stage 25 by ±90° from the reference angle (an example of ±R1°) above and below the upright image (captured image P1) so that the arrangement direction is up to down, and displays them on the display device 40. This makes it easier for the user to understand the positional relationship of each image.

[0070] Furthermore, in this embodiment, the control device 30 arranges multiple captured images P4 and P5, which are captured after rotating the stage 25 from the reference angle by +45° (an example of +R2° (the absolute value of R2 is smaller than the absolute value of R1)), to the upper right and lower left of the upright captured image P1, and displays multiple captured images P6 and P7, which are captured after rotating the stage 25 from the reference angle by -45° (an example of -R2°), to the lower right and upper left of the upright captured image P, on the display device 40. This makes it easier for the user to understand the positional relationship of each image.

[0071] Furthermore, in this embodiment, the control device 30 rotates the multiple images (captured images P4 to P9) taken after rotating the stage 25 from a reference angle, in the opposite direction to the rotation of the stage 25, by the rotation angle of the stage 25, and displays them on the display device 40. This makes it easier for the user to understand the positional relationship of each image.

[0072] Furthermore, in this embodiment, if the reference angle of the stage 25 when multiple images were captured before the stage 25 was rotated deviates from a predetermined angle (for example, the initial angle of 0°), the control device 30 rotates the multiple images captured before and after the stage 25 was rotated in the opposite direction by the rotation angle that deviated from the predetermined angle, and displays them on the display device 40. This allows the user to check the object of observation S in an orientation that is easy to view.

[0073] Furthermore, in this embodiment, the control device 30 determines multiple rotation angles of the stage 25 based on user specifications. This allows the user to check the object of observation from a desired direction.

[0074] Furthermore, in this embodiment, the control device 30 arranges multiple images (for example, captured images P1 to P9) and displays them on the display device 40 based on the user's display start operation. This allows the user to arrange multiple images at their desired time.

[0075] Furthermore, in this embodiment, the control device 30 displays multiple images on the display device 40 in an arrangement along with at least the tilt angle of the head 21. This allows the user to confirm the tilt angle of the head 21 when each image was captured, making it easier to understand the positional relationship of each image.

[0076] Furthermore, in this embodiment, the control device 30 stores multiple images (for example, captured images P1 to P9) as header information, including at least the tilt angle of the head 21 when multiple images are captured. This makes it easier for the user to confirm the observed object S when reviewing the stored images.

[0077] Furthermore, in this embodiment, the control device 30 stores multiple images (for example, captured images P1 to P9) with the coordinates of at least the stage 25 (for example, XY coordinates) as header information. This makes it easier for the user to confirm the observed object S when reviewing the stored images.

[0078] Furthermore, in this embodiment, the control device 30 saves the image selected by the user from among the multiple images arranged and displayed on the display device 40 (for example, the captured images P7 and P8 with a checkmark, as shown in Figure 8). This allows the user to reconfirm the selected image, such as the image in which a defect or malfunction was found.

[0079] Furthermore, in this embodiment, the control device 30 arranges images captured by the digital camera 27 with the head 21 tilted at multiple angles on one side (e.g., the right side) (e.g., captured images P2, P11) on one side (e.g., the right side) of the upright image (e.g., captured image P1), in order from the smallest tilt angle of the head 21 to the largest, so as to move away from the upright image (e.g., captured image P1). Images captured by the digital camera 27 with the head 21 tilted at multiple angles on the other side (e.g., the left side) (e.g., captured images P3, P12) on the opposite side (e.g., the left side) of the upright image (e.g., captured image P1), in order from the smallest tilt angle of the head 21 to the largest, so as to move away from the upright image (e.g., captured image P1). These images are then displayed on the display device 40. This allows the user to check the observation target S in the direction in which they wish to view the image.

[0080] Furthermore, in this embodiment, the control device 30 displays on the display device 40 images taken by the digital camera 27 at some angles when the head 21 is tilted to one side at multiple angles, and images taken by the digital camera 27 at some angles when the head 21 is tilted to the other side at multiple angles, in separate arrangements, and stores at least the images that are not displayed. This makes it easier for the user to confirm the observed object S when reviewing the stored images.

[0081] The above-described embodiment provides a concrete example to facilitate understanding of the invention. Therefore, the present invention is not limited to this embodiment and may include modified forms of this embodiment and alternative forms that replace the above-described embodiment. In other words, the components of this embodiment can be modified without departing from its spirit and scope. Furthermore, new embodiments can be implemented by appropriately combining the multiple components disclosed in this embodiment. In addition, some components may be deleted from the components shown in this embodiment, or some components may be added to the components shown in this embodiment. Moreover, the processing procedures shown in each embodiment may be performed in a different order, as long as they do not contradict each other. That is, the observation method, observation apparatus, and program of the present invention can be modified and changed in various ways without departing from the scope of the claims. [Explanation of symbols]

[0082] 10 Observation device 20 Microscopes 21 heads 22 Light source 23 Observation Optical System 231 Objective lens 24 Tilt section 241 Rotating shaft section 242 Z-axis holding part 25 stages 251 XY Stages 251x X-axis movable mechanism 251xa Ball Screw 251xb scale 251xc Scale Head 251xd motor 251xe handle 251xf connection part 251y Y-axis movable mechanism 251ya Handle 252 Rotating Stage 253 Z Stage 26 frames 27 Digital Cameras 271 Image Sensor 28 Control section 30 Control device 40 Display device 41 Display screen 41a Operation Buttons 41b Image display section 41c Preview image display section 41d Comment section 50 Input devices 100 Computers 101 Processors 102 memory 103 Storage device 104 Reading device 105 Storage medium 106 Communication Interface 107 Input / Output Interfaces 108 Bus A Observation optical axis C is the center of rotation. P1-P9, P11, P12 Acquired Images S Observation Subject

Claims

1. It has an imaging unit for capturing images of the object to be observed, and a tiltable head, A display unit that displays the image of the object to be observed captured by the imaging unit, A tilt angle detection unit detects the tilt angle, which is a parameter that includes the tilt direction of the head, A control unit that arranges multiple images captured by the imaging unit in multiple states with different tilt angles of the head, based on the tilt angle of the head, and displays them on the display unit; A rotatable stage on which the object to be observed is placed, The system includes a rotation angle detection unit that detects a rotation angle, which is a parameter including the rotation direction of the stage, The control unit arranges the multiple images captured by the imaging unit in multiple states where the tilt angle of the head is different for each of the multiple rotation angles of the stage, based on the tilt angle of the head and the rotation angle of the stage, and displays them on the display unit. An observation device characterized by the following features.

2. The control unit arranges the images captured by the imaging unit with the head tilted to one side on one side and the images captured by the imaging unit with the head tilted to the other side on the opposite side, and displays them on the display unit, flanking the image captured by the imaging unit with the head tilted to the other side. The observation apparatus according to feature 1.

3. The control unit displays the images captured by the imaging unit with the head tilted to one side on one side, and the images captured by the imaging unit with the head tilted to the other side on the opposite side, with the arrangement direction differing for each rotation angle of the stage, on the display unit. The observation apparatus according to feature 1.

4. The control unit arranges the plurality of images captured on the stage at the reference angle to the left and right of the upright image so that the arrangement direction is left-right, and then arranges the plurality of images captured after rotating the stage by ±R1° from the reference angle to the top and bottom of the upright image so that the arrangement direction is up and down, and displays them on the display unit. The observation apparatus according to claim 3.

5. The control unit arranges the plurality of images captured after rotating the stage by +R2° from the reference angle (the absolute value of R2 is smaller than the absolute value of R1) in the upper right and lower left corners of the upright image, and displays the plurality of images captured after rotating the stage by -R2° from the reference angle in the lower right and upper left corners of the upright image on the display unit. The observation apparatus according to feature 4.

6. The control unit rotates the plurality of images captured after the stage has been rotated from a reference angle, in the opposite direction to the direction in which the stage was rotated, by the rotation angle of the stage, and displays them on the display unit. The observation apparatus according to any one of claims 1 to 5.

7. If the control unit determines that the reference angle of the stage when the multiple images were captured before the stage was rotated deviated from a predetermined angle, it rotates the multiple images captured before and after the stage was rotated in the opposite direction by the rotation angle that deviated from the predetermined angle, and displays them on the display unit. The observation apparatus according to any one of claims 1 to 6, characterized in that

8. The control unit determines a plurality of rotation angles of the stage based on user specifications. The observation apparatus according to any one of claims 1 to 7, characterized by the following:

9. The control unit arranges the plurality of images and displays them on the display unit based on the user's display start operation. The observation apparatus according to any one of claims 1 to 8.

10. The control unit arranges the plurality of images along with the tilt angle of the head and displays them on the display unit. The observation apparatus according to any one of claims 1 to 9.

11. The control unit stores the plurality of images as header information, including at least the tilt angle of the head when the plurality of images are captured. The observation apparatus according to any one of claims 1 to 10.

12. The control unit stores the multiple images as header information, which includes at least the coordinates of the stage at the time the multiple images were captured. The observation apparatus according to any one of claims 1 to 11.

13. The control unit stores the image selected by the user from among the multiple images arranged and displayed on the display unit. The observation apparatus according to any one of claims 1 to 12.

14. The control unit arranges the images captured by the imaging unit with the head tilted at multiple angles on one side so that they are further away from the upright image, in order from the smallest tilt angle of the head to the largest, and displays the images captured by the imaging unit with the head tilted at multiple angles on the other side so that they are further away from the upright image, in order from the smallest tilt angle of the head to the largest, on the opposite side of the upright image, and displays them on the display unit. The observation apparatus according to claim 2 or 3, characterized by the features described herein.

15. The control unit arranges and displays on the display unit images captured by the imaging unit at some angles when the head is tilted at multiple angles on one side, and images captured by the imaging unit at some angles when the head is tilted at multiple angles on the other side, and stores at least the images that are not displayed. The observation apparatus according to claim 2 or 3, characterized by the features described herein.

16. The tilt angle, which is a parameter including the tilt direction of a tiltable head having an imaging unit that images the object to be observed, is acquired. Multiple images captured by the imaging unit are acquired in multiple states with different tilt angles of the head. The rotation angle, which is a parameter including the rotation direction of the rotatable stage on which the object to be observed is placed, is detected. For each of the multiple rotation angles of the stage, the multiple images captured by the imaging unit in multiple states where the tilt angle of the head is different are arranged and displayed on the display unit based on the tilt angle of the head and the rotation angle of the stage. An observation method characterized by the following.

17. A function to acquire the tilt angle, which is a parameter including the tilt direction of a tiltable head having an imaging unit that images the object to be observed, A function to acquire multiple images captured by the imaging unit in multiple states with different tilt angles of the head, A function to detect the rotation angle, which is a parameter including the rotation direction of the rotatable stage on which the object to be observed is placed, The function includes displaying the multiple images captured by the imaging unit in multiple states where the tilt angle of the head is different for each of the multiple rotation angles of the stage, arranged on the display unit based on the tilt angle of the head and the rotation angle of the stage. A program characterized by causing a computer to execute something.

Images