Observation method, observation apparatus, and program

Abstract

To provide an observation method, observation device and program which allow a user to easily confirm the outer peripheral surface of an observation object.SOLUTION: An observation method comprises the steps of: imaging an observation object S placed on a rotatable stage 25 from a direction different from the rotation center C of the stage 25 at each of a plurality of positions having different rotation angles of the stage 25; acquiring the rotation angle of the stage 25 in each of a plurality of images (for example, photographed images P1-P4) captured at the plurality of positions; and displaying the plurality of images on a display unit (for example, a display device 40) in association with the rotation angle of the corresponding stage 25.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The disclosure of the present specification relates to an observation method, an observation apparatus, and a program. [Background Art]

[0002] Conventionally, a magnified observation apparatus is known, in which a head unit mounted on a head tilting mechanism can swing about a swing shaft, and a stage rotates an observation object (see, for example, Patent Document 1).

[0003] Furthermore, a defect inspection apparatus has been proposed, in which an imaging unit repeatedly captures one-dimensional images of a side surface of a cylindrical inspection object rotating at a predetermined speed, and a control unit combines these one-dimensional images to create a side developed image (inspection image) for one full circumference of the inspection object as a multi-value image (see, for example, Patent Document 2). [Prior Art Documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Unexamined Patent Publication No. 2015-127777 [Patent Document 2] Japanese Patent No. 6595800 [Summary of the Invention] [Problem to be Solved by the Invention]

[0005] By the way, when a head having an imaging unit that images an observation target is tiltable, the outer peripheral surface of the observation target can be observed. Furthermore, when a stage on which the observation target is placed is rotatable, any position on the outer peripheral surface of the observation target can be observed. As a result, for example, for an easily rollable observation target such as a cylindrical shape, any position on the outer peripheral surface of the observation target can be observed without laying the observation target sideways and holding it with a V-shaped block or the like. Therefore, it is possible to avoid shortening the stroke of the head by the size of the block.

[0006] However, if an image of an arbitrary position on the outer surface of the object being observed is displayed on the screen, the user will have to check the image multiple times when searching for a defect on the outer surface of the object being observed. As a result, it becomes difficult to intuitively grasp the outer surface of the object being observed.

[0007] Furthermore, if the outer surface of the object being observed is curved, the display surface of the display unit will be planar, making it particularly difficult to intuitively grasp the outer surface of the object being observed.

[0008] Based on the circumstances described above, one aspect of the present invention is to provide an observation method, observation apparatus, and program that facilitate the user's confirmation of the outer surface of an object to be observed. [Means for solving the problem]

[0009] In one embodiment, the observation method includes the steps of: imaging an object to be observed, placed on a rotatable stage, from a direction different from the center of rotation of the stage, at each of a plurality of positions where the rotation angle of the stage is different; obtaining the rotation angle of the stage for each of the plurality of images taken at the plurality of positions; and displaying the plurality of images on a display unit in association with the corresponding rotation angle of the stage.

[0010] In another embodiment, the observation device comprises a rotatable stage on which an object to be observed is placed; an imaging unit that images the object to be observed from a direction different from the center of rotation of the stage at each of a plurality of positions with different rotation angles of the stage; a display unit that displays the images of the object to be observed captured by the imaging unit; a rotation angle detection unit that detects the rotation angle of the stage in each of the plurality of images captured at the plurality of positions; and a control unit that displays the plurality of images on the display unit in association with the corresponding rotation angles of the stage.

[0011] In another embodiment, the program causes the computer to perform the following functions: acquire images of an object placed on a rotatable stage, taken from a direction different from the center of rotation of the stage, at each of a plurality of positions where the rotation angle of the stage is different; acquire the rotation angle of the stage in each of the plurality of images taken at the plurality of positions; and display the plurality of images on a display unit in association with the corresponding rotation angle of the stage. [Effects of the Invention]

[0012] According to the above embodiment, the user can easily confirm the outer surface of the object being observed. [Brief explanation of the drawing]

[0013] [Figure 1] This is a right side view showing the configuration of an observation device according to one embodiment. [Figure 2] This is a right side view showing a part of an observation device according to one embodiment. [Figure 3] This is a front view showing a part of an observation device according to one embodiment. [Figure 4] This is a front view showing the XY stage and the rotary stage in one embodiment. [Figure 5] This figure shows the display screen in one embodiment. [Figure 6] This is a flowchart illustrating an observation method according to one embodiment. [Figure 7] This is an explanatory diagram illustrating the captured image and stitched image in one embodiment. [Figure 8] This figure shows an example of the display content of the display screen in one embodiment. [Figure 9] This is a plan view illustrating the imaging of an object to be observed in one embodiment. [Figure 10] This is a plan view illustrating the shape of the object being observed in one embodiment. [Figure 11] This is a plan view illustrating the shape of the object to be observed and the imaging method in one embodiment. [Figure 12] It is a plan view for explaining an imaging method of an observation object in one embodiment. [Figure 13] It is a diagram showing a relationship between contrast and Z position for explaining focusing position discrimination in one embodiment. [Figure 14] It is a diagram illustrating an example of the hardware configuration of a computer. Mode for Carrying Out the Invention

[0014] Hereinafter, an observation method, an observation apparatus, and a program according to an embodiment of the present invention will be described with reference to the drawings.

[0015] FIG. 1 is a right side view showing the configuration of an observation apparatus 10 according to one embodiment.

[0016] Note that, the X-axis direction, Y-axis direction, and Z-axis direction shown in FIG. 1, FIGS. 2 to 4 described later, and FIGS. 9 to 12 are, for example, horizontal directions in which the X-axis direction and the Y-axis direction are orthogonal to each other, and the Z-axis direction is a vertical direction.

[0017] The observation apparatus 10 shown in FIG. 1 includes a microscope 20, a control device 30 that controls the microscope 20, a display device 40, and an input device 50. Note that in a case where the microscope 20 is a digital microscope or the like that includes a display unit (display device 40) and a control unit (control device 30), the microscope 20 itself can function as the observation apparatus.

[0018] The microscope 20 includes a head 21, a light source 22, an observation optical system 23, a tilt unit 24, a stage 25, a frame 26, a digital camera 27, and an operation unit 28. The light source 22 and the digital camera 27 are provided inside the head 21 together with an optical path splitting element (splitter) such as a half mirror, a dichroic mirror, or a polarizing beam splitter. The light source 22 is disposed on one of the optical paths branched by the splitter, and the digital camera 27 is disposed on the other optical path.

[0019] The head 21 is held by the frame 26 via the tilt section 24. The head 21 is an example of a focusing unit that changes the distance between the observation target (sample) S and the observation optical system 23, and is movable parallel to the optical axis of the objective lens 231 (observation optical axis A shown in Figure 3) while held by the tilt section 24. The head 21, together with the tilt section 24 held by the frame 26, rotates (tilts) as shown in Figure 3, about the rotation axis section 241 shown in Figure 2 as its center (Y-axis direction). The center of the rotation axis section 241 preferably extends through the upper surface of the stage 25 or the observation target S or its vicinity, enabling eucentric observation. The rotation axis section 241 is preferably fixed to the frame 26 and held by the bearing of the tilt section 24. The rotation axis section 241 may also have a scale head that reads the position information of a scale provided on the frame 26. This scale head detects the rotation angle of the rotation axis section 241 by reading the position information of the scale on the frame 26. Furthermore, the scale head is preferably a rotary encoder, for example. Thus, the microscope 20 is equipped with a scale head (rotary encoder), which means that it is equipped with a tilt angle detection unit that detects the tilt angle of the head 21 (and tilt unit 24).

[0020] The movement of the head 21 in the direction of the observation optical axis A (see Figure 3) can be performed manually, for example, but may also be controlled by the control device 30. When the head 21 moves in the direction of the observation optical axis A, the control device 30 detects the position of the head 21 in the direction of the observation optical axis A (Z-axis direction) by reading the position information of the linear scale of the Z-axis holding part 242 of the tilt part 24 using a scale head fixed to the head 21. Note that the direction of movement of the head 21, the direction of the observation optical axis A, is the Z-axis direction (vertical direction) when the head 21 is upright, but when the head 21 is tilted, it becomes a direction inclined with respect to the Z-axis direction.

[0021] The tilt of the head 21 is performed manually by the user. The tilt angle of the head 21 is, for example, within a range of plus or minus 90° from the upright position.

[0022] Light source 22 is a light source for incident illumination, and is, for example, a white LED. Light source 22 may be other light sources such as a xenon lamp or a halogen lamp. The illumination light emitted from light source 22 is collimated by an illumination lens. The collimated illumination light is reflected by a splitter and irradiated onto the object of observation S placed on the stage 25 via the objective lens 231.

[0023] The observation optical system 23 projects an optical image of the object being observed S, illuminated by illumination light, onto the digital camera 27. The observation optical system 23 includes, for example, an objective lens 231 and an imaging lens. The objective lens 231 is switchable via a revolving nosepiece. Multiple objective lenses of different magnifications may be mounted on the revolving nosepiece, and any objective lens from among the multiple objective lenses may be positioned on the observation optical axis A according to the user's selection.

[0024] The digital camera 27 is an example of an imaging unit that images the object of observation S. The digital camera 27 includes an image sensor 271. This image sensor 271 is, for example, an image sensor such as a CCD image sensor or a CMOS image sensor. The digital camera 27 images the object of observation and outputs the image of the object of observation S to the control device 30. As a result, the control device 30 functions as an acquisition unit that acquires the captured image.

[0025] The stage 25 on which the object of observation S is placed includes an XY stage 251, a rotation stage 252, and a Z stage 253.

[0026] As shown in Figure 4, the XY stage 251 includes an X-axis movable mechanism 251x and a Y-axis movable mechanism 251y. The X-axis movable mechanism 251x includes a feed mechanism such as a ball screw 251xa for moving the mounting plate of the object to be observed S in the X-axis direction, an X-coordinate detection mechanism such as a scale 251xb and a scale head 251xc, an actuator such as a motor 251xd, a handle 251xe for manual operation, and a connecting portion 251xf with a screw (not shown) for fixing the X-axis movable mechanism 251x. The Y-axis movable mechanism 251y also includes a feed mechanism such as a ball screw for moving the mounting plate of the object to be observed S in the Y-axis direction, a Y-coordinate detection mechanism such as a scale and a scale head, and an actuator such as a motor, although only a handle 251ya for manual operation is shown.

[0027] The XY stage 251 is, for example, an electric stage, and its position in the XY direction is controlled by the control device 30. The XY stage 251 is mounted on the rotary stage 252. In the example shown in Figure 4, a motor 251xd and handles 251xe, 251ya are provided for both the X-axis movable mechanism 251x and the Y-axis movable mechanism 251y, respectively, allowing for movement by both electric and manual means. However, one of the motor 251xd and the handles 251xe, 251ya may be omitted, allowing for movement by either electric or manual means only.

[0028] As shown in Figure 3, the rotating stage 252 rotates the observation target S by rotating the XY stage 251 around a rotation center C that extends vertically. The rotation center C of the rotating stage 252 coincides with the observation optical axis A of the upright head 21. The rotating stage 252 functions as an example of a rotation angle detection unit that detects the rotation angle of the stage 25 by having, for example, a scale scale head (e.g., a rotary encoder) that detects the rotation angle of the stage 25. The rotating stage 252 also includes an actuator such as a motor. The rotation angle detection unit that detects the rotation angle of the stage 25 may be a member that directly detects the rotation angle, such as a rotary encoder or a magnetic sensor, or it may be the control device 30 itself, for example, when the control device 30 counts the number of steps of a stepping motor or detects the orientation of a mark (indicator: e.g., a single line) on the stage 25.

[0029] The Z-stage 253 is an example of a focusing unit that changes the distance between the observation target S and the observation optical system 23. The Z-stage 253 is, for example, an electric stage or a manual stage, and in the case of a manual stage, its vertical position (Z-direction) can be adjusted with a handle (not shown). Note that the XY stage 251, the rotation stage 252, and the Z-stage 253 are not limited to this example and may each be an electric stage or a manual stage.

[0030] The control unit 28 shown in Figure 1 receives various operational information from the user regarding the operation of the microscope 20, and sends this input information to the control device 30, which will be described later. For example, the control unit 28 may have at least one of the following for operating the stage 25 and the head 21: a handle, switch, button, mouse, keyboard, joystick, dial, touch panel, etc.

[0031] The control device 30 is a device that controls the microscope 20 and is an example of a control unit of the observation device 10. The control device 30 may be composed of multiple devices. For example, the control device 30 may include a microscope controller that is mainly responsible for controlling the operation of the motorized part of the microscope 20 and a general-purpose computer that is mainly responsible for image processing of images acquired by the microscope 20.

[0032] The display device 40 is an example of the display unit of the observation device 10, and is any display such as a liquid crystal display or an organic EL display.

[0033] The display device 40 displays, for example, the display screen 41 shown in Figure 5. The contents of this display screen 41 will be described later, but the display screen 41 may have multiple operation buttons 41a such as an image acquisition button and an image save button, a preview display unit 41b that displays the acquired image, a live image display unit 41c that displays the current observation area, and a map image display unit 41d that displays a map image created by stitching together the acquired images. The operation buttons 41a are provided in the display area of ​​the display screen 41 and are selected by touch panel operation or cursor operation using the input device 50, but they may also be provided in an area other than the display area and pressed. Alternatively, the operation buttons 41a may be omitted, and operations similar to those using the operation buttons 41a may be performed on the operation unit 28 of the microscope 20, the input device 50, etc.

[0034] The input device 50 is any input device such as a keyboard, mouse, touchpad, or joystick.

[0035] Next, the observation method according to this embodiment will be described with reference to Figure 6.

[0036] First, the control device 30 accepts the setting of the outer perimeter observation mode (step S1). This outer perimeter observation mode, as will be described later, is a mode in which multiple images taken with the head 21 tilted (the observation optical axis A of the head 21 is not parallel to the rotation center C of the stage 25) at multiple positions with different rotation angles of the stage 25 are displayed on the display device 40 in relation to the rotation angle of the stage 25 (a mode in which displays such as the preview image display process S11 described later).

[0037] The outer perimeter observation mode may be set based on a manual operation by the user, for example, by selecting the setting button on the operation button 41a of the display screen 41. Alternatively, the outer perimeter observation mode may be set automatically or the user may be notified to prompt the user to perform the setting when any or more of the following conditions are met: the head 21 is tilted (as in step S4 described later, the observation optical axis A of the head 21 is tilted relative to the rotation center C of the stage 25) and tilts by more than a threshold (e.g., 90°); the head 21 is tilted and the stage 25 (rotating stage 252) is rotated; the observation target S is detected to have a curved surface (e.g., with the head 21 tilted); the height of the observation target S is detected to be above a threshold when creating a 3D model; the head 21 is upright and the digital camera 27 determines that the perimeter of the observation target S is circular (including perfect circles and ellipses); or the center of the observation target S coincides with the rotation center C of the stage 25. Furthermore, notifications to the user may be provided, for example, by display on the display device 40 or by audio output from an audio output unit (not shown).

[0038] Next, the control device 30 determines whether the microscope 20 is upright, that is, whether the inclination of the observation optical axis A of the head 21 with respect to the rotation center C of the stage 25 is 0°. If it is not upright, the control device 30 notifies the user to manually set the microscope 20 to an upright position (step S2). The control device 30 also identifies the shape of the periphery of the object to be observed S and identifies the height of the object to be observed S when creating a 3D model, based on the image captured by the digital camera 27. If the control device 30 does not perform these identifications, the process in step S2 can be omitted.

[0039] Next, the control device 30 notifies the user to align the center of the object to be observed S with the rotation center C of the stage 25 (step S3). The user, having been notified to align the center of the object to be observed S with the rotation center C of the stage 25, may move the object to be observed S based on, for example, the "+" mark, grid lines, or auxiliary lines placed on the stage 25. Alternatively, the control device 30 may move the XY stage 251 of the stage 25 in the XY direction to align the center of the object to be observed S with the rotation center C of the stage 25.

[0040] Next, the control device 30 notifies the user to manually tilt the head 21 (90° in this case) if the head 21 is not tilted, that is, to tilt the observation optical axis A of the head 21 with respect to the rotation center C of the stage 25 (step S3). The control device 30 also acquires the tilt angle of the observation optical axis A (head 21) detected by, for example, the scale head of the rotation axis section 241.

[0041] With the head 21 tilted, the control device 30 determines, based on the image acquired from the digital camera 27, whether the observation optical axis A is perpendicular to at least a portion of the imaged portion of the object to be observed S (for example, a surface if it is a prism shape, or a part of a surface if it is a cylinder) (step S5).

[0042] For example, if the shape of the object to be observed S is a cone with its vertex located at the top, and the head 21 is tilted 90°, the control device 30 determines that the observation optical axis A is not perpendicular to the imaged portion of the object to be observed S (step S5: NO). In this case, the process returns to step S4 described above, but the control device 30 may inform the user of the tilt angle of the observation optical axis A that would make it perpendicular to the imaged portion, based on the image captured by the digital camera 27. Whether or not the observation optical axis A is perpendicular to the imaged portion of the object to be observed S can be determined based on a certain area in the captured image, or a threshold can be set so that even if there is a deviation from perpendicular, if the deviation is within the threshold range, it can be considered perpendicular (step S5: YES) and the process can proceed to S6.

[0043] For example, when the observation target S has a cylindrical shape with a central axis extending vertically with respect to the stage XY plane, and the head 21 is tilted 90°, the control device 30 determines, based on the image captured by the digital camera 27, that the observation optical axis A is perpendicular to the portion of the observation target S that is being captured (Step S5: YES).

[0044] In this case, the control device 30 determines whether the head 21 is in the focused position (step S6). For example, the control device 30 can look at the contrast value of a certain pixel (area) and determine that the Z position (position 3) at the peak value with the highest contrast value among multiple positions (e.g., positions 1, 2, 3, and 4) with different Z positions (positions in the direction of the observation optical axis A) shown in Figure 13 is the focused position. By performing this on all or some of the pixels (areas) (e.g., only the area of ​​interest) and moving the head 21 to the Z position, a full-focus image or a 3D image can be generated by a known method. When generating a full-focus image or a 3D image in this way, the roughness of the outer surface of the object to be observed S may be measured. Note that the step of determining whether the head 21 is in the focused position (step S6) and the focus position adjustment step (step S7) described later can be omitted if it has been determined in advance that the peripheral shape of the object to be observed S is perfectly circular in the upright state of the head 21. On the other hand, if the object being observed S is elliptical in shape, it is advisable to perform a determination step (step S6) to check whether the head 21 is in focus at each rotation angle of the stage 25.

[0045] If the head 21 is not in the focused position (Step 6: NO), the control device 30 adjusts the head 21 to the focused position by moving it in the direction of the observation optical axis A relative to the tilt unit 24, or by moving the observation target S with the stage 25 (Step S7).

[0046] If the head 21 is in the focus position (Step 6: YES) or after the head 21 has been set to the focus position (Step S7), the control device 30 acquires the image captured by the digital camera 27 (Step S8). The control device 30 also acquires and stores the rotation angle of the stage 25 at the time of acquisition for each of the captured images.

[0047] Unlike the observation target S shown in Figure 9, in the case of an observation target S-1 which is cylindrical with irregularities on its outer surface as shown in Figure 10, the head 21 in the upright position can draw a virtual circle from the coordinates of three points (dashed line), and the irregularities can be identified based on this virtual circle. At least when imaging this irregular portion, the observation target S-1 may be imaged multiple times by the digital camera 27 at different focal points while the rotation angle of the stage 25 is the same. In this case, a composite image (all-focus image) obtained by combining images of the observation target S taken at the same position but at different focal points can be used to arrange or stitch together multiple images as described later and display them on the display device 40.

[0048] Furthermore, as with the observation target S-2 shown in Figure 11, if only a portion of the outer surface has irregularities (protrusions in Figure 11), a composite image can be created by taking multiple images of only these irregularities (for example, at the three locations indicated by the vertical lines in Figure 12). If only other parts (for example, areas without irregularities or areas where the irregularity height (the difference in height between the recess and the protrusion) is below a specified threshold) are to be imaged, then the observation target S may be imaged only once at the same rotation angle of the stage 25.

[0049] Image acquisition by the digital camera 27 can be performed manually or automatically. When image acquisition is performed manually, the control device 30 may inform the user of the recommended rotation angles of the stage 25 for imaging (e.g., 0°, 90°, 180°, and 270°). When image acquisition is performed automatically, the control device 30 controls the digital camera 27 to automatically image the object of observation S when the stage 25 is rotated manually or automatically to a predetermined rotation angle, or at predetermined intervals during the period when the stage 25 is rotating manually or automatically. When image acquisition is performed by the digital camera 27, it is advisable to temporarily lock the rotation of the stage 25. If the control device 30 has identified the shape of the periphery of the object of observation S in step 2 described above, it can determine the rotation angle of the stage 25 during imaging based on this periphery shape. For example, if the object S has a cylindrical shape with a central axis extending vertically, and the entire side surface of the object S is to be imaged, it is advisable to set the rotation angle of the stage 25 at equal intervals such as 0°, 90°, 180°, and 270° during imaging. On the other hand, if only a portion of the area of ​​interest of the object S is to be imaged, it is advisable to set the rotation angle of the stage 25 to one or more rotation angles corresponding to the area of ​​interest. Note that if the object S has a cylindrical shape with a small diameter, the imaged portion of the object S may be nearly half of the outer surface, and the entire imaged portion will not be perpendicular to the observation optical axis A. Therefore, imaging may be performed so that the rotation angle of the stage 25 is at intervals shorter than the aforementioned 90°. Furthermore, if the object being observed S has a triangular prism shape with a central axis extending vertically, the rotation angle of the stage 25 during imaging can be set to 0°, 120°, and 240° in 120-degree increments, with the stage rotation angle where the observation optical axis A is perpendicular to one face of the object being observed S being 0°. This allows imaging of the entire side surface and enables the creation of stitched images. The recommended stage rotation angle should be set appropriately according to the shape of the object being observed S. However, imaging may be performed at the user's desired rotation angle.

[0050] The control device 30 determines whether the stage 25 has taken images at all target rotation angles (step S9). If the imaging is incomplete (step S9: NO), it rotates the stage 25 to the next rotation angle (step S10) and returns to step S6. Note that the digital camera 27 may also image the object S at the same focus based on the shape of the periphery of the object S identified by the upright head 21 as described above (for example, a cylinder). In this case, the focus adjustment in step S6 is omitted, and the process proceeds to imaging by the digital camera 27 (step S8).

[0051] If imaging is complete (step S9: YES), the control device 30 displays the image captured by the digital camera 27 on the display device 40 (step S11). This image display step (step S11) may be performed before the end of imaging. The end of imaging may be determined based on any timing decided by the user (for example, when the preview image display button of the operation button 41a is selected or when input indicating the end of imaging is received).

[0052] Here, as shown in Figure 7, we will explain using the example of imaging performed over the entire circumference of a cylindrical object S when the rotation angles of the stage 25 are 0°, 90°, 180°, and 270°. Eight letters, A to H, are printed on the outer surface of this object S, each at 45° intervals. The image P1 captured when the rotation angle of the stage 25 is 0° shows the four letters A, B, C, D; the image P2 captured at 90° shows the four letters CDEF; the image P3 captured at 180° shows the four letters EFGH; and the image P4 captured at 270° shows the four letters GHAB.

[0053] The control device 30 displays each image P1 to P4 on the preview display unit 41b of the display screen 41 shown in Figure 8 (step S11). The displayed images P1 to P4 may be saved automatically, but for example, if one or more captured images P1, P2 are selected by checking the "✓" box on the preview display unit 41b using touch panel operation or cursor operation, the save button on the operation button 41a may be selected to save them. In this case, it is preferable that the captured images P1, P2 be saved along with information such as the rotation angle of the stage 25 and the tilt angle of the head 21.

[0054] As shown in Figures 7 and 8, when multiple captured images P1 to P4 are displayed from left to right in the order of 0°, 90°, 180°, and 270°, it can be said that the multiple captured images P1 to P4 are arranged and displayed on the display device 40 (display screen 41) according to the positional relationship of the captured portion of the observation target S contained in each of the multiple captured images P1 to P4. The control device 30 displays the multiple captured images P1 to P4 on the display device 40 in association with the corresponding rotation angles of the stage 25. Alternatively, an index representing the rotation angle of the stage 25, such as a scale bar, may be displayed, and the captured images P1 to P4 may be displayed at the scale bar position corresponding to the acquired angle, or the captured images P1 to P4 may be displayed when the index is selected. The control device 30 may also display the corresponding rotation angle of the stage 25 within or around the captured images P1 to P4. In this case, even if the multiple captured images P1 to P4 are not arranged according to the positional relationship of the captured portion of the object S being observed, the control device 30 can be said to be displaying the multiple captured images P1 to P4 on the display device 40 in relation to the rotation angle of the corresponding stage 25. Thus, in this embodiment, instead of arranging the multiple captured images P1 to P4 in the order in which they were taken, displaying them on the display device 40 in relation to the rotation angle of the stage 25 allows for an intuitive understanding of the positional relationship between the images.

[0055] Furthermore, the control device 30 may, for example, based on the selection of the stitching button on the operation button 41a, display a stitched image P10 on the display device 40 (display screen 41) by stitching together the multiple captured images P1 to P4, as shown in Figure 7, according to the positional relationship of the captured portion of the observation target S contained in each of the multiple captured images P1 to P4. In this case as well, it can be said that the control device 30 is displaying the multiple captured images P1 to P4 on the display device 40 in relation to the rotation angles of the corresponding stage 25.

[0056] When the control device 30 stitches together multiple captured images P1 to P4, it is preferable to create a stitched image P10 by, for example, image matching using the coordinates of the stage 25, the rotation angle of the stage 25, the Z position coordinates of the head 21, etc.

[0057] The control device 30 may, for example, create a stitched image P10 by adjusting the brightness values ​​of the overlapping portions created by stitching together multiple captured images P1 to P4, based on the selection of the brightness adjustment button on the operation button 41a. For example, if the right edge of captured image P1 has a brightness value of 20%, and the central portion of captured image P2 that overlaps with the right edge of captured image P1 has a brightness value of 80%, the brightness may be averaged to 50%, or the brighter one may be selected and adjusted to 80%, or it may be adjusted to a predetermined brightness value (for example, always 50%). In addition, the brightness values ​​of the overlapping portions may be adjusted based on the peripheral shape of the observation target S identified by the upright head 21. Here, unlike when captured images P1 to P4 are arranged individually, the stitched image P10 can omit the overlapping portions of multiple images with different rotation angles of the stage 25, so for example, the observation target S may be imaged at short intervals such as 10° and 15° of rotation angle of the stage 25. Furthermore, if the object of observation S has a conical shape and each captured image contains a vertex of the object of observation S, the stitched image P10 should be a stitched image that resembles a unfolded diagram with the vertices coinciding.

[0058] As shown in Figure 8, the stitched image P10 is displayed on the map image display unit 41d. The control device 30 may display not only the stitched image P10, but also an enlarged image P20, which is an enlarged version of the selected area 41e of the stitched image P10, selected by touch panel operation or cursor operation, on the display device 40 (live image display unit 41c).

[0059] Furthermore, whether the captured images P1 to P4 are arranged and displayed on the display device 40, or the stitched image P10 is displayed on the display device 40, the display may be performed using not only multiple images with different rotation angles of the stage 25, but also multiple images with different positions in the Z-axis direction of the stage 25. For example, multiple images with the same Z-axis position of the stage 25 but different rotation angles of the stage 25 may be arranged horizontally, and images with different Z-axis positions of the stage 25 may be placed above and below the images with the same rotation angle of the stage 25, with the Z-axis position corresponding to the rotation angle of the stage 25, or they may be stitched together and displayed on the screen.

[0060] Figure 14 is a diagram illustrating the hardware configuration of a computer 100 for realizing the control device 30 according to the above-described embodiment.

[0061] As shown in Figure 14, the computer 100 comprises a processor 101, memory 102, storage device 103, reader 104, communication interface 106, and input / output interface 107 as its hardware configuration. The processor 101, memory 102, storage device 103, reader 104, communication interface 106, and input / output interface 107 are connected to each other, for example, via a bus 108.

[0062] The processor 101 may be a single processor, a multi-processor, or a multi-core processor. The processor 101 operates as the control unit of the observation device 10 by reading and executing the program stored in the storage device 103.

[0063] Memory 102 is, for example, a semiconductor memory and may include a RAM area and a ROM area. Storage device 103 is, for example, a semiconductor memory such as a hard disk or flash memory, or an external storage device.

[0064] The reader 104 accesses the removable storage medium 105, for example, according to instructions from the processor 101. The removable storage medium 105 can be implemented by, for example, a semiconductor device, a medium through which information is input / output by magnetic action, or a medium through which information is input / output by optical action. A semiconductor device is, for example, a USB (Universal Serial Bus) memory. A medium through which information is input / output by magnetic action is, for example, a magnetic disk. A medium through which information is input / output by optical action is, for example, a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disk), a Blu-ray Disc, etc. (Blu-ray is a registered trademark).

[0065] The communication interface 106 communicates with other devices, for example, according to instructions from the processor 101. The input / output interface 107 is, for example, an interface between an input device and an output device. The input device is, for example, a device such as a keyboard, mouse, or touch panel that receives instructions from the user. The output device is, for example, a display device such as a display, and an audio device such as a speaker.

[0066] The program executed by the processor 101 is provided to the computer 100 in the following form, for example. (1) It is pre-installed on the storage device 103. (2) Provided by a removable storage medium 105. (3) Provided from a server such as a program server.

[0067] The hardware configuration of the computer 100 for realizing the control device 30, as described with reference to Figure 14, is illustrative and the embodiment is not limited thereto. For example, some of the above configuration may be deleted, or new configurations may be added. In another embodiment, for example, some or all of the functions of the calculation unit 42 described above may be implemented as hardware such as an FPGA (Field Programmable Gate Array), SoC (System-on-a-Chip), ASIC (Application Specific Integrated Circuit), and PLD (Programmable Logic Device). That is, any electrical circuit included in the control device 30 may perform the internal prediction processing described above.

[0068] In the embodiment described above, the observation method performed by the observation device 10, for example, includes the steps of: capturing an observation target S placed on a rotatable stage 25 from a direction different from the rotation center C of the stage 25 at each of a plurality of positions with different rotation angles of the stage 25; acquiring the rotation angle of the stage 25 in each of the plurality of images (captured images P1 to P4) captured at the plurality of positions; and displaying the plurality of images in association with the corresponding rotation angles of the stage 25, for example, as captured images P1 to P4 or a stitched image P10, on a display device 40, which is an example of a display unit.

[0069] From another perspective, the observation device 10 comprises a rotatable stage 25 on which the object to be observed S is placed; a digital camera 27, which is an example of an imaging unit, that images the object to be observed S from a direction different from the rotation center C of the stage 25 at each of a plurality of positions of the stage 25 with different rotation angles; a display device 40, which is an example of a display unit, that displays the image of the object to be observed S captured by the digital camera 27; a rotation angle detection unit (e.g., a rotating stage 252) that detects the rotation angle of the stage 25 in each of a plurality of images (captured images P1 to P4) captured at a plurality of positions of the stage 25 with different rotation angles; and a control device 30, which is an example of a control unit, that displays the plurality of images on the display device 40 in association with the corresponding rotation angles of the stage 25.

[0070] From another perspective, the program causes the computer 100 (for example, the control device 30) to perform the following functions: acquire images of the object of observation S, which is placed on a rotatable stage 25, at each of several positions where the rotation angle of the stage 25 is different, and from a direction different from the rotation center C of the stage 25; acquire the rotation angle of the stage 25 for each of the multiple images (captured images P1 to P4) taken at the multiple positions; and display the multiple images on a display device 40, which is an example of a display unit, in association with the corresponding rotation angles of the stage 25.

[0071] As a result, multiple images with different rotation angles of the stage 25, related to the rotation angle of the stage 25, are displayed on the display device 40. This allows the user to intuitively grasp the outer surface of the object being observed S, especially in the low-magnification region, and to identify defects and malfunctions. Therefore, according to this embodiment, it is possible for the user to easily check the outer surface of the object being observed S.

[0072] Furthermore, in this embodiment, in the step of displaying multiple images, the multiple images are arranged and displayed on the display device 40 according to the positional relationship of the captured portion of the observation target S contained in each of the multiple images (for example, the preview display unit 41b in Figure 8). This makes it easier for the user to understand the positional relationship of the captured object within each image.

[0073] Furthermore, in this embodiment, during the process of displaying multiple images, a composite image P10 (see map image display unit 41d in Figure 8) is created by stitching together the multiple images, according to the positional relationship of the captured portion of the observation target S contained in each of the multiple images, and this composite image is displayed on the display device 40. This allows the user to perceive the observation target S appearing in each image as a single unit.

[0074] Furthermore, in this embodiment, during the process of displaying multiple images, the display device 40 displays a composite image P10 and an enlarged image P20 (see live image display unit 41c in Figure 8) which is an enlarged version of a selected area 41e arbitrarily selected from the composite image P10. This allows the user to perceive the observation target S appearing in each image as a whole while confirming the details of the observation target S.

[0075] Furthermore, in this embodiment, during the process of displaying multiple images, the brightness values ​​of the overlapping portions resulting from the stitching of multiple images are adjusted and displayed on the display device 40. This makes it easier for the user to confirm the stitched image P10.

[0076] Furthermore, in this embodiment, the observation method either executes a step of displaying multiple images (for example, the preview image display step S11 in Figure 6) when predetermined conditions are met, or notifies the user to set the execution of this step. When predetermined conditions are met, for example, the observation optical axis A of the head 21, which has a digital camera 27, an example of an imaging unit that images the object of observation S, is tilted by a threshold or more with respect to the rotation center C of the stage 25; the observation optical axis A of the head 21 is tilted with respect to the rotation center C of the stage 25 and the stage 25 is rotated; the head 21 detects that the object of observation S has a curved surface using the digital camera 27 while the observation optical axis A is tilted with respect to the rotation center C of the stage 25; the digital camera 27 detects that the height of the object of observation S is above a threshold; or the observation optical axis A of the head 21 is parallel to the rotation center C of the stage 25 and the digital camera 27 determines that the periphery of the object of observation S is circular, and the center of the object of observation S coincides with the rotation center C of the stage. As a result, when certain conditions are met, such as when the head 21 is tilted, the user can view images captured at multiple positions with different rotation angles of the stage 25, and images associated with the rotation angle of the stage 25, on the display device 40 without having to consciously perform operations such as selecting the preview image display button on the operation button 41a.

[0077] Furthermore, in this embodiment, during the process of imaging the object of observation S, the object of observation S is automatically imaged when the stage 25 rotates to a predetermined rotation angle. This eliminates the need for the user to manually rotate the stage 25 and stop it at a predetermined rotation angle, or to perform operations such as pressing the imaging button on the operation button 41a to execute imaging, thereby improving user convenience.

[0078] Furthermore, in this embodiment, the process of imaging the object to be observed S involves imaging the object to be observed S over its entire circumference. This allows the user to confirm the entire outer surface of the object to be observed S.

[0079] Furthermore, in this embodiment, during the process of imaging the object to be observed S, the object to be observed S is imaged when the observation optical axis A of the head 21 is perpendicular to at least a portion of the portion of the object to be imaged S. This prevents the object to be observed S from being imaged when the tilt of the head 21 does not match the shape of the object to be observed S (the portion to be imaged).

[0080] Furthermore, in this embodiment, in the step of imaging the object to be observed S, the object to be observed S is imaged multiple times at at least one position where the rotation angle of the stage 25 is the same, with different focal points. In the step of displaying multiple images, multiple images are displayed on the display device 40 using all-focus images captured at the same position. This makes it easier for the user to confirm the object to be observed S appearing in each image, even when the object to be observed S has irregularities on its outer surface or when its outer surface is curved.

[0081] Furthermore, in this embodiment, in the process of imaging the object to be observed S, the observation optical axis A of the head 21 is parallel to the rotation center C of the stage 25, and the object to be observed S, for example, is imaged at the same focal point based on the shape of the periphery of the object to be observed S determined using the digital camera 27. This simplifies or eliminates the focusing process when imaging the object to be observed S for each rotation angle of the stage 25.

[0082] The above-described embodiment provides a concrete example to facilitate understanding of the invention. Therefore, the present invention is not limited to this embodiment and may include modified forms of this embodiment and alternative forms that replace the above-described embodiment. In other words, the components of this embodiment can be modified without departing from its spirit and scope. Furthermore, new embodiments can be implemented by appropriately combining the multiple components disclosed in this embodiment. In addition, some components may be deleted from the components shown in this embodiment, or some components may be added to the components shown in this embodiment. Moreover, the processing procedures shown in each embodiment may be performed in a different order, as long as they do not contradict each other. That is, the observation method, observation apparatus, and program of the present invention can be modified and changed in various ways without departing from the scope of the claims. [Explanation of Symbols]

[0083] 10 Observation device 20 Microscopes 21 heads 22 Light source 23 Observation Optical System 231 Objective lens 24 Tilt section 241 Rotating shaft section 242 Z-axis holding part 25 stages 251 XY Stages 251x X-axis movable mechanism 251xa Ball Screw 251xb scale 251xc Scale Head 251xd motor 251xe handle 251xf connection part 251y Y-axis movable mechanism 251ya handle 252 Rotating Stage 253 Z Stage 26 frames 27 Digital Cameras 271 Image Sensor 28 Control section 30 Control device 40 Display device 41 Display screen 41a Operation Buttons 41b Preview display section 41c Live Image Display Unit 41d Map image display section 41e Selection Area 50 Input devices 100 Computers 101 Processors 102 memory 103 Storage device 104 Reading device 105 Storage medium 106 Communication Interface 107 Input / Output Interfaces 108 Bus A Observation optical axis C is the center of rotation. P1-P4 Acquired Images P10 Composite image P20 Enlarged Image S Observation Subject

Claims

1. A step of imaging an object placed on a rotatable stage from a direction different from the center of rotation of the stage, at each of several positions where the rotation angle of the stage is different, A step of obtaining the rotation angle of the stage in each of the multiple images captured at the multiple positions, The process of displaying the multiple images on a display unit by either displaying a composite image obtained by stitching together the multiple images in accordance with the positional relationship of the portion of the subject to be observed contained in each of the multiple images, arranging the multiple images in accordance with the positional relationship of the portion of the subject to be observed, or displaying the corresponding rotation angle of the stage within or around each of the multiple images. Includes, In the step of imaging the object to be observed, it is determined whether the observation optical axis of the head having the imaging unit for imaging the object to be observed is perpendicular to at least a part of the portion to be imaged. If it is determined that it is perpendicular, an image of the object to be observed is acquired. An observation method characterized by the following.

2. In the step of displaying the multiple images, when displaying the stitched image, the display unit displays the stitched image and an enlarged image obtained by enlarging a selected area arbitrarily chosen from the stitched image. The observation method according to claim 1, characterized by the feature.

3. In the process of displaying the multiple images, when displaying the stitched image, the brightness value of the overlapping portion resulting from the stitching of the multiple images is adjusted and displayed on the display unit. The observation method according to claim 1 or 2, characterized by the features described herein.

4. If the predetermined conditions are met, the process of displaying the multiple images is executed, or the user is notified to prompt the execution of the said process. The observation method according to any one of claims 1 to 3, characterized by the features described herein.

5. The condition described above is met when the observation optical axis is tilted by a threshold or more with respect to the rotation center of the stage. The observation method according to feature 4.

6. The predetermined conditions described above are met when the observation optical axis is tilted with respect to the rotation center of the stage, and the stage is rotated. The observation method according to feature 4.

7. The condition described above is met when the imaging unit detects that the object being observed has a curved surface. The observation method according to feature 4.

8. The condition described above is met when the imaging unit detects that the height of the object being observed is equal to or greater than a threshold. The observation method according to feature 4.

9. If the above predetermined conditions are met, at least one of the following is the case: when the observation optical axis is parallel to the rotation center of the stage and the imaging unit determines that the periphery of the object being observed is circular; or when the center of the object being observed coincides with the rotation center of the stage. The observation method according to feature 4.

10. In the process of imaging the object to be observed, the object to be observed is automatically imaged when the stage rotates to a predetermined rotation angle. The observation method according to any one of claims 1 to 9, characterized by the following:

11. In the step of imaging the object to be observed, the object to be observed is imaged over its entire circumference. The observation method according to any one of claims 1 to 10, characterized by the features described herein.

12. The head is held in the frame via a tilt section, and the observation optical axis is tilted by tilting the head around the rotation axis of the tilt section. The observation method according to claim 1, characterized by the feature.

13. A step of imaging an object placed on a rotatable stage from a direction different from the center of rotation of the stage, at each of several positions where the rotation angle of the stage is different, A step of obtaining the rotation angle of the stage in each of the multiple images captured at the multiple positions, The process of displaying the multiple images on a display unit by either displaying a composite image obtained by stitching together the multiple images in accordance with the positional relationship of the portion of the subject to be observed contained in each of the multiple images, arranging the multiple images in accordance with the positional relationship of the portion of the subject to be observed, or displaying the corresponding rotation angle of the stage within or around each of the multiple images. Includes, In the step of imaging the object to be observed, the object to be observed is imaged multiple times at at least one position where the rotation angle of the stage is the same, with different focal points. In the step of displaying the multiple images, the multiple images are displayed on the display unit using the full-focus images captured at the same position. An observation method characterized by the following.

14. A step of imaging an object placed on a rotatable stage from a direction different from the center of rotation of the stage, at each of several positions where the rotation angle of the stage is different, A step of obtaining the rotation angle of the stage in each of the multiple images captured at the multiple positions, The process of displaying the multiple images on a display unit by either displaying a composite image obtained by stitching together the multiple images in accordance with the positional relationship of the portion of the subject to be observed contained in each of the multiple images, arranging the multiple images in accordance with the positional relationship of the portion of the subject to be observed, or displaying the corresponding rotation angle of the stage within or around each of the multiple images. Includes, In the process of imaging the object to be observed, the observation optical axis of the head having the imaging unit for imaging the object to be observed is parallel to the rotation center of the stage, and the object to be observed is imaged at the same focal point based on the shape of the periphery of the object to be observed determined using the imaging unit. An observation method characterized by the following.

15. A rotatable stage on which the object to be observed is placed, An imaging unit that images the object to be observed from a direction different from the rotation center of the stage at each of a plurality of positions where the rotation angle of the stage is different, A display unit that displays the image of the object to be observed captured by the imaging unit, A rotation angle detection unit that detects the rotation angle of the stage in each of the multiple images captured at the multiple positions, A control unit that displays the plurality of images on the display unit by either displaying a composite image obtained by stitching together the plurality of images in accordance with the positional relationship of the portion of the object to be observed contained in each of the plurality of images, arranging the plurality of images in accordance with the positional relationship of the portion of the object to be observed, or displaying the corresponding rotation angle of the stage within or around each of the plurality of images. Equipped with, The control unit determines whether the observation optical axis of the head having the imaging unit is perpendicular to at least a portion of the area to be imaged, and if it is determined to be perpendicular, it acquires the image of the object being observed. An observation device characterized by the following features.

16. The head is held in the frame via a tilt section, and the observation optical axis is tilted by tilting the head around the rotation axis of the tilt section. The observation apparatus according to claim 15, characterized by the features described herein.

17. A function to acquire images of an object placed on a rotatable stage, taken from a direction different from the center of rotation of the stage, at each of multiple positions with different rotation angles of the stage, A function to acquire the rotation angle of the stage in each of the multiple images captured at the multiple positions, The function of displaying the multiple images on the display unit is to display a composite image obtained by stitching together the multiple images in accordance with the positional relationship of the portion of the object to be observed contained in each of the multiple images, to arrange the multiple images in accordance with the positional relationship of the portion of the object to be observed, or to display the corresponding rotation angle of the stage within or around each of the multiple images. Have the computer run it, The function for acquiring the aforementioned image determines whether the observation optical axis of the head having the imaging unit for imaging the object being observed is perpendicular to at least a part of the portion being imaged, and if it is determined to be perpendicular, the captured image is acquired. A program characterized by the following features.

18. The head is held in the frame via a tilt section, and the observation optical axis is tilted by tilting the head around the rotation axis of the tilt section. The program according to claim 17, characterized by the features described herein.

Images