Observation apparatus

Abstract

To provide a highly convenient observation apparatus.SOLUTION: An observation apparatus 10 comprises: an observation section 40 that generates an observation image of an observation target OB; an analysis section 50 that analyzes the observation target OB; and an information processing section 30 that communicates with the observation section 40 and the analysis section 50. The analysis section 50 includes an analysis beam device that emits an analysis beam to the observation target OB. The information processing section 30 has a cluster region identification section, a sequence determination section, and an analysis execution section. The cluster region identification section identifies a plurality of cluster regions included in the observation image. The sequence determination section determines a sequence of analyzing the cluster regions. The analysis execution section causes the analysis beam device of the analysis section 50 to sequentially irradiate each of the cluster regions with the analysis beam according to an analysis sequence determined by the sequence determination section.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an observation device. [Background technology]

[0002] Observation devices such as microscopes are sometimes used to observe and analyze objects. The microscope disclosed in Patent Document 1 includes an observation optical system including a camera that captures an image of the object, an analysis optical system including an electromagnetic wave emitting unit that emits electromagnetic waves for analyzing the object, and a horizontal drive mechanism that moves the observation optical system and the analysis optical system. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-064854 Summary of the Invention [Problem to be solved by the invention]

[0004] In the observation device of Patent Document 1, an image of the observation target area is captured by a camera, and then the imaged area is irradiated with electromagnetic waves, thereby enabling analysis of the observation target area. However, it is difficult to analyze multiple analysis areas simultaneously, which poses a problem in terms of convenience.

[0005] In view of the above problems, an object of the present invention is to provide a highly convenient observation device. [Means for solving the problem]

[0006] In order to solve the above problem, an observation device as an example of an embodiment of the present invention is an observation device that observes and analyzes an observation object, and comprises: a mounting table on which the observation object is placed; an observation unit that generates an observation image of the observation object; an analysis unit that analyzes the observation object; and an information processing unit that communicates with the observation unit and the analysis unit. The observation unit includes an observation objective lens that receives light from the observation object, and an observation camera that generates an observation image by capturing an image of the observation object through the observation objective lens. The analysis unit includes an analysis beam device that emits an analysis beam to the observation object, and a detector that detects energy absorbed by or emitted from the observation object when irradiated with the analysis beam. The information processing unit has an imaging control unit that generates an observation image by controlling the observation camera of the observation unit to capture an image of the observation object, a mass region identification unit that identifies multiple mass regions included in the observation image, an order determination unit that determines the analysis order for analyzing the mass regions, and an analysis execution unit that irradiates each of the mass regions with an analysis beam using the analysis beam device of the analysis unit in accordance with the analysis order determined by the order determination unit, and causes the analysis unit to analyze the mass regions based on the detection results of the detector. [Effects of the Invention]

[0007] According to the present invention, an analysis order for analyzing multiple mass regions contained in the object to be observed is determined, and an analysis beam is sequentially irradiated onto each mass region in accordance with the analysis order, so there is no need to repeatedly capture an observation image using the observation unit for each mass region to be analyzed, thereby increasing the convenience of the observation device. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram schematically illustrating the configuration of an observation device. [Figure 2] FIG. 2 is a diagram showing a schematic diagram of the positional relationship between the observation unit, the analysis unit, and the mounting table of the imaging unit. [Figure 3] FIG. 3 is a diagram showing a state in which the relative position of the analysis unit with respect to the mounting table has changed from that shown in FIG. 2; [Figure 4] FIG. 2 is a block diagram schematically illustrating the configuration of an imaging unit. [Figure 5]FIG. 2 is a block diagram schematically showing an information processing unit and a communication target of the information processing unit. [Figure 6] FIG. 2 is a block diagram schematically illustrating the configuration of an information processing unit. [Figure 7] Flow diagram showing the flow of observation and analysis. [Figure 8] FIG. 10 is a diagram showing an example of a connected image. [Figure 9] FIG. 4 is a diagram showing the relationship between an observation image and an analysis image. [Figure 10] FIG. 10 is a diagram showing a case where an irradiation point of an analysis beam is determined using a thinning process. [Figure 11] FIG. 10 is a diagram showing a case where the geometric center of gravity of a mass region is outside the mass region. [Figure 12] FIG. 10 is a diagram showing a case where the mass region is fiber. [Figure 13] FIG. 10 is a diagram showing a case where a mass region in an analysis image is shifted from the center of the field of view. [Figure 14] FIG. 10 is a diagram showing a case where a correction operation by a user is received. [Figure 15] FIG. 10 is a diagram showing an example of a result screen. [Figure 16] FIG. 10 is a diagram showing an example of a result screen when a spectrum analysis is performed. [Figure 17] FIG. 10 is a diagram showing an example of an analysis list. DETAILED DESCRIPTION OF THE INVENTION

[0009] An observation device 10 as an example of an embodiment of the present invention will be described below with reference to the drawings. Note that identical or corresponding parts in the drawings are designated by the same reference numerals, and description thereof will not be repeated. Furthermore, in the following description, terms indicating positions or directions such as front, rear, left, right, up, and down may be used, but these terms are used for convenience to facilitate understanding of the embodiment. Unless otherwise specified, these terms are not limited to the strict geometric meaning of front, rear, left, right, up, and down.

[0010] FIG. 1 is a diagram schematically illustrating the configuration of an observation device 10. The observation device 10 includes a display unit 12, an operation unit 14, an imaging unit 20, and an information processing unit 30. The information processing unit 30 communicates with the display unit 12, the operation unit 14, and the imaging unit 20. The information processing unit 30 is connected to the display unit 12, the operation unit 14, and the imaging unit 20, for example, by a communication cable. Examples of communication cables that can be used include cables that transmit and receive electrical signals, such as a USB cable (Universal Serial Bus), a LAN cable (Local Area Network), and an HDMI cable (HDMI: High-Definition Multimedia Interface, a registered trademark), and optical cables that transmit and receive optical signals. Note that cables of different standards may be used depending on the components to be connected. Furthermore, signals may be transmitted and received between components via wireless communication using electromagnetic waves.

[0011] The information processing unit 30 is a unit including a processor such as a CPU (Central Processing Unit) and a storage device such as a RAM (Random Access Memory), and executes various types of information processing in the observation device 10.

[0012] The display unit 12 is a unit that includes an image display device such as an LCD (Liquid Crystal Display), and displays images to the user of the observation device 10. The operation unit 14 is a unit that accepts operations from the user of the observation device 10. The operation unit 14 includes user interface devices such as a keyboard, a mouse, and a joystick. The operation unit 14 may also include a controller designed specifically for the observation device 10.

[0013] The observation device 10 is a device that observes and analyzes an observation object OB. The imaging unit 20 is a unit that captures and analyzes the observation object OB. The imaging unit 20 includes an observation unit 40, an analysis unit 50, and a movement unit 60. The information processing unit 30 communicates with the observation unit 40, analysis unit 50, and movement unit 60 of the imaging unit 20. The observation unit 40 is a unit that generates an observation image of the observation object OB. The analysis unit 50 is a unit that analyzes the observation object OB. The observation unit 40 and the analysis unit 50 may be an integrated unit or separate units. When the observation unit 40 and the analysis unit 50 are separate units, the movement unit 60 changes the relative positions of the observation unit 40 and the analysis unit 50 with respect to the observation object OB.

[0014] When observing and analyzing the observation object OB, first, the observation unit 40 captures an image of the observation object OB to generate an observation image. Here, a display screen based on the observation image may be displayed on the display unit 12. The display screen may be generated by the information processing unit 30 based on the observation image.

[0015] After the observation unit 40 generates an observation image, the observation object OB is analyzed by the analysis unit 50. Details of the analysis will be described later, but for example, a component analysis of the analysis object is performed. Then, the information processing unit 30 generates a result screen showing the analysis results, and the result screen is displayed on the display unit 12.

[0016] The imaging unit 20 will be described with reference to Fig. 2. Fig. 2 is a diagram schematically showing the positional relationship between the observation unit 40, analysis unit 50, and mounting table 70 of the imaging unit 20. The observation unit 40 is a unit covered with an observation housing 40a. The analysis unit 50 is a unit covered with an analysis housing 50a. The imaging unit 20 further includes the mounting table 70, a Z-direction stage 72, a support base 74, and a support column 76.

[0017] The mounting table 70 is a table on which the observation object OB is placed. The mounting table 70 is supported by a mounting table control unit 64. The mounting table control unit 64 moves the mounting table 70 in the horizontal direction. In FIG. 2, the left-right direction in the figure is the horizontal direction. The horizontal direction includes the X direction (the left-right direction relative to the entire imaging unit 20) and the Y direction (the front-rear direction relative to the entire imaging unit 20), but FIG. 2 shows only the Y direction, and the X direction is the direction perpendicular to the paper surface. When the mounting table control unit 64 moves the mounting table 70 in the X direction and the Y direction, the relative positions of the observation unit 40 and the analysis unit 50 with respect to the mounting table 70 and the observation object OB change. The mounting table control unit 64 is supported by a support table 74. The support table 74 supports the entire imaging unit 20.

[0018] The observation unit 40 is disposed above the mounting table 70. The observation unit 40 includes an observation camera 42 for capturing an image of the observation object OB. The observation camera 42 is an element that converts reflected light or transmitted light from the observation object OB into an electrical signal, and for example, an image sensor using a CCD (Charge Coupled Device) or an image sensor using a CMOS (Complementary Metal Oxide Semiconductor) is used.

[0019] 2 is connected to an observation communication cable C1 for communicating with the information processing unit 30 and a light guide cable C2 for guiding light from the outside. The observation unit 40 includes an observation camera 42, an observation half mirror 44, a lens switching unit 46, an observation objective lens 48, and a side illumination 48a.

[0020] The observation half mirror 44 reflects the light guided by the light guide cable C2 and guides it to the surface of the observation object OB through the observation objective lens 48. The light guided by the observation half mirror 44 is preferably coaxial with the observation optical axis Ao of the observation objective lens 48. When the external light guided by the light guide cable C2 and the observation half mirror 44 is coaxial with the observation optical axis Ao of the observation objective lens 48, the guided light functions as coaxial illumination. Although only one observation half mirror 44 is shown in FIG. 2, light may be guided by a mirror group including multiple half mirrors. The mirror group may also include a total reflection mirror. The observation unit 40 may also include an observation light source that irradiates the observation object OB with light for observation. If the observation light source is included in the observation unit 40, the light guide cable C2 is not necessarily required. The observation half mirror 44 also transmits light from the observation object OB received by the observation objective lens 48 and guides it to the observation camera 42.

[0021] 2, the side illuminator 48a is configured as a ring illuminator arranged to surround the observation objective lens 48. The side illuminator 48a irradiates the observation object OB with illumination light from diagonally above. Although detailed illustration is omitted, when the side illuminator 48a is considered to be a ring, its central axis coincides with the observation optical axis Ao. The side illuminator 48a is divided into multiple blocks in the circumferential direction, and each block can be individually lit.

[0022] The observation objective lens 48 has an observation optical axis Ao along the vertical direction (Z direction, up and down in the figure). The observation objective lens 48 receives (collects) light from the observation object OB and guides an image of the observation object OB along the observation optical axis Ao to the observation camera 42. Note that various other optical elements, such as an imaging lens and an optical diaphragm, may be arranged in the optical path between the observation objective lens 48 and the observation camera 42 to change optical conditions such as magnification and focal length. Note that the observation optical axis Ao is preferably parallel to the reference axis As of the mounting table 70. The reference axis As is an axis perpendicular to the upper surface of the mounting table 70.

[0023] The observation objective lens 48 in FIG. 2 may be disposed below the observation unit 40 via a lens switching unit 46. The lens switching unit 46 includes multiple objective lenses with different magnifications, and one of the objective lenses is directed toward the observation object OB as the observation objective lens 48 for observing the observation object OB. The lens switching unit 46 is, for example, a revolver that rotates to switch the objective lens directed toward the observation object OB. The lens switching unit 46 switches the objective lens so that the observation optical axis Ao of the observation objective lens 48 coincides with the optical path guided by the observation half mirror 44, regardless of which objective lens is selected as the observation objective lens 48. Note that the observation objective lens 48 is not limited to a configuration in which the objective lens is switched via the lens switching unit 46. The observation housing 40a may also include a zoom lens whose magnification is variable, or an imaging lens used in conjunction with the observation objective lens 48.

[0024] The observation unit 40 is attached to the Z-direction stage 72 via a unit moving unit 62. In addition to the observation unit 40, the analysis unit 50 is also attached to the unit moving unit 62. In FIG. 2, the analysis unit 50 is positioned closer to the Z-direction stage 72 than the observation unit 40. The unit moving unit 62 moves the observation unit 40 and the analysis unit 50 horizontally (the Y direction in FIG. 2). The unit moving unit 62 is, for example, a unit including an arm that supports the observation unit 40 and the analysis unit 50 and an actuator (not shown) that moves them horizontally along the arm. As the unit moving unit 62 moves along the arm, the relative positions of the observation unit 40 and the analysis unit 50 with respect to the mounting table 70 change. Note that the mounting table control unit 64 may move the mounting table 70 horizontally to change the relative positions of the observation unit 40 and the analysis unit 50 with respect to the mounting table 70.

[0025] The Z-direction stage 72 moves the observation unit 40 and the analysis unit 50 relative to the mounting table 70 in the vertical direction (Z direction). The observation unit 40 and the analysis unit 50 can also be swung around a swing axis 79 that passes through a support column 76 and extends in the Y direction. The Z-direction stage 72 is attached to the support column 76. The support column 76 is connected to the support table 74 so as to be rotatable around the swing axis 79. A user of the observation device 10 can swing the observation unit 40 and the analysis unit 50 around the swing axis 79. By swinging the observation unit 40 and the analysis unit 50, the user can observe the observation object OB not only vertically but also from an oblique direction.

[0026] Figure 3 shows a state in which the relative position of the analysis unit 50 with respect to the mounting table 70 has changed from that in Figure 2. In Figure 3, the unit moving unit 62 has moved along the arm, so that the observation optical axis Ao of the observation unit 40 is away from the observation object OB, and the analysis optical axis Aa of the analysis unit 50 is directed toward the observation object OB. Therefore, in Figure 3, the analysis unit 50 is disposed above the mounting table 70.

[0027] 2 and 3 is connected to the analysis unit 50 via an analysis communication cable C3 for communicating with the information processing unit 30. The analysis unit 50 includes an analysis camera 52, an analysis beam deflector 53, an analysis half mirror 54, a detector 55, a detection half mirror 56, an analysis objective lens 58, and a side illumination unit 58a. The analysis camera 52 is a camera for capturing an image of the observation object OB when the analysis optical axis Aa of the analysis unit 50 is directed toward the observation object OB.

[0028] The analytical beam device 53 is a unit that emits an analytical beam to the observation object OB. Different types of particle beams or waves are used as the analytical beam depending on the analytical method. Examples of analytical beams include infrared light, visible light, X-rays, lasers, and electron beams. The analytical beam device 53 is, for example, a laser device or an electron gun.

[0029] The analytical half mirror 54 reflects the analytical beam emitted from the analytical beam deflector 53 and directs it to the surface of the observation object OB through the analytical objective 58. The beam directed by the analytical half mirror 54 is preferably coaxial with the analytical optical axis Aa of the analytical objective 58. Although only one analytical half mirror 54 is shown in FIG. 3, the analytical beam may be directed by a mirror group including multiple half mirrors. The mirror group may also include a total reflection mirror. The analytical half mirror 54 also transmits electromagnetic waves from the observation object OB received by the analytical objective 58 and directs them to the analytical camera 52 and the detection half mirror 56. Note that depending on the type of analytical beam (for example, when the analytical beam is an electron beam), the analytical half mirror 54 may not be provided, and the analytical beam may be directly irradiated onto the observation object OB from the analytical beam deflector 53.

[0030] The detector 55 is a unit that detects the reaction of the observation object OB to the analysis beam (particularly, the energy absorbed by or emitted from the observation object OB). The detector 55 may be, for example, a Czerny-Turner spectrometer or a photomultiplier tube that detects light (electromagnetic waves), or an electron detector that detects electrons. The detector 55 receives electromagnetic waves from the observation object OB via the analysis objective lens 58, the analysis half mirror 54, and the detection half mirror 56, and detects the reaction of the observation object OB based on the received electromagnetic waves. Depending on the analysis method, the detector 55 may receive particles emitted from the observation object OB instead of electromagnetic waves. For example, the detector 55 may be an electron detector for backscattered electrons, secondary electrons, etc., and may receive electrons emitted from the observation object OB. The detector 55 may also be a displacement meter that measures the displacement caused in the observation object OB due to the reaction of the observation object OB, or a refractometer that measures the change in the refractive index caused in the observation object OB. Depending on the type of detector 55, the detector 55 may directly receive electromagnetic waves or particles emitted from the observation object OB without going through the analytical objective lens 58, the analytical half mirror 54, the detection half mirror 56, etc. The detector 55 can detect energy absorbed by or emitted from the observation object OB based on the electromagnetic waves or particles emitted from the observation object OB.

[0031] The detection half mirror 56 reflects the electromagnetic waves received by the analytical objective lens 58 and guides them to the detector 55. The detection half mirror 56 also transmits the electromagnetic waves received by the analytical objective lens 58 from the observation object OB and guides them to the analytical camera 52. The detection half mirror 56 may be a mirror group including multiple half mirrors or a total reflection mirror. Depending on the analysis method, the detection half mirror 56 may not be provided.

[0032] The side illuminator 58a is arranged to surround the outer periphery of the analytical objective lens 58. More specifically, the side illuminator 58a is configured as an annular illuminator that annularly surrounds the analytical objective lens 58. The central axis of the annulus corresponding to the side illuminator 58a (the central axis when the side illuminator 58a is considered as a ring) is arranged to be coaxial with the analytical optical axis Aa. The side illuminator 58a irradiates illumination light through an optical path inclined with respect to the analysis beam emitted from the analytical beam projector 53, irradiating the observation object OB with the illumination light from diagonally above. Although not shown in FIGS. 2 and 3 , the analysis unit 50 may preferably include, in addition to the side illuminator 58a, a coaxial illumination light source (such as an LED light source) that irradiates illumination light coaxially with the analytical optical axis Aa.

[0033] The analytical objective lens 58 has an analytical optical axis Aa along the vertical direction (Z direction, up and down in the figure). The analytical objective lens 58 receives (focuses) electromagnetic waves from the observation object OB and directs them to the detector 55 along the analytical optical axis Aa. The analytical objective lens 58 also receives light (particularly visible light) from the observation object OB and directs it to the analytical camera 52 along the analytical optical axis Aa. Note that various additional optical elements, such as an imaging lens and an optical aperture, may be arranged in the optical path between the analytical objective lens 58 and the analytical camera 52 to change optical conditions such as magnification and focal length. Furthermore, the analytical optical axis Aa is preferably parallel to the reference axis As of the mounting table 70. Depending on the analysis method, the irradiation path of the analytical beam, the path of the electromagnetic waves or particles received by the detector 55, and the optical path of the analytical camera 52 may each be separate.

[0034] 2, when the observation optical axis Ao of the observation unit 40 is aligned with the observation object OB (i.e., when the analysis optical axis Aa of the analysis unit 50 is deviated from the observation object OB), a shielding cover 59 may be arranged below the analysis objective lens 58. When the shielding cover 59 is arranged below the analysis objective lens 58, the analysis beam guided to the surface of the observation object OB through the analysis objective lens 58 is blocked by the shielding cover 59. Therefore, even if the analysis beam is unintentionally emitted when the analysis optical axis Aa of the analysis unit 50 is deviated from the observation object OB, that is, when the analysis unit 50 is not in use, the analysis beam is blocked by the shielding cover 59. This improves the safety of the observation device 10.

[0035] Next, the configuration of the imaging unit 20 will be described with reference to Fig. 4. Fig. 4 is a block diagram schematically showing the configuration of the imaging unit 20. As shown in Fig. 4, the imaging unit 20 has an observation unit 40, an analysis unit 50, a movement unit 60, a mounting table 70, and a Z-direction stage 72.

[0036] As shown in FIG. 4 , the observation unit 40 may be a unit covered with an observation housing 40a that houses an observation camera 42 and an observation objective lens 48. Similarly, the analysis unit 50 may be a unit covered with an analysis housing 50a that houses an analysis camera 52 and an analysis objective lens 58. The observation housing 40a and the analysis housing 50a are preferably separate housings. Since the observation unit 40 and the analysis unit 50 are units covered with different observation housings 40a and analysis housings 50a, respectively, the movement unit 60 can change the relative positions of the observation unit 40 and the analysis unit 50 with respect to the mounting table 70 by moving the observation housing 40a and the analysis housing 50a. However, the observation housing 40a and the analysis housing 50a may be the same housing, and the observation unit 40 and the analysis unit 50 may be an integrated unit.

[0037] 4 includes an observation camera 42 and an observation objective lens 48, as well as an observation focus control unit 41 and an observation illumination unit 45. The observation focus control unit 41 controls the lens switching unit 46, various optical elements in the observation unit 40, the Z-direction stage 72, etc., to adjust the focus of the observation objective lens 48. The observation illumination unit 45 controls various optical elements in the observation unit 40, such as the side illumination 48a and the observation half mirror 44, to adjust the state of illumination for the observation object OB.

[0038] 4 includes an analytical camera 52 and an analytical objective 58, as well as an analytical focus control unit 51, an analytical beam unit 53, a detector 55, and an analytical illumination unit 57. The analytical focus control unit 51 controls various optical elements in the analytical unit 50, such as the analytical half mirror 54 and detection half mirror 56, and the Z-direction stage 72, to adjust the focus of the analytical objective 58. The analytical illumination unit 57 controls various optical elements in the analytical unit 50, such as the lateral illumination 58a and coaxial illumination light source, to adjust the illumination state for the observation object OB.

[0039] The movement section 60 includes a unit movement section 62 and a mounting table control section 64. The unit movement section 62 moves the observation section 40 and the analysis section 50 in the horizontal direction, thereby changing the relative positions of the observation section 40 and the analysis section 50 with respect to the mounting table 70. The mounting table control section 64 moves the mounting table 70 in the horizontal direction, thereby changing the relative positions of the observation section 40 and the analysis section 50 with respect to the mounting table 70.

[0040] Next, the configuration of the information processing unit 30 will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a block diagram schematically showing the information processing unit 30 and a communication target of the information processing unit 30. Fig. 6 is a block diagram schematically showing the configuration of the information processing unit 30.

[0041] 5, the information processing unit 30 communicates with the display unit 12, the operation unit 14, and the imaging unit 20. The imaging unit 20 includes an observation unit 40 and an analysis unit 50, and therefore the information processing unit 30 communicates with the observation unit 40 and the analysis unit 50.

[0042] The information processing unit 30 includes a calculation unit 31, a storage unit 32, and an input / output unit 33. The calculation unit 31 is a unit including a processor such as a CPU. The storage unit 32 is a unit including a storage device such as a RAM. The input / output unit 33 is a communication interface unit that controls signals input to and output from the information processing unit 30 in accordance with various communication standards so that the information processing unit 30 can communicate with external devices such as the imaging unit 20, display unit 12, and operation unit 14.

[0043] The various functions of the information processing unit 30 shown in Fig. 6 are realized by the calculation unit 31 of the information processing unit 30 executing a program stored in the storage unit 32. As shown in Fig. 6, the information processing unit 30 has an imaging control unit 34, a mass region identification unit 35, an order determination unit 36, an analysis execution unit 37, an image processing unit 38, a movement control unit 39, and an input / output control unit 33a.

[0044] The imaging control unit 34 controls the observation camera 42 of the observation unit 40 to generate an observation image based on light from the observation object OB. The imaging control unit 34 may also control the analysis camera 52 of the analysis unit 50 to generate an analysis image based on light from the observation object OB. The imaging control unit 34 may be configured to control the observation illumination unit 45 of the observation unit 40 or the analysis illumination unit 57 of the analysis unit 50 when generating the observation image or analysis image, so that illumination light is irradiated onto the observation object OB by the lateral illumination 48a and coaxial illumination of the observation unit 40, or the lateral illumination 58a and coaxial illumination of the analysis unit 50.

[0045] The cluster region identifying unit 35 identifies multiple cluster regions contained in the observation image. A cluster region is a region in the observation image where pixels that satisfy predetermined conditions (such as brightness, hue, and saturation) are clustered together. For example, the cluster region identifying unit 35 identifies a region where pixels that exceed or fall below a threshold value set for brightness (brightness) are clustered together as a single cluster region. Note that such cluster regions in the observation image are sometimes called "particles" or "contamination." Hereinafter, a region on the observation object OB that corresponds to a region observed as a cluster region in the observation image will also be referred to as a "clump region."

[0046] The order determination unit 36 ​​determines the analysis order in which the analysis unit 50 will analyze the mass regions identified by the mass region identification unit 35. The order determination unit 36, for example, performs image processing on the observed image to identify the characteristics of each mass region, and determines the analysis order based on the characteristics of the mass regions, such as their size and shape.

[0047] The analysis execution unit 37 causes the analysis beam detector 53 of the analysis unit 50 to sequentially irradiate each of the mass regions with an analysis beam in accordance with the analysis order determined by the order determination unit 36, and causes the analysis unit 50 to analyze the mass regions based on the reaction of the mass regions irradiated with the analysis beam (detection results of the detector 55). For example, the analysis execution unit 37 aligns the analysis unit 50 with the mounting table 70 by sending a command to the movement unit 60. In aligning the analysis unit 50 with the mounting table 70, the movement unit 60 sequentially changes the relative position of the analysis unit 50 with respect to the mounting table 70 in accordance with the analysis order so that the analysis beam is irradiated onto each of the mass regions.

[0048] Then, each time the moving unit 60 completes the positioning of the analysis unit 50 for each mass region, the analysis execution unit 37 causes the analysis beam emitter 53 to irradiate the mass region with an analysis beam. The reaction (absorbed energy or emitted energy) of the mass region irradiated with the analysis beam is detected by the detector 55. The analysis unit 50 analyzes the mass region based on the reaction (detection result) detected by the detector 55.

[0049] The image processing unit 38 performs image processing on the image captured by the imaging unit 20 and on the image displayed on the display unit 12. For example, when the mass region identifying unit 35 identifies a mass region, the image processing unit 38 performs binarization processing on brightness and size measurement processing on each mass region. Furthermore, when the observation image or analysis image is displayed on the display unit 12, the image processing unit 38 performs synthesis processing to overlay visual information for the user on the observation image or analysis image.

[0050] The movement control unit 39 sends commands to the movement unit 60 of the imaging unit 20 to control the movement of the observation unit 40 and the analysis unit 50 by the unit movement unit 62 and the movement of the mounting table 70 by the mounting table control unit 64 .

[0051] The input / output control unit 33a controls the input / output unit 33 of the information processing unit 30, and controls communication between the information processing unit 30 and external devices such as the imaging unit 20, the display unit 12, and the operation unit .

[0052] Next, the flow of observation and analysis using the observation device 10 will be described with reference to the flow diagram in Figure 7. As an example, the following describes a case where a liquid to be analyzed is filtered through a circular filter paper, and foreign matter (contamination) remaining on the filter paper is observed and analyzed, but the observation device 10 can also be used for a variety of other tasks. First, a user of the observation device 10 places an observation object OB (e.g., a filter paper that has filtered a liquid) on the mounting table 70. Then, the user issues a command to the observation device 10 to start observing and analyzing the observation object OB, and the observation and analysis begin (START).

[0053] When observation and analysis are started, first, in step S11, the imaging control unit 34 controls the observation camera 42 to capture an image of the observation object OB, thereby generating an observation image. Here, the imaging control unit 34 may generate only one observation image by capturing an image of the entire or part of the observation object OB using the observation camera 42, or may generate multiple observation images. When generating multiple observation images, the imaging control unit 34 may generate multiple observation images by causing the observation camera 42 to capture images of multiple locations on the observation object OB while changing the relative position of the observation unit 40 with respect to the mounting table 70 using the movement unit 60.

[0054] Furthermore, when generating a plurality of observation images, the imaging control unit 34 may generate a concatenated image 80 by concatenating a plurality of observation images 81. FIG. 8 shows an example of the concatenated image 80. In FIG. 8, the concatenated image 80 of the observation object OB, which is circular in plan view, is formed by concatenating a plurality of observation images 81. Each of the observation images 81 is generated by capturing an image of a part of the observation object OB. The concatenated image 80 generated by concatenating a plurality of observation images 81 includes an entire image of the observation object OB.

[0055] Here, the imaging range of the multiple observation images 81 captured by the imaging control unit 34 (the range of the imaging target to be included in the concatenated image 80) may be, for example, a rectangle defined by multiple endpoints specified by the user via the operation unit 14, or an area included within a specified range from a predetermined reference point may be determined as the imaging range. Furthermore, the relative position of the observation unit 40 with respect to the mounting table 70 may be sequentially changed by the movement unit 60 so that the center of the field of view of the observation camera 42 is at three points on the circumference of the filter paper, and the imaging range may be a circle defined based on the coordinates on each of the mounting tables 70 when the center of the field of view becomes three points on the circumference of the filter paper.

[0056] 8, the concatenated image 80 is generated by concatenating a plurality of observation images 81 obtained by capturing images of continuous regions within the observation object OB, but the concatenated image 80 may also be generated by concatenating a plurality of observation images 81 obtained by capturing images of regions (discrete regions) that are separated from one another within the observation object OB. For example, if only a portion of the observation object OB is to be observed and analyzed, and the region to be observed and analyzed is a plurality of discrete regions, the concatenated image 80 may be generated by concatenating the observation images 81 of the multiple discrete regions based on the stage coordinates (coordinates on the mounting table 70).

[0057] Furthermore, when generating an observation image by controlling the observation camera 42, the imaging control unit 34 may generate a plurality of observation images while changing the height of the observation unit 40 using the Z-direction stage 72. Specifically, a plurality of observation images are generated while changing the vertical position (height) of the observation unit 40 while keeping the horizontal position of the observation unit 40 relative to the mounting table 70 the same. In this case, a plurality of observation images are generated that are captured from different heights relative to the observation object OB.

[0058] When the observation unit 40 generates a plurality of observation images at different heights relative to the mounting table 70, the information processing unit 30 may generate a depth-combined image or a three-dimensional image of the observation object OB based on the plurality of observation images generated at different heights. Each observation image shows the planar shape of the observation object OB, and the in-focus portion of the observation image corresponds to the height of the observation unit 40 when the observation image was captured. Therefore, by combining the in-focus portions of each observation image for the plurality of observation images (depth composition), a depth-combined image or a three-dimensional image of the observation object OB can be generated.

[0059] 7, the mass region identifying unit 35 identifies a plurality of mass regions contained in the observed image. Here, the mass region identifying unit 35 may receive designation of extraction conditions for identifying mass regions from the observed image, and identify (extract) regions of the observed image that match the extraction conditions as mass regions. Note that the extraction conditions may be determined in advance.

[0060] For example, in step S12, a screen for accepting the specification of extraction conditions may be displayed on the display unit 12, and the extraction conditions may be set in response to a user's operation on the screen via the operation unit 14. For example, the user may specify a brightness threshold, and an area in the observed image where pixels having brightness above the specified threshold or below a specific threshold are clustered together is identified as a cluster area.

[0061] For example, in an observation image (here, a concatenated image 80) of filter paper that has filtered liquid, as shown in Figure 8, when foreign matter remaining on the filter paper is identified as a lump region 85, the foreign matter appears as a dark region, so it is preferable to identify the region below a specified brightness threshold as the lump region 85.

[0062] 8, when a concatenated image 80 is generated by concatenating a plurality of observational images 81, the mass region identifying unit 35 may identify a plurality of mass regions included in the concatenated image 80. In the concatenated image 80, a mass region 88 may exist that spans a plurality of observational images 81. The mass region identifying unit 35 may identify such a mass region 88 that spans a plurality of observational images 81 as a single mass region within the concatenated image 80. When mass region identification is performed for each individual observational image 81, a mass region 88 that spans a plurality of observational images 81 may be identified as a single mass region in each of the two observational images 81 and may be considered as two mass regions as a whole. However, when mass region identification is performed for the entire concatenated image 80, the mass region is correctly identified as a single mass region.

[0063] Furthermore, when identifying the mass regions, the mass region identifying unit 35 may also identify the size of each mass region. The user may then specify the size of the mass regions as an extraction condition. When the size of the mass regions is specified as an extraction condition, only regions that meet the specified size condition are identified as mass regions, and other regions are excluded. For example, as shown in FIG. 8, when small mass regions 85 and large mass regions 86 coexist, if a size condition is specified to exclude the small mass region 85, only the large mass region 86 (and the mass region 88 that is large enough to span multiple observation images 81) will be regarded as a mass region, and the small mass region 85 will no longer be regarded as a mass region.

[0064] Furthermore, when a three-dimensional image of the observation object OB has been generated, the mass region specifying section 35 may specify the height of each mass region based on the three-dimensional image.

[0065] Furthermore, the mass region identification unit 35 may classify multiple mass regions into two or more attributes. When identifying the characteristics of the mass regions, the mass region identification unit 35 may classify mass regions with similar characteristics as mass regions with the same attribute. For example, the mass region identification unit 35 may classify multiple mass regions with sizes within a specific range as mass regions with the same attribute, or multiple mass regions with a specific color or brightness value as mass regions with the same attribute. In this case, a relatively bright mass region may be determined to be metallic based on the brightness values ​​of multiple pixels included in the mass region, such as classifying a mass region with a high average pixel brightness as metallic.

[0066] Once the mass region identification unit 35 has completed identification of the mass region, the order determination unit 36 ​​then determines the analysis order in step S13 of Fig. 7. For example, the order determination unit 36 ​​determines the analysis order in the order of mass regions found by searching the observation image from the edge.

[0067] Specifically, the order determination unit 36 ​​first searches for mass regions in the X direction to the right, using the range near the top end of the observation object OB as the search range for a concatenated image 80 such as that shown in Figure 8, and determines the analysis order in the order in which the mass regions are found, starting with the first mass region found as the first mass region in the analysis order. Then, once the search has been completed up to the right end in the X direction, the search range is shifted slightly in the Y direction, and mass regions are again searched for in the X direction to the right. Once the search for mass regions has been completed at the bottom end of the observation object OB, the analysis order has been determined for all mass regions in the observation object OB.

[0068] 7, the order determination unit 36 ​​may determine the analysis order based on the size of the lump region in step S13. The order determination unit 36 ​​may also determine the lump region to be analyzed (which lump region should be irradiated with the analysis beam) based on the size of the lump region.

[0069] For example, the order determination unit 36 ​​may determine the analysis order in descending order of the size of the block regions. Large block regions are likely to be of great interest to the user, so by analyzing them in descending order of size, the analyses will be performed in the order that the user is most interested in, and there is a high possibility that the analysis order will be as desired by the user.

[0070] Alternatively, the order determination unit 36 ​​may determine the analysis order in ascending order of the size of the mass regions. For example, if the analysis method performed by the analysis unit 50 is a method that causes irreversible changes to the observation object OB (destructive testing), when a change occurs in a large mass region, the small mass regions may be affected by the change. In such a case, by performing the analysis in descending order, it becomes possible to analyze the small mass regions thoroughly and more accurately before they are affected by the analysis of the large mass regions.

[0071] Furthermore, when determining the agglomerate regions to be analyzed based on the size of the agglomerate regions, the order determination unit 36 ​​may, for example, accept a user's designation of the size of the agglomerate regions to be analyzed and determine agglomerate regions larger than the designated predetermined size as the agglomerate regions to be analyzed.The order determination unit 36 ​​then determines the analysis order of the agglomerate regions to be analyzed.Note that the order determination unit 36 ​​may also designate agglomerate regions smaller than the predetermined size designated by the user as the agglomerate regions to be analyzed.

[0072] Furthermore, when a three-dimensional image of the observation object OB has been generated and the mass region identification unit 35 has identified the height of each mass region, the order determination unit 36 ​​may determine the analysis order based on the size and height of the mass region. Since the three-dimensional size of the mass region can be identified from the size and height of the mass region, the order determination unit 36 ​​may determine the analysis order based on the three-dimensional size of the mass region. For example, the analysis order may be determined in descending or ascending order of three-dimensional size.

[0073] The order determination unit 36 ​​may also accept a designation of priority for the mass regions and determine the analysis order based on the designation of priority. For example, in step S13, a screen for accepting designation of priority may be displayed on the display unit 12, and the priority may be designated in accordance with an operation of the user on that screen via the operation unit 14. For example, the user may designate a mass region that the user wishes to analyze preferentially as a mass region with a high priority. Then, the order determination unit 36 ​​determines the analysis order in descending order of the designated priorities.

[0074] Note that when the user specifies priorities, priorities do not need to be specified for all agglomerate regions. If the user specifies priorities for only some agglomerate regions, the order determination unit 36 ​​may automatically determine the analysis order for the agglomerate regions for which priorities have not been specified. For example, the order determination unit 36 ​​may divide the analysis order into a first sequence and a second sequence that is executed subsequent to the first sequence. Then, in the first sequence, the analysis order may be determined based on the priorities specified by the user, and in the second sequence, the analysis order for the agglomerate regions for which priorities have not been specified may be determined. The analysis order in the second sequence may be determined, for example, in the order in which the agglomerate regions are found by searching the observed image from the edge, or may be determined based on the size of the agglomerate regions.

[0075] Furthermore, when the mass region identifying unit 35 classifies the mass regions into two or more attributes, the order determining unit 36 ​​may determine the analysis order based on the attributes of the mass regions. For example, the order determining unit 36 ​​may analyze mass regions classified into the same attribute in consecutive order, or may analyze mass regions classified into a specific attribute in early or late order. Alternatively, the order determining unit 36 ​​may analyze only mass regions classified into a specific attribute by the mass region identifying unit 35, and determine the analysis order for mass regions having that attribute. When determining the analysis order based on the attributes of the mass regions, the order determining unit 36 ​​may determine the attribute of each mass region and determine the analysis order for the mass regions to be analyzed based on the attribute determination result.

[0076] Once the analysis order has been determined by the order determination unit 36, unit switching from the observation unit 40 to the analysis unit 50 is then performed in step S14 of Fig. 7. That is, the movement unit 60 changes the relative position of the analysis unit 50 with respect to the mounting table 70 so that the analysis beam is irradiated onto the portion of the observation object OB where the observation image was captured by the observation camera 42. Note that if the observation unit 40 and the analysis unit 50 are an integrated unit, unit switching from the observation unit 40 to the analysis unit 50 is not performed.

[0077] More specifically, regarding the unit switching in step S14, the relative position of the analysis unit 50 with respect to the mounting table 70 is changed so that the center of the field of view of the observation camera 42 coincides with the center of the field of view of the analysis camera 52 and the irradiation point of the analysis beam from the analysis beam device 53.

[0078] The fact that the center of the field of view of the observation camera 42 coincides with the center of the field of view of the analysis camera 52 will be described with reference to FIG. 9 . FIG. 9 is a diagram illustrating the relationship between the observation image 81 and the analysis image 90. As shown in FIG. 9 , if the center of the field of view Co of the observation image 81 captured by the observation camera 42 is aligned with the large mass region 86, the relative position of the analysis unit 50 with respect to the mounting table 70 is changed after unit switching so that the center of the field of view Ca of the analysis image 90 captured by the analysis camera 52 also coincides with the same large mass region 86. In this embodiment, if the analysis beam guided by the analysis half mirror 54 of the analysis unit 50 is coaxial with the analysis optical axis Aa of the analysis objective lens 58, the analysis beam is irradiated onto the center of the field of view Ca of the analysis image 90.

[0079] When switching units, the unit moving section 62 of the moving section 60 or the mounting table control section 64 moves the analysis section 50 or the mounting table 70 horizontally by an amount equal to the distance between the observation optical axis Ao and the analysis optical axis Aa, but simple horizontal movement may result in a misalignment between the center of field of view Co of the observation image 81 and the center of field of view Ca of the analysis image 90 (and the irradiation point of the analysis beam). Therefore, it is advisable to calculate in advance the amount of misalignment of the center of field of view that occurs when switching between the observation section 40 and the analysis section 50, and then, when switching units, correct the amount of movement by the moving section 60 by the amount of the pre-calculated misalignment.

[0080] In the unit switching in step S14 (when changing the relative position of the analysis unit 50 with respect to the mounting table 70 so that the analysis beam is irradiated onto the location where the observation image was captured), it is preferable that the moving unit 60 adjust the relative position of the analysis unit 50 with respect to the mounting table 70 in accordance with the optical magnification of the observation objective lens 48 of the observation unit 40 when the observation image 81 was generated. Specifically, it is preferable to calculate a movement amount corresponding to the optical magnification of the observation objective lens 48 of the observation unit 40 when the observation image was generated, and adjust the relative position of the analysis unit 50 with respect to the mounting table 70 based on the calculated movement amount. For example, if the lens switching unit 46 of the observation unit 40 has multiple observation objective lenses 48, the amount of deviation between the center of view Co of the observation image 81 and the center of view Ca of the analysis image 90 will differ depending on which observation objective lens 48 was used when the observation image 81 was generated. Therefore, it is preferable to calculate in advance the amount of displacement of the center of the field of view for each optical magnification of the observation objective lens 48, and when switching units, correct the amount of movement by the moving unit 60 by the amount of displacement calculated in advance according to the optical magnification of the observation objective lens 48. Note that adjustment of the relative position of the analysis unit 50 with respect to the mounting table 70 in accordance with the optical magnification of the observation objective lens 48 is not limited to when the observation objective lens 48 is switched. For example, when switching the imaging lens or changing the magnification using the zoom lens, the amount of movement by the moving unit 60 may be corrected by the amount of displacement calculated in advance according to the optical magnification of the observation objective lens 48.

[0081] Furthermore, simply correcting the amount of movement by the moving unit 60 by the amount of deviation of the field of view calculated in advance to align the field of view center Co of the observation image 81 with the field of view center Ca of the analysis image 90 may result in misalignment when the relative position of the analysis unit 50 with respect to the mounting table 70 is changed to align each mass region in the next step S15. Therefore, it is preferable that the correspondence between one pixel in the observation image and the actual distance (or the required movement amount) is stored for each optical magnification of the observation objective lens 48, and the movement amount by the moving unit 60 is calibrated based on this correspondence.

[0082] When the unit switching from the observation unit 40 to the analysis unit 50 is completed, next in step S15 of FIG. 7, the movement unit 60 aligns the analysis unit 50 and the mounting table 70 according to the analysis order determined by the order determination unit 36.

[0083] That is, the moving unit 60 sequentially changes the relative position of the analysis unit 50 with respect to the mounting table 70 in accordance with the analysis order determined by the order determination unit 36 ​​so that the analysis beam is irradiated onto each of the mass regions. The moving unit 60 first moves the analysis unit 50 or the mounting table 70 so that the relative position of the analysis unit 50 with respect to the mounting table 70 corresponds to the first mass region in the analysis order. Then, once the analysis of the first mass region has been performed, the moving unit 60 next moves the analysis unit 50 or the mounting table 70 so that the analysis unit 50 or the mounting table 70 corresponds to the second mass region in the analysis order. The moving unit 60 repeats this process sequentially in accordance with the analysis order.

[0084] The moving unit 60 may change the irradiation point of the analysis beam by rotating an optical element such as the analysis half mirror 54 or by using a deflector that deflects the analysis beam, without changing the relative position of the analysis unit 50 with respect to the mounting table 70. Furthermore, based on a determination of whether a process of changing the irradiation point by rotating an optical element or polarizing the analysis beam is more advantageous than movement by the moving unit 60, both processes may be used by switching between them. The determination of whether a process of changing the irradiation point without movement is more advantageous than movement by the moving unit 60 may be made based on, for example, the time required for execution or the analytical accuracy.

[0085] Once the alignment of the analysis unit 50 and the mounting table 70 by the moving unit 60 is completed, analysis of the mass region is then performed by the analysis unit 50 in step S16 of Figure 7. Specifically, an analysis beam is irradiated onto the mass region by the analysis beam device 53, the detector 55 detects the reaction of the mass region (part of the observation object OB) irradiated with the analysis beam, and the mass region is analyzed based on the detection result of the detector 55.

[0086] Various analytical techniques may be employed in the analysis of the lump region by the analysis unit 50. For example, when analysis is performed using LIBS (Laser-Induced Breakdown Spectroscopy), the analysis unit 50 uses a high-energy laser as an analysis beam to locally plasmatize the lump region with the analysis beam, and performs analysis based on the spectrum of light generated by the plasmatized lump region. The plasmatized lump region emits electromagnetic waves with wavelengths corresponding to the respective elemental components of the lump region and the content of each component. The detector 55 detects energy emitted from or absorbed by the observation object OB based on the electromagnetic waves generated from the plasmatized lump region. Although analysis using LIBS is a so-called destructive test, it can identify the respective elemental components contained in the lump region and their content.

[0087] Furthermore, when analysis is performed using LIBS, it is preferable that the analysis unit 50 captures an image of the observation object OB using the analysis camera 52 to generate an analysis image before the mass region is irradiated with the analysis beam. Because analysis using LIBS is a destructive test, generating an analysis image before the analysis beam is irradiated makes it possible to confirm, after analysis, the state of the mass region before the analysis beam caused irreversible changes to the mass region.

[0088] Furthermore, when analysis is performed by LIBS, it is preferable that the analysis unit 50 also generates an analysis image by the analysis camera 52 after the analysis beam is irradiated onto the mass region. By generating an analysis image after the analysis beam is irradiated, it becomes possible to compare the state before and after irreversible changes occur in the mass region due to the analysis beam.

[0089] Furthermore, an analysis image generated by imaging the observation object OB with the analysis camera 52 before the lump region is irradiated with the analysis beam, an analysis image generated by imaging the observation object OB with the analysis camera 52 after the lump region is irradiated with the analysis beam, and the analysis results of the lump region (for example, each component of the elements contained in the lump region and their content) may be associated and stored in the storage unit 32. This makes it easy to compare the analysis results of a given lump region with the analysis images before and after the analysis.

[0090] The analysis unit 50 can also analyze the agglomerate region using analytical techniques other than LIBS. For example, the analysis unit 50 may perform analysis using SEM / EDS (Scanning Electron Microscope / Energy Dispersive X-ray Spectroscopy). In SEM / EDS, the analysis unit 50 irradiates an electron beam as an analysis beam and performs analysis based on electrons or characteristic X-rays emitted from the agglomerate region in response to the electron beam irradiation. The electrons irradiated from the agglomerate region in response to the electron beam irradiation are electrons from atoms in the agglomerate region that have been ejected by the electron beam of the analysis beam. Note that the characteristic X-rays irradiated from the agglomerate region in response to the electron beam irradiation are X-rays emitted according to the energy difference before and after the electrons move to fill the vacancies created by the ejection of electrons from atoms in the agglomerate region. The detector 55 detects the energy emitted from or absorbed by the observation object OB based on the electrons or characteristic X-rays generated from the agglomerate region.

[0091] The analysis unit 50 may also perform analysis using the XRF (X-ray fluorescence) method. In the XRF method, the analysis unit 50 irradiates X-rays as an analysis beam and performs analysis based on characteristic X-rays irradiated from the bulk region in response to the X-ray irradiation. Similar to the SEM / EDS method, the characteristic X-rays irradiated from the bulk region in response to the X-ray irradiation are X-rays emitted according to the energy difference before and after the movement of electrons that were at a higher energy level when they move to fill vacancies created by electrons being ejected by the X-ray irradiation. The detector 55 detects the energy emitted from or absorbed by the observation object OB based on the characteristic X-rays generated from the bulk region.

[0092] The analysis unit 50 may also perform analysis using Raman spectroscopy. In Raman spectroscopy, the analysis unit 50 performs analysis based on Raman scattered light generated from the mass region at a wavelength different from that of the incident light irradiated as an analysis beam. The scattered light generated when the incident light collides with molecules of the substance being analyzed is mostly light with the same wavelength as the incident light (Rayleigh scattered light), but also contains a small amount of light with a different wavelength from that of the incident light (Raman scattered light). Since Raman scattered light is generated by an interaction between the incident light and the substance being analyzed, the molecular structure of the object being analyzed can be analyzed by examining the peaks and spectrum of the Raman scattered light. The detector 55 detects energy emitted from or absorbed by the object being observed OB based on the Raman scattered light generated from the mass region.

[0093] The analysis unit 50 may perform analysis by infrared spectroscopy. In infrared spectroscopy, the analysis unit 50 irradiates infrared light as an analysis beam and performs analysis based on the light transmitted through or reflected from the mass region. By comparing the light transmitted through or reflected from the analysis target with the irradiated infrared light, the wavelength of the infrared light absorbed by the analysis target can be determined, allowing the absorption wavelength spectrum of the substance for infrared light to be analyzed.

[0094] The analysis unit 50 may also perform an analysis utilizing the photothermal effect. In an analysis utilizing the photothermal effect, the analysis unit 50 uses, for example, an analysis beam that thermally expands the mass region, such as infrared light, and performs an analysis based on the thermal expansion of the mass region due to irradiation of the analysis beam. The degree to which infrared light is absorbed by the substance being analyzed depends on the wavelength of the infrared light. The greater the degree to which infrared light is absorbed by the substance, the higher the temperature of the substance. Therefore, by examining the wavelength of infrared light and changes in characteristics such as thermal expansion that correlate with the temperature increase of the substance corresponding to each wavelength based on the displacement occurring in the observation object OB, the absorption wavelength spectrum of the substance for infrared light can be analyzed. Furthermore, in an analysis utilizing the photothermal effect, the analysis unit 50 may irradiate the mass region with infrared light and a refraction detection laser as an analysis beam and perform an analysis based on changes in the refractive index of the refraction detection laser due to changes in the temperature of the mass region due to irradiation of the infrared light. When the mass region being analyzed absorbs infrared light, the refractive index changes due to the increase in temperature. Therefore, by examining the change in the refractive index of the refraction detection laser (the change in the refractive index of the mass region due to irradiation with infrared light) using a refractometer, the absorption wavelength spectrum of the object of analysis for infrared light can be analyzed. Note that examining (observing) phenomena (thermal expansion, etc.) caused by temperature changes in the mass region corresponds to the detector 55 detecting the energy absorbed by the object of observation OB due to irradiation with the analysis beam. In analysis using the photothermal effect, the detector 55 functions as a unit that observes changes in the properties of the mass region, such as thermal expansion, based on an image of the mass region included in the analysis image captured by the analysis camera 52.

[0095] When performing analysis using the analysis beam, the analysis unit 50 captures an image of the observation object OB using the analysis camera 52 to generate an analysis image, and controls the analysis focus control unit 51 to focus the analysis camera 52 on the mass region of the analysis target, and then irradiates the analysis beam using the analysis beam device 53. Since the analysis optical axis Aa of the analysis camera 52 is coaxial with the analysis beam, when the focus of the analysis camera 52 is on the mass region of the analysis target, the analysis beam is irradiated most efficiently onto the mass region.

[0096] In order to reliably irradiate the mass region with the analysis beam, the analysis unit 50 preferably fine-tunes the irradiation point of the analysis beam based on the observation image generated by the observation camera 42 or the analysis image generated by the analysis camera 52. For example, the analysis unit 50 may irradiate the analysis beam with the geometric center of gravity of the mass region, which is calculated based on the observation image.

[0097] In many cases, the geometric center of gravity of a mass region is located inside the mass region, so irradiating the analysis beam onto the geometric center of gravity of the mass region ensures that the analysis beam hits the mass region. If the irradiation point of the analysis beam does not match the geometric center of gravity of the mass region, the analysis unit 50 changes the irradiation point of the analysis beam by changing the relative position of the mounting table 70 and the analysis unit 50 using the movement unit 60. Note that the analysis unit 50 may change the irradiation point of the analysis beam without changing the relative position of the mounting table 70 and the analysis unit 50 using the movement unit 60, for example, by rotating an optical element such as the analysis half mirror 54 or by using a deflector that deflects the analysis beam.

[0098] The analysis unit 50 may also determine the irradiation point of the analysis beam using a thinning process. FIG. 10 illustrates a case in which the analysis unit 50 determines the irradiation point of the analysis beam using a thinning process. A cluster region in the observation image 81 is referred to as a cluster region 85. A thinned region 85a is obtained by performing a thinning process on the cluster region 85. The thinning process involves shrinking the pixels of the cluster region 85 one pixel at a time from the periphery, and creating the thinned region 85a from the continuum of pixels equivalent to one pixel remaining. The analysis unit 50 irradiates the analysis beam onto a point (thinned end 92) corresponding to the end of the thinned region 85a thus obtained. As shown in FIG. 10, the thinned end 92 is located inside the cluster region 85 relative to the point (cluster region end 91) where the search path Sc first intersects with the cluster region 85 when searching for the cluster region 85 from the edge of the observation image 81. Therefore, by determining the irradiation point of the analysis beam using the thinning process, the analysis unit 50 can reliably apply the analysis beam to the mass region 85.

[0099] Furthermore, depending on the shape of the mass region 85, the geometric center of gravity 85g of the mass region 85 may be outside the mass region 85. As shown in Figure 11, when the geometric center of gravity 85g of the mass region 85 is outside the mass region 85 in the observation image 81, the analysis unit 50 may irradiate the analysis beam from the geometric center of gravity 85g of the mass region 85 to the nearest edge 93 of the mass region 85 along a first direction in the analysis image 90. That is, rather than setting the center of gravity 85g of the mass region 85 as the irradiation point of the analysis beam, the analysis unit 50 searches for the mass region 85 from the center of gravity 85g along any direction in the observation image 81 (horizontal direction, X direction in Figure 10, but Y direction is also acceptable), and sets the point of the mass region 85 closest to the center of gravity 85g along the search direction (nearest edge 93) as the irradiation point of the analysis beam. In this way, even if the geometric center of gravity 85 g of the mass region 85 is outside the mass region 85 , the analysis unit 50 can irradiate the analysis beam onto the mass region 85 .

[0100] 11 , the analysis unit 50 may search for a nearest end 93 of the mass region 85 and a far end 95, which is the outer edge of the mass region 85, further along a first direction (here, the X direction) from the nearest end 93, and irradiate the analysis beam onto a central portion 94 between the nearest end 93 and the far end 95. By irradiating the central portion 94 with the analysis beam, the analysis unit 50 can direct the analysis beam onto a location further inside the mass region 85.

[0101] Furthermore, the analysis unit 50 determines whether the clump region 85 is a fiber based on the observed image 81, and if the clump region 85 is a fiber, it may irradiate the analysis beam onto a portion that corresponds to the end of the clump region 85 when thinned (fiber end 85e), or onto the nearest end 93, or onto a central portion 94 between the nearest end 93 and the far end 95. Figure 12 shows a case where the clump region 85 is a fiber. The determination of whether the clump region 85 is a fiber is performed by the following procedure.

[0102] First, the analysis unit 50 uses image processing to linearly stretch the clump region 85, which is being determined to be a fiber, and measures the stretched length L. The analysis unit 50 then measures the maximum inner diameter R of the linearly stretched clump region 85. If the ratio of L to R exceeds 20 (L / R>20) and the maximum inner diameter R is 50 μm or less, the clump region 85 is determined to be a fiber.

[0103] If the clump region 85 is determined to be fiber, the analysis unit 50 may perform a thinning process on the clump region 85 and irradiate the analysis beam onto the end of the thinned clump region 85 (fiber end 85e). By irradiating the fiber end 85e with the analysis beam, the analysis unit 50 can apply the analysis beam even if the clump region 85 is fiber. Alternatively, the analysis unit 50 may determine the geometric center of gravity 85g of the clump region 85 determined to be fiber, search for the closest end 93 from the center of gravity 85g, as in the case shown in FIG. 11, and set the closest end 93 as the irradiation point. Alternatively, the analysis unit 50 may search for the closest end 93 and the farthest end 95 from the center of gravity 85g, and set the central portion 94 between them as the irradiation point. In FIG. 12, the nearest end portion 93, the central portion 94, and the far end portion 95 are located very close to each other, and are therefore shown as being in the same position. However, in reality, as shown in FIG. 11, the nearest end portion 93, the central portion 94, and the far end portion 95 are located in different positions.

[0104] 7, the analysis unit 50 performs the above-described analysis for each of the mass regions. Then, in step S17, it is determined whether or not analysis has been completed for all of the mass regions for which the analysis order has been determined by the order determination unit 36. If analysis has not been completed for all of the mass regions (NO in step S17), the observation device 10 returns to step S15 and aligns the analysis unit 50 with the mounting table 70 for the mass region that is next in the analysis order.

[0105] In step S15, the moving unit 60 may accept a correction operation to correct the relative position of the analysis unit 50. In particular, it is preferable that the correction operation be accepted when the moving unit 60 sets the relative position of the analysis unit 50 with respect to the mounting table 70 to a position corresponding to the first mass region in the analysis order.

[0106] 13, when the moving unit 60 adjusts the relative position of the analysis unit 50 with respect to the mounting table 70 to a position corresponding to the first mass region 85 in the analysis sequence, the position of the mass region 85 in the analysis image 90 may not coincide with the center of view Ca of the analysis image 90. Even if the amount of movement by the moving unit 60 is calibrated, such a deviation may occur due to mechanical looseness of the mechanism, etc. Therefore, when the moving unit 60 adjusts the relative position of the analysis unit 50 with respect to the mounting table 70 to a position corresponding to the first mass region 85 in the analysis sequence, the analysis image 90 is displayed on the display unit 12, and the user's correction operation is accepted.

[0107] If the user confirms that the position of the mass region 85 in the analysis image 90 does not coincide with the center of field of view Ca of the analysis image 90, the user performs a correction operation via the operation unit 14 and changes the relative position of the analysis unit 50 with respect to the mounting table 70 so that the center of field of view Ca of the analysis image 90 coincides with the position of the mass region 85.

[0108] FIG. 14 is a diagram showing a case where the moving unit 60 has received a correction operation from the user. The moving unit 60 stores the amount of change in the relative position of the analysis unit 50 corrected by the correction operation. In FIG. 14, the position of the block region 85 in the analysis image 90 has been corrected by the correction operation from the pre-correction position 85x by ΔX in the X direction and ΔY in the Y direction. The information processing unit 30 stores the amount of displacement corrected by this correction operation in the memory unit 32.

[0109] The moving unit 60 corrects the relative position of the analysis unit 50 corresponding to the second or subsequent mass regions in the analysis order based on the displacement amount corrected by the correction operation and stored in the memory unit 32. That is, after the analysis of the first mass region 85 in the analysis order is completed in step S16 of FIG. 7, the process returns to step S15, and when the relative position of the analysis unit 50 with respect to the mounting table 70 is aligned to a position corresponding to the second or subsequent mass region in the analysis order, the moving unit 60 corrects the relative position of the analysis unit 50 based on the stored displacement amount (ΔX in the X direction, ΔY in the Y direction). This allows the correction received by the correction operation to be reflected in the alignment of all mass regions included in the analysis order.

[0110] In the above example, when the moving unit 60 adjusts the relative position of the analysis unit 50 with respect to the mounting table 70 to a position corresponding to the first mass region 85 in the analysis order during the alignment in step S15, if the position of the mass region 85 in the analysis image 90 does not coincide with the center of view Ca of the analysis image 90, an operation to align the center of view Ca with the mass region 85 is accepted as a correction operation by the user. However, the user's designation of an arbitrary mass region 85 may also be accepted as a correction operation. In this case, the information processing unit 30 accepts the designation of the arbitrary mass region 85 as a correction operation to correct the relative position of the analysis unit 50 with respect to the mounting table 70 so that the relative position of the analysis unit 50 corresponds to the predetermined (specified) mass region 85. When the arbitrary mass region 85 is designated, the moving unit 60 changes the relative position of the analysis unit 50 with respect to the mounting table 70 so that the center of view Ca of the analysis image 90 coincides with the position of the (specified) mass region 85 designated by the user. The information processing unit 30 stores the amount of displacement of the relative position at this time as a corrected amount of displacement (stores it in the memory unit 32). Then, when the relative position of the analysis unit 50 with respect to the mounting table 70 is aligned with the position corresponding to each mass region, the moving unit 60 may correct the relative position of the analysis unit 50 by the stored amount of displacement.

[0111] The above steps are repeated until the analysis of all mass regions is completed (YES in step S17 in FIG. 7), at which point the observation device 10 proceeds to step S18 and displays a result screen 13 on the display unit 12. An example of the result screen 13 is shown in FIG. 15.

[0112] The results of component analysis performed on each lump region are displayed on the results screen 13 in Figure 15. Cases where the lump region is organic matter, aluminum, or stainless steel copper are shown on the results screen 13. For materials that are mixtures of multiple elements, such as stainless steel copper, the components and content of the material are also displayed.

[0113] Furthermore, when a spectral analysis is performed on a lump region, the results of the spectral analysis can also be displayed on the results screen 13. FIG. 16 is a diagram showing an example of the results screen 13 when a spectral analysis is performed. As shown in FIG. 16, for example, if the lump region is brass made of zinc (Zn) and copper (Cu), the results of the spectral analysis on brass are displayed on the results screen 13. The results of the spectral analysis show the spectral characteristics of each of zinc and copper.

[0114] Furthermore, on the results screen 13, as shown in an example in FIG. 17, an analysis list 17 listing the analysis results of each agglomerate region may be created by the information processing unit 30 and displayed on the display unit 12. The analysis list 17 displays a list of component analysis results, such as the size and height of each agglomerate region, a classification selected from categories such as "metal," "fiber," and "other," and component analysis results such as the constituent elements and their content rates. As the analysis unit 50 analyzes the agglomerate region and obtains component analysis results, the item columns in the analysis list 17 showing the component analysis results (e.g., aluminum, stainless steel, copper, etc.) may be sequentially updated. The analysis list 17 in FIG. 17 also displays information about each agglomerate region, such as the number (No.) assigned to each agglomerate region, the file name of the observation or analysis image containing the agglomerate region (here, all agglomerate regions in the list are assumed to be contained in the same image file), and the average pixel brightness of each agglomerate region, as well as statistical information such as the maximum size of the agglomerate region and the number of agglomerate regions (count).

[0115] The information processing unit 30 may also update the classification of a lump region in response to the component analysis results of the lump region. Specifically, if the component analysis results indicate that a metal is included, the lump region may be classified as "metal." The classification results are not limited to "metal," "fiber," or "other." For example, additional classification items may be added based on user-defined criteria. Specifically, if Fe, Ni, and Cr are included in the component analysis results of the lump region, the lump region may be classified as "iron" based on the ratio of these elements defined by the user. For example, if a lump region contains Fe, a threshold for determining whether or not to classify the lump region as "iron" may be defined by the user as a condition for the ratio of each element. Depending on whether the ratio of each element exceeds the threshold, the information processing unit 30 may classify the lump region as "iron" or "non-ferrous." The analysis list 17 may also allow the user to specify attributes to be displayed. For example, an input field for specifying the display target may be provided, and the input field may allow the user to specify attributes to be displayed, such as "all," "metal," "fiber," or "other," using a pull-down list. In FIG. 17, the display target is specified as "all," so block regions of all attributes are displayed.

[0116] After the result screen 13 is displayed, a determination is made as to whether or not to perform a replay in step S19 of Fig. 7. The information processing unit 30 of the observation device 10 can store the operating conditions of the observation unit 40, the analysis unit 50, and the unit movement unit 60, and can cause the observation unit 40 and the analysis unit 50 to generate an observation image of the observation object OB and re-execute the analysis by the analysis unit 50 under the same operating conditions as the stored conditions.

[0117] For example, the operating conditions referenced in the replay include extraction conditions (brightness threshold, size range considered to be a mass region, etc.) for the mass region identifying unit 35 to identify a mass region from the observed image, and also include output conditions (beam type, beam wavelength, output intensity, etc.) for the analysis beam irradiated by the analysis unit 50.

[0118] In addition, the operating conditions referenced in the replay can include all of the contents that can be set arbitrarily by the user, such as the lighting conditions for the observation object OB by the observation unit 40, the shutter speed of the observation camera 42 and the analysis camera 52, etc.

[0119] If replay is not to be performed (NO in step S19), the observation device 10 completes the observation and analysis and waits for the next user operation. On the other hand, if replay is to be performed (YES in step S19), the observation device 10 proceeds to step S20, where the generation of the observed image and the analysis by the analysis unit 50 are re-executed in accordance with the stored operating conditions.

[0120] When re-executing observation and analysis by replay, an observation object OB different from the observation object OB that was the subject of the first observation and analysis may be placed on the mounting table 70. Even if the observation object OB placed on the mounting table 70 is different from the first observation object OB, the observation device 10 can identify mass regions contained in the new observation object OB based on the extraction conditions included in the stored operating conditions, and can appropriately analyze each mass region. [Explanation of symbols]

[0121] 10 Observation equipment 20 Imaging unit 30 Information Processing Department 34 Imaging control unit 35 Mass area identification part 36 Order determining part 37 Analysis Execution Department 40 Observation section 42 Observation Camera 48 Observation Objective Lens 50 Analysis Department 52 Analysis Camera 53 Analytical beam instrument 55 detector 58 Analytical Objective Lens 60 Moving Part 70 Mounting table OB Observation object

Claims

1. An observation device for observing and analyzing an observation object, a mounting table on which the observation object is placed; an observation unit that generates an observation image of the observation object; an analysis unit that analyzes the object to be observed; an information processing unit that communicates with the observation unit and the analysis unit; Equipped with the observation unit includes an observation objective lens that receives light from the observation object, and an observation camera that generates the observation image by capturing an image of the observation object through the observation objective lens, the analysis unit includes an analysis beam device that emits an analysis beam to the observation object, and a detector that detects energy absorbed by or emitted from the observation object due to irradiation with the analysis beam, The information processing unit an imaging control unit that controls the observation camera of the observation unit to capture an image of the observation object, thereby generating the observation image; a mass region identifying unit that identifies a plurality of mass regions included in the observation image; an analysis order determination unit that determines an analysis order for analyzing the mass region; an analysis execution unit that causes the analysis beam device of the analysis unit to sequentially irradiate each of the mass regions with the analysis beam in accordance with the analysis order determined by the order determination unit, and causes the analysis unit to analyze the mass regions based on the detection results of the detector; An observation device having:

2. a moving unit that changes the relative positions of the observation unit and the analysis unit with respect to the mounting table, 2. The observation device according to claim 1, wherein the moving unit changes the relative position of the analysis unit with respect to the mounting table so that, after the observation image is generated by the observation unit, the analysis beam is irradiated onto a location of the observation object where the observation image was captured by the observation camera.

3. 3. The observation device according to claim 2, wherein, when changing the relative position of the analysis unit with respect to the mounting table so that the analysis beam is irradiated onto the location where the observation image was captured, the movement unit calculates a movement amount corresponding to an optical magnification of the observation objective lens of the observation unit when the observation image was generated, and adjusts the relative position of the analysis unit with respect to the mounting table based on the calculated movement amount.

4. 3. The observation device according to claim 2, wherein the moving unit sequentially changes the relative position of the analysis unit with respect to the mounting table in accordance with the analysis order determined by the order determination unit so that the analysis beam is irradiated onto each of the mass regions.

5. the information processing unit receives a correction operation for correcting the relative position of the analysis unit with respect to the mounting table so that the relative position of the analysis unit is a position corresponding to the predetermined mass region, and stores the amount of displacement corrected by the correction operation; 5. The observation device according to claim 4, wherein the moving unit corrects the relative position of the analysis unit with respect to the mounting table based on the displacement amount stored by the information processing unit when sequentially changing the relative position of the analysis unit with respect to the mounting table in accordance with the analysis order.

6. The observation device according to claim 1 , wherein the analysis unit locally converts the mass region into plasma using the analysis beam, detects light generated from the mass region by converting the mass into plasma using a detector, and performs analysis based on the spectrum of the light.

7. The observation device described in claim 6, wherein the analysis unit further includes an analysis camera that captures the object to be observed and generates an analysis image, and the analysis camera generates the analysis image before the analysis beam is irradiated onto the mass region, and also generates the analysis image after the analysis beam is irradiated onto the mass region, and associates and saves the results of the analysis of the mass region with the analysis image before irradiation of the analysis beam and the analysis image after irradiation of the analysis beam.

8. 2. The observation device according to claim 1, wherein the analysis unit irradiates an electron beam as the analysis beam, and performs analysis based on electrons or characteristic X-rays emitted from the mass region in response to the irradiation of the electron beam.

9. The observation device according to claim 1 , wherein the analysis unit irradiates X-rays as the analysis beam and performs analysis based on characteristic X-rays emitted from the mass region in response to the X-ray irradiation.

10. The observation device according to claim 1 , wherein the analysis unit performs analysis based on Raman scattered light generated from the mass region at a wavelength different from that of the incident light irradiated as the analysis beam.

11. The observation device according to claim 1 , wherein the analysis unit irradiates infrared light as the analysis beam and performs analysis based on light transmitted through or reflected by the mass region.

12. The observation device according to claim 1 , wherein the analysis unit irradiates infrared light as the analysis beam and performs an analysis based on a change in the refractive index of the mass region due to the irradiation of the infrared light, or an analysis based on thermal expansion of the mass region due to the irradiation of the infrared light.

13. The observation device according to claim 1 , wherein the analysis unit further includes an analysis camera that captures an image of the observation object to generate an analysis image, and irradiates the analysis beam with the focus of the analysis camera set on the mass region.

14. The observation device according to claim 1 , wherein the analysis unit irradiates the analysis beam onto a geometric center of gravity of the mass region calculated based on the observation image.

15. 2. The observation device according to claim 1, wherein the analysis unit determines whether the mass region is a fiber based on the observation image, and if the mass region is a fiber, irradiates the analysis beam onto a location that corresponds to an end of the mass region when thinned.

16. a moving unit that changes the relative positions of the observation unit and the analysis unit with respect to the mounting table, the imaging control unit causes the observation camera to capture images of a plurality of locations on the observation object while changing the relative position of the observation unit with respect to the mounting table using the moving unit, thereby generating a plurality of the observation images and generating a concatenated image by concatenating the plurality of the observation images; The observation device according to claim 1 , wherein the mass region identifying section identifies a plurality of the mass regions included in the connected image.

17. 2. The observation device according to claim 1, wherein the mass region identifying section receives a specification of extraction conditions for identifying the mass region from the observation image, and identifies a region in the observation image that matches the extraction conditions as the mass region.

18. The observation device according to claim 1 , wherein the order determination unit determines the analysis order in the order in which the mass regions are found by searching the observation image from one end to the other.

19. the mass region identifying unit identifies the size of each of the plurality of mass regions; The observation device according to claim 1 , wherein the order determination unit determines the mass region to be analyzed and the order of analysis of the mass region to be analyzed based on the size of the mass region.

20. a Z-direction stage that changes the height of the observation unit relative to the mounting table, the imaging control unit generates the plurality of observation images using the observation unit while changing the height of the observation unit using the Z-direction stage; the information processing unit generates a three-dimensional image of the observation object based on a plurality of the observation images generated by the observation unit at different heights relative to the stage, the mass region identifying unit identifies the height of each of the mass regions based on the three-dimensional image; The observation device according to claim 19 , wherein the order determination unit determines the analysis order based on the size and height of the mass region.

21. The observation device according to claim 19 , wherein the order determination unit determines the analysis order in descending order of size of the mass regions.

22. The observation device according to claim 1 , wherein the order determination unit receives a designation of a priority order for the mass regions, and determines the analysis order based on the designation of the priority order.

23. the sequence determination unit divides the analysis sequence into a first sequence and a second sequence that is executed subsequent to the first sequence; In the first sequence, the analysis order is determined based on the designation of the priority; The observation device according to claim 22 , wherein the second sequence determines the analysis order for the mass regions that are not designated in the designation of the priority order.

24. the block region identifying unit classifies the plurality of block regions into two or more attributes, 2. The observation device according to claim 1, wherein the order determination unit determines the attributes of each agglomerate region classified by the agglomerate region identification unit, and determines the analysis order for the agglomerate regions to be analyzed based on the determination results.

25. 2. The observation device according to claim 1, wherein the information processing unit is capable of storing operating conditions of the observation unit and the analysis unit, and is capable of causing the observation unit and the analysis unit to re-execute the generation of the observation image of the observation object and the analysis by the analysis unit under the same conditions as the stored operating conditions.

26. 26. The observation device according to claim 25, wherein the operating conditions include extraction conditions for the mass region specifying unit to specify the mass region from the observation image.

27. 26. The observation device according to claim 25, wherein the operating conditions include an output condition of the analysis beam irradiated by the analysis unit.

Images