Amplification observation device and image processing device for amplification observation device

Abstract

The invention provides a magnification observation device with high convenience and an image processing device for the magnification observation device. An analysis processing unit of the magnification observation device performs specifying processing for specifying an observation image to be an object of image analysis processing, and binarization processing for extracting block regions satisfying conditions. A shaping process for shaping the extracted block region; and a measurement process for measuring a feature amount of the shaped block region. The display control unit generates a result display screen corresponding to the measurement process and a setting screen corresponding to each of the designation process, the binarization process, and the shaping process. The display control unit generates, as a setting screen corresponding to the analysis processing, a screen in which setting items that can be accepted sequentially change in accordance with the processing being executed by the analysis processing unit. A result display screen is generated as a result display screen in which the feature amount measured by the measurement processing performed by the analysis processing unit is displayed as a measurement result.

Description

Technical Field

[0001] The present invention relates to a magnifying observation device and an image processing device used for the magnifying observation device. Background Art

[0002] In order to magnify and observe the observation object, a magnifying observation device can be used. In the magnifying observation device of Patent Document 1, an image based on image data provided by a microscope is displayed on a display unit. The image displayed on the display unit is binarized using a brightness threshold (critical value) to extract multiple regions from the image. Then, the extracted region is processed to change the state of the region.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2013-88997 Summary of the invention

[0006] Problem that the invention aims to solve

[0007] However, in the magnification observation device of Patent Document 1, when the threshold is reset, the extraction of the region is canceled. Therefore, when the user resets the threshold after processing, the extraction of the region after the processing is also canceled, so the user needs to perform the processing work performed previously again after resetting the threshold, which is problematic in terms of convenience.

[0008] In view of the above-mentioned problems, an object of the present invention is to provide a magnifying observation device and an image processing device for the magnifying observation device with high convenience.

[0009] Solutions for solving problems

[0010] In order to solve the above-mentioned problems, an enlarged observation device as an example of an embodiment of the present invention comprises: an imaging element that captures an observation object; an image generating unit that controls the imaging element to generate an observation image; a stage for placing the observation object; a stage control unit that moves the stage in a horizontal direction; a movement control unit that controls the movement performed by the stage control unit; an analysis processing unit that performs image analysis processing on the observation image; and a display control unit that generates a display screen based on the observation image, wherein the image analysis processing performed by the analysis processing unit includes a designated processing, a binarization processing, a shaping processing, and a measurement processing, and the display control unit generates a result display screen corresponding to the measurement processing and a setting screen corresponding to the designated processing, the binarization processing, and the shaping processing, respectively. In the designated processing, the analysis processing unit designates an observation image that is the object of the image analysis processing. In the binarization processing, the analysis processing unit binarizes the observation image based on the set conditions to extract a block area that satisfies the conditions. In the shaping processing, the analysis processing unit shapes the block area extracted by the binarization processing. In the measurement processing, the analysis processing unit measures the feature value of the block area shaped by the shaping processing. The display control unit generates a screen that sequentially changes to the setting items that can be accepted according to the processing being performed by the analysis processing unit as a setting screen corresponding to the designated processing, the binarization processing, and the shaping processing, respectively. The display control unit generates a screen that displays the feature value measured by the measurement processing of the analysis processing unit as a measurement result as a result display screen.

[0011] In addition, as another example of the embodiment involved in the present invention, an image processing device for a magnifying observation device is that the magnifying observation device includes: a camera element, which captures an observation object; a stage, which is used to place the observation object; and a stage control unit, which moves the stage in a horizontal direction, and the image processing device includes: an image generating unit, which controls the camera element to generate an observation image; a movement control unit, which controls the movement performed by the stage control unit; an analysis processing unit, which performs image analysis processing on the observation image; and a display control unit, which generates a display screen based on the observation image, wherein the image analysis processing performed by the analysis processing unit includes a designated processing, a binarization processing, a shaping processing, and a measurement processing, and the display control unit generates a result display screen corresponding to the measurement processing and a setting screen corresponding to the designated processing, the binarization processing, and the shaping processing, respectively, and in the designated processing The analysis processing unit specifies an observation image to be the object of image analysis processing. In the binarization processing, the analysis processing unit binarizes the observation image based on the set conditions to extract the block area that meets the conditions. In the shaping processing, the analysis processing unit shapes the block area extracted by the binarization processing. In the measurement processing, the analysis processing unit measures the feature amount of the block area shaped by the shaping processing. The display control unit generates a screen that sequentially changes to the setting items that can be accepted according to the processing being performed by the analysis processing unit as a setting screen corresponding to the specified processing, the binarization processing, and the shaping processing, respectively. The display control unit generates a screen that displays the feature amount measured by the measurement processing performed by the analysis processing unit as a measurement result as a result display screen. The measurement processing is performed by the analysis processing unit based on the setting items accepted via each setting screen.

[0012] Effects of the Invention

[0013] According to the present invention, the setting items that can be accepted in the setting screen related to the binarization processing and the setting screen related to the shaping processing are sequentially changed, so that the binarization processing and the shaping processing become independent processing, and the convenience of the magnification observation device and the image processing device used for the magnification observation device is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a perspective view of an enlarged imaging unit of the observation device as viewed from the front and above.

[0015] Figure 2 It is a perspective view of the camera unit as viewed from the rear and above.

[0016] Figure 3 It is a right side view of the camera unit.

[0017] Figure 4 This is the front view of the camera unit.

[0018] Figure 5 This is a block diagram schematically showing the structure of the magnification observation device.

[0019] Figure 6 FIG. 1 is a diagram showing a display screen during a designation process.

[0020] Figure 7 It is a figure which shows the display screen in the binarization process.

[0021] Figure 8 FIG. 1 is a diagram showing a display screen during shaping processing.

[0022] Fig. 9 FIG. 1 is a diagram showing a display screen after shaping by shaping processing is applied.

[0023] Fig.10 is a diagram showing a removal confirmation message.

[0024] Fig.11 It is a figure which shows the display screen in the measurement process.

[0025] Fig.12 is a diagram showing a measurement result window.

[0026] Fig.13 is a diagram showing a histogram window.

[0027] Fig.14 is a flowchart showing the flow of the designated processing.

[0028] Fig.15 : is a flowchart showing the flow of binarization processing.

[0029] Fig.16 : is a flowchart showing the flow of shaping processing.

[0030] Fig.17 It is a flowchart which shows the flow of a measurement process. DETAILED DESCRIPTION

[0031] Hereinafter, an enlarged observation device 10 as an example of an embodiment of the present invention will be described with reference to the accompanying drawings. In addition, the same reference figure numerals are marked on the same or equivalent parts in the drawings, so that the description is not repeated. In addition, in the following description, the terms such as front, back, left, right, up, and down are sometimes used to mean positions or directions, but these terms are used for convenience in order to facilitate understanding of the embodiments. These terms are not limited to the front, back, left, right, up, and down in a strict geometric sense unless they are specifically mentioned.

[0032] Figure 1 FIG. 1 is a perspective view of the imaging unit 1 of the magnifying observation device 10 according to the present embodiment, as viewed from the front and above. Figure 1 As shown, the magnification observation device 10 includes an imaging unit 1, an information processing unit 71, a display unit 72, and an operation unit 73. The imaging unit 1 is connected to the information processing unit 71 via a communication cable 9. The information processing unit 71 is connected to the display unit 72 and the operation unit 73 via a communication cable 9.

[0033] Here, the communication cable 9 is a cable for sending and receiving signals between the components of the magnifying observation device 10, that is, between the imaging unit 1, the information processing unit 71, the display unit 72, and the operation unit 73. As the communication cable 9, for example, a USB cable (USB: Universal Serial Bus), a LAN cable (LAN: Local Area Network), an HDMI cable (HDMI: registered trademark, High-Definition Multimedia Interface), etc., which are used for sending and receiving electrical signals, are used. In addition, cables of different specifications may be used according to the connected components. As long as the signal can be sent and received between the imaging unit 1, the information processing unit 71, the display unit 72, and the operation unit 73, the communication cable 9 may also be an optical cable for sending and receiving optical signals. In addition, instead of the communication cable 9, wireless communication based on electromagnetic waves may be used to send and receive signals between the components.

[0034] The information processing unit 71 is a unit including a processor such as a CPU (Central Processing Unit) and a storage device such as a RAM (Random Access Memory), and is used to perform various information processing in the magnifying observation device 10. The information processing unit 71 also functions as an image processing device for the magnifying observation device 10.

[0035] The display unit 72 is a unit including an image display device such as an LCD (Liquid Crystal Display), and displays images to the user of the magnifying observation device 10. The operation unit 73 is a unit for receiving operations performed by the user of the magnifying observation device 10. The operation unit 73 includes, for example, a user interface device such as a keyboard, a mouse, and a joy stick. In addition, the operation unit 73 may also include a controller specially designed for the magnifying observation device 10.

[0036] The magnified observation device 10 is a device for magnifying and observing the observation object OB. First, the observation object OB is photographed by the imaging element 7 of the imaging unit 1 to generate an observation image. Then, a display screen based on the observation image is generated by the information processing unit 71, and the display screen is displayed on the display unit 72. The display screen displayed on the display unit 72 includes an image of the observation object OB that is magnified and displayed. The user of the observation device 10 can magnify and observe the observation object OB by observing the display screen of the display unit 72.

[0037] Reference Figure 1 , Figure 2 , Figure 3 , Figure 4 The imaging unit 1 will be described. Figure 2 It is a perspective view of the imaging unit 1 as seen from the rear upper side. Figure 3 It is a right side view of the imaging unit 1 . Figure 4 1 is a front view of the camera unit 1. Figure 2 , Figure 3 , Figure 4 The information processing unit 71, the display unit 72, and the operation unit 73 are omitted in the figure. Figure 1 , Figure 2 , Figure 3 , Figure 4 The imaging unit 1 includes a mounting table 2 , an observation head 4 , a support column 5 , a Z-direction table 6 , and an imaging element 7 .

[0038] In addition, Figure 1 , Figure 2 , Figure 3 , Figure 4 In the embodiment, a rotating plate 100 is placed on the upper surface 20 of the mounting table 2. Furthermore, an observation object OB is placed on the upper surface 107 of the rotating plate 100. The upper surface 107 of the rotating plate 100 is circular in a plan view, and the observation object OB is placed at the center of the upper surface 107. In addition, radial pins 109 are radially arranged on the circumference of the rotating plate 100. The user of the magnifying observation device 10 can change the arrangement angle of the observation object OB placed at the center of the upper surface 107 by holding the radial pins 109 to rotate the rotating plate 100.

[0039] The stage 2 is supported by the stage control unit 23. The stage control unit 23 moves the stage 2 in the X direction and the Y direction (horizontal direction). The stage control unit 23 in the figure includes a movable stage 21 and a base unit 22. The base unit 22 is fixed, and the movable stage 21 is supported on the base unit 22. The movable stage 21 can move in the X direction and the Y direction relative to the base unit 22. By moving the movable stage 21 in the X direction and the Y direction, the stage 2 is moved in the X direction and the Y direction. The stage control unit 23 can change the range in which the observation object OB is photographed by the imaging element 7 by moving the stage 2 in the X direction and the Y direction.

[0040] The observation head 4 is arranged above the mounting table 2. The observation head 4 includes an image pickup element 7 for photographing the observation object OB. The image pickup element 7 is an element for converting reflected light or transmitted light from the observation object OB into an electrical signal, and for example, an image sensor using a CCD (Charge Coupled Device) or an image sensor using a CMOS (Complementary Metal Oxide Semiconductor) is used.

[0041] The observation head 4 is mounted on the Z-direction stage 6. The Z-direction stage 6 moves the observation head 4 along the Z direction (vertical direction) relative to the upper surface 20 of the mounting table 2. Moreover, the observation head 4 can be rotated by the Z-direction stage 6 around the first rotation axis 11 extending along the Y direction (horizontal direction).

[0042] A plurality of objective lenses 40 and a revolver 41 are provided at the lower portion of the observation head 4. The magnifications of the plurality of objective lenses 40 are different. Any one of the plurality of objective lenses 40 is selected as a lens for observing the observation object OB, and the objective lens is directed toward the observation object OB. By rotating the revolver 41, the objective lens 40 directed toward the observation object OB is switched.

[0043] The Z-direction stage 6 is mounted on the rotating holding body 8. Moreover, the rotating holding body 8 is connected to the support column 5 in a manner that it can rotate around the first rotation axis 11. The support column 5 extends in the Z direction and supports the rotating holding body 8, the Z-direction stage 6, and the observation head 4. In addition, a rod-shaped handle 81 protruding to the outside is provided on the rotating holding body 8. The user of the magnifying observation device 10 can rotate the observation head 4 together with the Z-direction stage 6 and the rotating holding body 8 around the first rotation axis 11 by operating the rod-shaped handle 81. By rotating the observation head 4, the user can observe the observation object OB not only from a vertical direction but also from an inclined direction.

[0044] Reference Figure 5 The structure of the magnifying observation device 10 will be described. Figure 5 1 is a block diagram schematically showing an example of the structure of the magnifying observation device 10. Figure 5 As shown, the observation head 4 of the imaging unit 1 includes a rotator control unit 45 , an imaging lens 240 , a lens control unit 241 , an illumination unit 250 , and an aperture control unit 260 in addition to the imaging element 7 and the rotator 41 .

[0045] The rotator control unit 45 rotates the rotator 41 to select one of the plurality of objective lenses 40 as a lens for observing the observation object OB. The imaging lens 240 forms an image of the light from the objective lens 40 that has passed through the rotator 41 into an image to be captured by the imaging element 7 .

[0046] The lens control unit 241 moves the imaging lens 240 to change the relative position with the objective lens 40 , thereby changing optical conditions such as magnification and focal length. In addition, various optical elements such as lenses and half mirrors may be further arranged in the optical path between the objective lens 40 and the imaging lens 240 .

[0047] The illumination unit 250 irradiates light to the observation object OB to obtain reflected light or transmitted light from the observation object OB. The illumination unit 250 includes, for example, annular illumination and coaxial epi-illumination. The aperture control unit 260 controls an aperture (not shown) provided in the optical path between the objective lens 40 and the imaging lens 240 to adjust the aperture amount.

[0048] The information processing unit 71 includes a calculation unit 300 and a storage unit 311. The calculation unit 300 includes an image generation unit 301, a movement control unit 302, an analysis processing unit 303, a display control unit 304, an illumination control unit 305, and an input / output control unit 306.

[0049] The operation unit 300 is a unit including a processor such as a CPU. The storage unit 311 is a unit including a storage device such as a RAM. The operation unit 300 executes the program stored in the storage unit 311 to realize the functions of the image generation unit 301, the movement control unit 302, the analysis processing unit 303, the display control unit 304, the lighting control unit 305, and the input / output control unit 306.

[0050] The image generation unit 301 controls the imaging element 7 to generate an observation image based on reflected light or transmitted light from the observation object OB. The movement control unit 302 sends a command to the stage control unit 23 of the imaging unit 1 to control the movement of the stage 2 by the stage control unit 23 .

[0051] The analysis processing unit 303 performs image analysis processing on the observation image generated by the image generating unit 301. The image analysis processing includes a designation process, a binarization process, a shaping process, and a measurement process, which will be described later.

[0052] The display control unit 304 generates a display screen to be displayed on the display unit 72 based on the observed image. In particular, the display control unit 304 generates a result display screen corresponding to the measurement processing performed by the analysis processing unit 303 and a setting screen corresponding to the designated processing, binarization processing, and shaping processing performed by the analysis processing unit 303.

[0053] The lighting control unit 305 controls the lighting unit 250 to adjust the light irradiated to the observation object OB. The input / output control unit 306 is a communication interface unit that controls the input and output signals to and from the information processing unit 71 according to various communication standards so that the information processing unit 71 can communicate with external devices such as the imaging unit 1, the display unit 72, and the operation unit 73.

[0054] When describing the general operation of the magnified observation device 10, first, the imaging element 7 of the imaging unit 1 magnifies and photographs the observation object OB according to the operation of the operation unit 73 by the user. The analysis processing unit 303 of the information processing unit 71 performs image analysis processing on the observation image obtained by the shooting. The display control unit 304 generates a display image based on the result of the image analysis processing and the observation image. The generated display image is displayed on the display unit 72. By confirming the display image displayed on the display unit 72, the user can magnify and observe the observation object OB.

[0055] The magnifying observation device 10 can assist the user in performing measurement operations on the observation object OB through image analysis. As an example, the measurement of graphite cast iron, especially the measurement of the spheroidization rate in ductile iron, is described below, but the magnifying observation device 10 can also be used for various operations. For example, in operations related to steel, the magnifying observation device 10 can also be used for the measurement of the content of non-metallic inclusions, the measurement of grain size, the measurement of DAS (Dendrite Arm Spacing, dendrite spacing), etc.

[0056] In the measurement of ductile iron, the distribution ratio of graphite in the cast iron (graphite spheroidization ratio) and the degree of sphericity of graphite (roundness coefficient) are measured. Specifically, the cross section of the cast iron to be measured is observed using the magnifying observation device 10, thereby measuring how much graphite with a shape close to spherical exists in the structure of the cast iron, and how close to spherical the graphite is, etc.

[0057] When measuring graphite cast iron, a sample obtained by cutting the graphite cast iron to be measured is placed on the surface of the graphite cast iron as the observation object OB. Figure 1 The mounting table 2 is provided, and an image showing the surface of the observation object OB is displayed on the display unit 72 in an enlarged manner.

[0058] Next, the designated processing, binarization processing, shaping processing, and measurement processing included in the image analysis processing performed by the analysis processing unit 303 are described. The designated processing, binarization processing, shaping processing, and measurement processing are changed in this order according to the user's operation. In addition, it is also possible to return to the previous processing according to the user's operation.

[0059] In the designation process, the analysis processing unit 303 designates an observation image to be the object of the image analysis process. In the binarization process, the analysis processing unit 303 binarizes the observation image based on the set conditions (conditions specified by the user), thereby extracting a block region that satisfies the conditions. In the shaping process, the analysis processing unit 303 shapes the block region extracted by the binarization process. In the measurement process, the analysis processing unit 303 measures the feature value (sphericity, roundness coefficient, etc.) of the block region shaped by the shaping process.

[0060] The display screen 400 displayed on the display unit 72 will be described. Figure 6 4 is a diagram showing a display screen 400 during a designation process. Figure 6 The display screen 400 includes an observation image screen 411 and a first setting screen 421. On the observation image screen 411, an image showing an observation object OB in an enlarged manner based on the observation image generated by the image generating unit 301 is displayed.

[0061] In the designation process, the observation image captured by the image pickup device 7 at the current time point is displayed on the observation image screen 411. During the designation process, the user designates the imaging range of the observation object OB to be imaged by the image pickup device 7 by moving the stage 2 in the horizontal direction (X direction, Y direction) using the operation unit 73. The image displayed on the observation image screen 411 also changes in accordance with the designation of the imaging range.

[0062] The first setting screen 421 is a screen generated by the display control unit 304 as a setting screen corresponding to the designated process. The first setting screen 421 displays the setting items that can be accepted by the analysis processing unit 30 in the designated process. The display control unit 304 generates the first setting screen 421 in the designated process, and displays the screen obtained by combining the observation image screen 411 based on the observation image generated by the image generation unit 301 and the first setting screen 421 on the display unit 72 as the display screen 400.

[0063] exist Figure 6The observation image screen 411 shows 15 block areas A1-A15 and 3 areas B1-B3. The block areas A1-A15 are graphite structures included in the observation object OB and are displayed as black (low brightness). Figure 6 In FIG. 1 , block regions A1 to A15 are shown with a dark dot pattern. Regions B1 to B3 are not graphite structures and are shown with a lighter dot pattern than block regions A1 to A15.

[0064] Block area A2 and block area A3 among block areas A1-A15 exist close to each other. They are collectively referred to as block area A4. Block area A5 includes a white (high brightness) hole area A5a inside. Block area A6 includes a missing area A6a extending from the outside to the inside of block area A6. Block area A8 includes a burr area A8a protruding to the outside of block area A8. Block area A12 and block area A13 are combined into one. They are collectively referred to as block area A14.

[0065] On the first setting screen 421 , a progress indicator 430 , an import button 440 , a designation completed image display portion 445 , a file designation button 449 , a specification selection portion 450 , and a “next” button 462 are displayed.

[0066] A progress indicator 430 showing the progress of the image analysis process is displayed at the top of the first setting screen 421. The progress indicator 430 is divided into a first part 431, a second part 432, a third part 433, and a fourth part 434 from the left. By highlighting one of these parts, the progress of the image analysis process is displayed in a visually easy-to-understand manner. Figure 6 , the first portion 431 at the left end is highlighted. Specifically, the first portion 431 displays a text “designated” indicating that the designated process is currently being executed.

[0067] When the user presses the import button 440 using the pointing device (mouse, etc.) included in the operation unit 73, the observation image displayed on the observation image screen 411 at that time is imported as the object of the subsequent image analysis processing. The imported observation image is designated as the object of the subsequent image analysis processing. The user can operate the operation unit 73 to move the stage 2 in the horizontal direction through the movement control unit 302 and the stage control unit 23, thereby changing the camera range regarding which part of the observation object OB is photographed. In the case where observation images captured for multiple parts of the observation object OB are required, the user imports observation images of multiple different parts of the observation object OB while changing the camera range, thereby designating multiple observation images as the objects of the image analysis processing.

[0068] The designated image display unit 445 displays thumbnails of observation images of the imaging range designated by the user. Figure 6 Thumbnails of four designated images (a first observation image 441 , a second observation image 442 , a third observation image 443 , and a fourth observation image 444 ) are displayed.

[0069] When the user presses the file designation button 449 , a file selection dialog box (not shown) is opened. When a pre-photographed observation image is prepared, the user can use the file designation button 449 to designate the pre-photographed observation image as a target of the image analysis process.

[0070] The first setting screen 421 corresponding to the designation process may be capable of accepting both the designation of the imaging range to be imaged by the imaging element 7 and the designation of an observation image captured in advance, or may be capable of accepting only one of the two.

[0071] The specification selection unit 450 is an interface capable of receiving information for determining a method of subsequent image analysis processing. Figure 6 In FIG. 4 , the text “Specification A” is displayed in the specification selection portion 450 . This means that the subsequent image analysis process is executed according to the conditions determined by the specification named “Specification A”.

[0072] Figure 6 The specification selection section 450 is a drop-down menu, and the user can determine the subsequent image analysis processing method from a plurality of options. For example, in the determination of the spheroidization rate of ductile iron, the specifications of various countries such as ASTM (United States), JIS (Japan), GB-T (China), ISO (International Standard) can be selected. In addition, in addition to the pre-prepared specifications, the user can also specify a personal image analysis processing method. For example, it is preferred to prepare a "custom" option in the drop-down menu. The user who selects "custom" then enters information on what basis the subsequent image analysis processing is to be performed. The information entered through "custom" includes, for example, the size of the camera range that is the object of measurement (field of view area), the number of observation images used for the measurement, calibration information (correspondence information between a quantity of pixels and the actual distance), and the type of measurement (determination of the content of non-metallic inclusions, determination of grain size, DAS measurement, etc.).

[0073] After the user has completed designating the observation image to be the target of the image analysis process and selecting the specification (determining the method of the image analysis process), he presses the "Next" button 462. Then, the image analysis process of the analysis processing unit 303 changes from the designation process to the binarization process.

[0074] Reference Figure 7 The binarization process will be described below. Figure 7 4 is a diagram showing a display screen 400 during binarization processing. Figure 7 The display screen 400 includes an observation image screen 411 and a second setting screen 422. When the image analysis process is changed from the designated process to the binarization process, the display control unit 304 generates the second setting screen 422 as a setting screen corresponding to the binarization process. In addition, when the binarization process is changed, the observation image that becomes the object of the image analysis process is also fixed to the observation image designated by the designated process, and even if the user moves the stage 2, the display of the observation image screen 411 does not change. Alternatively, it is also possible that the stage 2 cannot be moved after the binarization process.

[0075] The second setting screen 422 for the binarization process is a screen for accepting setting items that can be accepted by the analysis processing unit 303 in the binarization process. The second setting screen 422 accepts setting items different from the first setting screen 421 for the designated process. The display control unit 304 displays the screen obtained by combining the second setting screen 422 and the observation image screen 411 based on the observation image designated by the designated process as the display screen 400 on the display unit 72 in the binarization process.

[0076] exist Figure 7 The second setting screen 422 includes a progress indicator 430, a threshold setting unit 452, a measurement range setting unit 453, a "return" button 461, and a "next" button 462. During the binarization process, the second part 432 of the progress indicator 430 is highlighted. Specifically, the text "extract" indicating that the binarization process of the extraction block area is currently being performed is displayed in the second part 432 second from the left.

[0077] The threshold setting unit 452 is an interface for accepting a threshold for binarizing an observation image from the user in the second setting screen 422. Specifically, the user sets a threshold for brightness. Figure 7 The threshold setting unit 452 includes a threshold slider 452s. The user can set the brightness threshold by moving the threshold slider 452s left and right. The brightness threshold can be set within a range of 0 to 255, for example.

[0078] In the vicinity of the threshold setting unit 452 (in Figure 7 The current threshold value is displayed in the lower middle area. Figure 7 The medium threshold is "97". The user can increase the threshold by "1" each time by pressing the "+" button arranged on the right side of the threshold setting unit 452. In addition, the user can decrease the threshold by "1" each time by pressing the "-" button arranged on the right side of the threshold setting unit 452.

[0079] The display control unit 304 performs emphasized display on the extracted block region based on the comparison between the data (here, brightness) of the pixels included in the observed image and the threshold value set by the user using the threshold value setting unit 452. Figure 7 Specifically, the block area whose brightness is lower than the threshold value "97" (close to black) is highlighted.

[0080] exist Figure 7 In the example, the block areas A1-A15 with low brightness are highlighted. On the other hand, the areas B1-B3 with high brightness are not highlighted. Figure 7 In the block area A1-A15, Figure 6 The thick dot pattern is emphasized. In addition, if the display unit 72 is a color display, the block areas A1-A15 can also be emphasized by being colored in a color (for example, red). For example, the display control unit can also generate a graphic for emphasis display having the same shape as each of the block areas A1-A15, and emphasize the display by overlapping the graphic for emphasis display at the position of the block areas A1-A15 in the observation image screen 411.

[0081] In the binarization process, a binarization value "0" or "1" is assigned to each pixel of the observed image according to whether or not the threshold condition is satisfied (the observed image is binarized). For example, a binarization value "1" is assigned to pixels in the block area A1-A15 whose brightness is lower than the threshold, and a binarization value "0" is assigned to pixels in other areas (including the area B1-B3). Then, the area assigned with the binarization value "1" is highlighted and extracted as an area to be subjected to the subsequent shaping process and measurement process.

[0082] In the binarization process, it is also possible to accept settings regarding which range in the observation image becomes the object of measurement in the subsequent measurement process. Figure 7 In the second setting screen 422, the measurement range setting unit 453 receives the setting of the range (measurement range) to be measured in the measurement process.

[0083] Figure 7 The measurement range setting section 453 includes two check boxes, and the user can select whether to set the measurement range by a rectangle or a circle by selecting one of them. Figure 7 , the upper check box for selecting the measurement range by a rectangle is selected. When the upper check box is selected, a measurement range frame 401 corresponding to the numerical value input into the numerical value input field corresponding to the upper check box is displayed on the observation image screen 411. The measurement range frame 401 is a rectangle centered at the center (field of view) of the observation image screen 411.

[0084] exist Figure 7 , a value of 0.6 mm × 0.4 mm is entered in the value input column corresponding to the upper check box. This value indicates that the range of the actual size of the observed object OB, which is 0.6 mm in width (X direction) and 0.4 mm in length (Y direction), is set as the measurement range. Therefore, the width (X direction) of the measurement range frame 401 displayed on the observation image screen 411 corresponds to 0.6 mm, and the length (Y direction) corresponds to 0.4 mm.

[0085] When the check box at the bottom of the measurement range setting unit 453 is checked, a circle having a diameter equal to the size input into the lower numerical input field (the center of the circle is the center of the observation image screen 411 ) is set as the measurement range frame 401 .

[0086] Furthermore, the user can specify the measurement range by specifying the range within the observation image screen 411 using the pointing device of the operation unit 73, instead of inputting a numerical value into the measurement range setting unit 453. In this case, the actual size corresponding to the measurement range specified by the user is displayed in the numerical column of the measurement range setting unit 453.

[0087] After the user completes the setting of the threshold value and the measurement range (the range to be measured), he presses the "Next" button 462. Then, the image analysis processing of the analysis processing unit 303 changes from the binarization processing to the shaping processing. On the other hand, when the user presses the "Return" button 261, the image analysis processing of the analysis processing unit 303 returns from the binarization processing to the specified processing.

[0088] Furthermore, when returning from the binarization process to the designated process, it is preferable that the threshold value set by the user in the binarization process remains stored. As long as the threshold value remains stored, the user does not need to reset the threshold value again when the image analysis process is switched to the binarization process again, thereby improving the convenience for the user.

[0089] Reference Figure 8 The shaping process will be described below. Figure 8 4 is a diagram showing a display screen 400 during the shaping process. When the image analysis process changes from the binarization process to the shaping process, the display control unit 304 generates a third setting screen 423 as a setting screen corresponding to the shaping process.

[0090] In addition, when the process is changed to the shaping process, the block area (here, the block area with low brightness) that meets the condition of the threshold value set in the binarization process is assigned an identification number (labeling) for distinguishing each block area. For example, the area where pixels that are assigned a binary value of "1" based on the threshold value in the binarization process are connected is set as a block area, and each block area is assigned an identification number for distinguishing each other. The identification number is not known to the user, but is only used for the analysis and processing unit 303 to perform various processes. In addition, the assignment (labeling) of the identification number is not necessarily performed at the same time when the binarization process is changed to the shaping process, and it can also be performed appropriately. For example, labeling can also be performed appropriately every time the threshold value is changed in the binarization process.

[0091] The third setting screen 423 for shaping processing is a screen for accepting setting items that can be accepted by the analysis processing unit 303 in shaping processing. The third setting screen 423 accepts setting items different from the first setting screen 421 and the second setting screen 422. The display control unit 304 displays a screen obtained by combining the third setting screen 423 and the observation image screen 411 based on the observation image specified by the specifying process as the display screen 400 on the display unit 72 in shaping processing.

[0092] exist Figure 8 The third setting screen 423 includes a progress indicator 430, a shaping tool section 470, a reset button 465, a "return" button 461, and a "next" button 462. During the binarization process, the third part 433 of the progress indicator 430 is highlighted. Specifically, the third part 433 from the left displays the text "shaping" indicating that the shaping process of the block area is currently being performed.

[0093] The shaping tool section 470 includes a plurality of shaping tools for shaping the shape of the block region using various methods. Figure 8 In the figure, the shaping tool section 470 includes a hole filling tool 471 , a particle removal tool 472 , a bonding tool 473 , a separation tool 474 , a deburring tool 475 , a gap filling tool 476 , a reduction tool 477 , and an expansion tool 478 .

[0094] These shaping tools perform shaping of the type corresponding to each shaping tool on the block area selected by the user. For example, after selecting the block area to be shaped, the user selects the shaping tool corresponding to the type of shaping to be performed, thereby performing the shaping of the selected shaping tool on the selected block area. In addition, it is also possible to select the shaping tool first and then select the block area to be shaped. In addition, it is also possible that the user selects multiple block areas at once, and the selected multiple block areas can be shaped uniformly. In addition, it is also possible that the shaping tool can perform shaping on all block areas within the measurement range frame 401 at once. For example, it is also possible that when the shaping tool is selected in a state where the block area to be shaped is not selected, shaping is performed on all block areas at once.

[0095] The display control unit 304 updates the shape of the block area after shaping by each shaping tool to the shaped shape and displays it on the display screen 400. For example, the display control unit 304 preferably updates the shape of the block area displayed on the display screen 400 to the shaped shape by updating the graphic for highlighting displayed on the block area in the observation image screen 411 to the shaped shape.

[0096] The reset button 465 is a button for returning the shape change caused by shaping by the shaping tool unit 470 to its original state. The reset button 465 can return the shapes of the shaped block areas to their original state one by one or can also return the shapes of a plurality of block areas to their original state collectively.

[0097] Reference Figure 8 , Fig. 9 The shaping based on the shaping process will be described. Fig. 9 FIG. 4 is a diagram showing a display screen 400 to which shaping by shaping processing is applied.

[0098] In the shaping process, when the hole filling tool 471 is selected, hole filling of the block area is performed. Hole filling refers to the process of, for example, Figure 8 When a portion of the interior of the block area A5 includes a hole area A5a (here, an area with high brightness) that does not satisfy the threshold condition, processing is performed to fill the hole area A5a.

[0099] For example, when Figure 8 When the hole filling is performed in the block area A5, the hole area A5a is filled, and the shape of the block area A5 is as follows: Fig. 9 As shown, the block area A5 is updated to a shape that does not include the hole area A5a.

[0100] When filling the hole, the analysis processing unit 303 checks the size of the hole area A5a, for example, and if the size is smaller than a reference value (e.g., a diameter of 10 μm), the pixel of the hole area A5a to which the binary value “0” is assigned is reassigned a binary value “1”, thereby filling the hole area A5a. As long as the binary value is reassigned when the size of the hole area A5a is smaller than the reference value, all block areas in the observation image can be processed at once.

[0101] In addition, in addition to the above-mentioned method of reallocating the binary value assigned to the hole area A5a, various methods can be used to fill the holes. For example, the hole filling can be achieved by combining the expansion and reduction described later. When the block area A5 is expanded several times (for example, twice), the hole area A5a is filled. In this state, the size of the block area A5 is larger than the original size, so the size of the block area A5 filled with the hole area A5a is restored to its original state by reducing the number of times the expansion is performed. This can also achieve hole filling.

[0102] When the particle removal tool 472 is selected, particle removal of the block region is performed. Particle removal is a process of removing (unemphasizing) a block region designated by the user or a block region whose size is smaller than a preset reference value (eg, a diameter of 10 μm).

[0103] When performing particle removal on a block area designated by the user, the analysis processing unit 303 changes the binary value of the pixel assigned to the designated block area from "1" to "0", thereby releasing the emphasized display of the block area. In addition, when performing particle removal on the entirety of the observed image, the analysis processing unit 303 confirms the size of each block area. Then, if there is a block area whose size is smaller than the reference value, the binary value of the pixel assigned to the block area is changed from "1" to "0", thereby releasing the emphasized display of the block area.

[0104] When the combine tool 473 is selected, the block regions are combined. Combining refers to the process of combining two adjacent block regions into one block region. The two block regions to be combined can be specified by the user or automatically selected according to a pre-set benchmark (for example, the distance between the two block regions is less than 5 μm).

[0105] For example, when Figure 8 When the block area A2 and the block area A3 are combined, the block area A2 and the block area A3 are combined, and their shapes are as follows: Fig. 9 As shown, it is updated to a block area A4.

[0106] When the separation tool 474 is selected, the block area is separated. Separation refers to the process of dividing a block area into two or more block areas. The block area to be separated can be specified by the user or automatically selected according to a preset reference (for example, a portion with a width less than 5 μm is included in the block area). When the user specifies the block area as the separation object, the user can also specify which portion of the block area is to be separated.

[0107] For example, when Figure 8 When the block area A14 is separated, the block area A14 is separated into the block area A12 and the block area A13. The shape of the block area A14 is as follows: Fig. 9 As shown, it is updated into two block areas A12 and A13.

[0108] When the deburring tool 475 is selected, deburring of the block area is performed. Deburring refers to the process of removing the portion (burr) protruding from the block area to the outside. The block area to be deburred can be specified by the user or automatically selected according to a pre-set standard (for example, burrs protruding from the block area with a width of less than 5 μm).

[0109] For example, when Figure 8 When deburring is performed on the block area A8, the burr area A8a included in the block area A8 is removed, and the shape of the block area A8 is as follows: Fig. 9 As shown, the shape is updated to not include the burr area A8a.

[0110] When performing burr removal, the analysis processing unit 303 can also achieve burr removal by, for example, combining the reduction and expansion described later. When the block area A5 is reduced several times (for example, twice), the burr area A8a with a small width disappears. In this state, the size of the block area A8 is smaller than the original size, so the size of the block area A8 after the burr area A8a disappears is restored to its original state by expanding the same number of times as the reduction. The number of repetitions of reduction and expansion can also be set by the user.

[0111] When the gap filling tool 476 is selected, the gap filling of the block area is performed. Gap filling refers to the process of filling the missing part when there is a missing part inside the block area. The block area to be filled can be specified by the user or automatically selected according to a preset standard (for example, there is a gap with a width of less than 5μm inside the block area).

[0112] For example, when Figure 8 When the missing area A6 of the block area A6 is filled, the missing area A6a included in the block area A6 is filled, and the shape of the block area A6 is as follows: Fig. 9As shown, the missing area A6a is updated to be filled.

[0113] When performing the gap filling, the analysis processing unit 303 can also achieve the gap filling by, for example, combining the expansion and reduction described later. When the block area A6 is expanded several times (for example, twice), the missing area A6a with a small width is filled. In this state, the size of the block area A6 is larger than the original size, so the size of the block area A6 filled with the missing area A6a is restored to its original state by reducing the number of times the expansion is performed. The number of repetitions of expansion and reduction can also be set by the user.

[0114] When the reduction tool 477 is selected, the block area is reduced. Reduction refers to the process of reducing the block area. When reducing, the analysis processing unit 303 reduces the size of the block area by, for example, changing the binary value "1" assigned to the pixels in the peripheral part of the block area to "0".

[0115] When the dilation tool 478 is selected, dilation of the block region is performed. Dilation refers to a process of dilating the block region to increase its size. When dilating, the analysis processing unit 303, for example, changes the binary value "0" assigned to the pixels adjacent to the outer peripheral portion of the block region to "1", thereby dilating the size of the block region.

[0116] In addition, the shaping tool section 470 may also include shaping tools that use other methods for shaping, such as a thinning tool that reduces the width of a block area to one pixel (1 pixel), an inversion tool that swaps the binary value assigned to the block area (inverts the extraction result in the binarization process), and so on.

[0117] After the shaping of the block area is completed, the user presses the "Next" button 462. Then, the image analysis processing of the analysis processing unit 303 changes from the shaping processing to the measurement processing. On the other hand, when the user presses the "Return" button 461, the image analysis processing of the analysis processing unit 303 returns from the shaping processing to the binarization processing.

[0118] Furthermore, when returning from the shaping process to the binarization process, it is preferable that the shaping results performed by the user on each block region in the shaping process remain stored. As long as the shaping results remain stored, when the image analysis process is changed to the shaping process again, the user does not need to perform shaping on each block region again, thereby improving the convenience for the user.

[0119] When the image analysis process changes from the shaping process to the measurement process, the analysis processing unit 303 performs shape filtering. Shape filtering refers to the process of removing the display of block areas that are not the object of the measurement performed in the measurement process from the display screen 400. For example, in a case where the measurement performed in the measurement process only takes block areas with a diameter of 10 μm or more as the object, the display of block areas with a diameter of less than 10 μm is removed. The removal criteria for what kind of block areas are to be removed can be set by the user. For example, the removal criteria in shape filtering can also be included in the specifications (image analysis processing method) selected in the specified process. In shape filtering, removal is sometimes performed not only based on the size (dimensions) of the block area, but also based on the geometric shape of the block area. For example, block areas with shapes that deviate greatly from a circle (such as an elongated shape) can also be removed.

[0120] When the measurement process is switched, the analysis processing unit 303 first measures the feature values ​​(circularity coefficient, area, diameter, etc.) of all block regions within the measurement range by image analysis processing, and then removes the block regions that meet the removal criteria from the display screen 400 by shape filtering. When the shape filtering is executed, the user is informed that the block regions that meet the removal criteria are to be removed from the display screen 400. Fig.10 is a diagram showing a confirmation message 412 .

[0121] When the image analysis process changes from the shaping process to the measurement process, a confirmation message 412 is displayed on the display screen 400 before the shape filtering is executed. Fig.10 The confirmation message 412 shows that “particles smaller than the reference value are removed”.

[0122] For example, the removal criterion is to remove block regions (particles) with a diameter smaller than 10 μm. If the diameter of the block region A11 is smaller than 10 μm, the block region A11 is removed by shape filtering, and thus a confirmation message 412 is displayed on the display screen 400 .

[0123] After the user confirms confirmation message 412 (for example, by pressing an “OK” button in confirmation message 412 ), the image analysis process transitions to a measurement process in which the feature amount of the block region is measured and the block region satisfying the removal criterion is removed from display screen 400 .

[0124] Reference Fig.11 , Fig.12 , Fig.13 Next, the display screen 400 during the measurement process will be described. Fig.11 4 is a diagram showing a display screen 400 during measurement processing. Fig.12 4 is a diagram showing the measurement result window 413 . Fig.13 is a diagram showing the histogram window 414 .

[0125] When the image analysis process changes to the measurement process, the display control unit 304 generates a measurement result window 413 and a histogram window 414 as result display screens corresponding to the measurement process. In addition, the display control unit 304 generates a display format selection screen 424 for selecting the display format of the result display screen.

[0126] During the measurement process, the display control unit 304 displays on the display unit 72 as a display screen 400 a screen that combines the measurement result window 413 , the histogram window 414 , the display format selection screen 424 , and the observation image screen 411 based on the observation image specified by the specifying process.

[0127] The numerical value of the feature amount of the block region measured by the measurement process is displayed in the measurement result window 413. The distribution graph (histogram 418) of the feature amount of the block region is displayed in the histogram window 414.

[0128] The measurement result window 413 and the histogram window 414 are displayed superimposed on the observation image screen 411. The user can also change the positions of the measurement result window 413 and the histogram window 414 in the display screen 400 using a pointing device included in the operation unit 73.

[0129] The display format selection screen 424 includes a progress indicator 430 , a measurement result button 481 , a histogram button 482 , a “return” button 461 , a replay button 463 , a boundary processing switching unit 483 , a display switching unit 484 , a save button 485 , and a report button 486 .

[0130] During the measurement process, the fourth portion 434 of the progress indicator 430 is highlighted. Specifically, the fourth portion 434 on the far right displays the words "Result" indicating that the measurement process of measuring the feature amount and displaying the result is currently being performed.

[0131] The measurement result button 481 is a button for switching whether to display the measurement result window 413. Each time the user presses the measurement result button 481, the state in which the measurement result window 413 is displayed and the state in which the measurement result window 413 is not displayed are switched.

[0132] like Fig.12 As shown, the measurement result window 413 is a screen showing the numerical value of the measurement result of the characteristic quantity of the block area. As the characteristic quantity of the block area, for example, the roundness coefficient can be cited. The roundness coefficient is a coefficient indicating how close the block area is to a circle. For example, the maximum diameter (long axis) of the block area is set to L, the area is set to S, and the circle with the maximum diameter as the diameter is set to D. The roundness coefficient is calculated by dividing the area of ​​S by the area of ​​D (roundness coefficient = S / D = 4S / (L2 ×π)).

[0133] exist Fig.12 The measurement result window 413 displays statistical data of the block area included in the measurement object. For example, the number of particles (block area) and the number of particles per unit area, the average value of the roundness coefficient, the minimum value and the maximum value of the size, etc. are displayed. In addition, the measurement conditions in the measurement process are also displayed in the measurement result window 413. For example, the type of specification (information for determining the method of image analysis processing) of the measurement method selected by the specified processing, the field number (the number of observation images that become the measurement object), the calibration (the relationship between a pixel in the image and the actual distance), the evaluation area (the area of ​​the range that becomes the measurement object in the observation object OB), etc. are displayed. In addition, it is also possible to display statistical data of the measurement results for each of these multiple observation images when image analysis processing is performed on multiple observation images.

[0134] The measurement result window 413 includes a data save button 415. When the user presses the data save button 415, the measurement result data displayed in the measurement result window 413 is saved. The measurement result data may be stored in the storage unit 311 of the information processing unit 71 or may be sent to an external storage device.

[0135] The histogram button 482 is a button for switching whether to display the histogram window 414. Every time the user presses the histogram button 482, the state in which the histogram window 414 is displayed and the state in which the histogram window 414 is not displayed are switched.

[0136] like Fig.13 As shown in FIG. 4 , the measurement result window 413 is a screen for displaying a distribution diagram (histogram 418) of a feature value (e.g., a roundness coefficient) of a block region. The feature value of a block region includes various types such as area in addition to the roundness coefficient. In the display condition section 419 provided on the upper left side of the histogram window 414, it is displayed which feature value of the block region the currently displayed histogram 418 was generated with attention paid to.

[0137] exist Fig.13 In FIG. 4 , a histogram 418 of the roundness coefficient focusing on the block area is shown. Fig.13 In the figure, the roundness factor is shown as a percentage (%). Fig.13 It can be seen from the histogram 418 that there are about 3500 block particles with a roundness coefficient below 1% (0.01).

[0138] The "return" button 461 in the measurement process is a button for changing (restoring) the image analysis process of the analysis processing unit 303 from the measurement process to the shaping process. When there is an unfinished task in the shaping process, the user can press the "return" button 461 to restore the image analysis process of the analysis processing unit 303 from the measurement process to the shaping process. When returning from the measurement process to the shaping process, it is preferred that the shaping results performed by the user on each block area in the previous shaping process remain in a stored state. As long as the shaping results remain in a stored state, when returning to the shaping process, the user does not need to perform shaping again on the block area that has already been shaped, so the convenience for the user is improved.

[0139] The repeat button 463 is a button for executing (reproducing) again a series of image analysis processes, namely, the designation process, the binarization process, the shaping process, and the measurement process, which were executed by the analysis processing unit 303 this time under the same conditions as this time.

[0140] The analysis processing unit 303 can store the operation conditions including the contents of the setting items set in the executed image analysis processing (the conditions set by the user in the first setting screen 421, the second setting screen 422, and the third setting screen 423) as the measurement history in the storage unit 311. When the user presses the redo button 463, the analysis processing unit 303 can read the stored measurement history and execute the image analysis processing again under the same conditions as the operation conditions stored as the measurement history.

[0141] Furthermore, in the case where a plurality of observation images are designated in the designation process, the designation method of the plurality of observation images is also stored as a measurement history record. In the case of reproduction, the designation of the observation images is also reproduced under the same conditions as the designation method stored as a measurement history record. That is, in the case where the user uses the operation unit 73 to move the stage 2 in the horizontal direction (X direction, Y direction) to designate the imaging range of the observation object OB in the designation process, the movement of the stage 2 is also reproduced. Furthermore, in the case where a plurality of observation images are captured in the designation process, the positional relationship between the plurality of observation images is also reproduced. That is, conditions such as the distance to which the imaging ranges of the plurality of observation images are separated and the number of observation images captured are also reproduced.

[0142] When reproducing, the designated processing, binarization processing, and shaping processing are automatically executed without receiving user operations, and the image analysis processing is switched to the measurement processing. If there are setting items that the designated processing, binarization processing, and shaping processing want to be modified during reproducing, the user can modify the setting items by returning from the measurement processing to the processing including the target setting items (for example, using the "return" button 461).

[0143] The boundary processing switching unit 483 is a part for switching the processing of the block area overlapping the boundary of the measurement range frame 401. The boundary processing switching unit 483 is provided with a check box, and the processing of the block area overlapping the boundary of the measurement range frame 401 is switched according to whether the user checks the check box.

[0144] The processing of the block area overlapping with the boundary of the measurement range frame 401 may differ depending on the type of specification (information for determining the method of image analysis processing) of the measurement method selected by the designation process. For example, there are specifications for processing the block area overlapping with the boundary of the measurement range frame 401 as a measurement object and specifications for not processing the block area as a measurement object. In addition, depending on the difference in specifications, there is also a case where the block area overlapping with the left and upper sides of the measurement range frame 401 is not processed as a measurement object, but the block area overlapping with the right and lower sides is processed as a measurement object.

[0145] As described above, the processing of the block area overlapping with the boundary of the measurement range frame 401 is different depending on the specification. The user can use the check box of the boundary processing switching unit 483 to switch whether to apply the processing determined by the specification in the measurement processing. When the check box is checked, the processing determined by the specification is applied to the block area overlapping with the boundary of the measurement range frame 401. When the check box is not checked, all the block areas overlapping with the boundary of the measurement range frame 401 are processed as measurement objects. In addition, when the check box of the boundary processing switching unit 483 is switched, the number of block areas included in the measurement object changes, so the statistical data of the feature amount of the block area is calculated again.

[0146] The display switching unit 484 is a part for switching the display form of the result display screen, especially the histogram window 414. A pull-down menu is provided in the display switching unit 484, and the user can use the pull-down menu to select the feature quantity of interest from the type of feature quantity of the block area (circularity coefficient, area, etc.). The name of the feature quantity selected by the display switching unit 484 is displayed in the display condition unit 419 of the histogram window 414. Moreover, when the feature quantity selected by the display switching unit 484 is switched, the histogram 418 focusing on the selected feature quantity is generated again as the histogram 418 displayed in the histogram window 414, and the display of the histogram window 414 is updated.

[0147] The save button 485 is a button for saving the data of the measurement result. When the user presses the save button 485, the data of the measurement result corresponding to the multiple observation images designated as the measurement object and the respective observation images are saved. Here, when saving the data of the measurement result, it is preferred that the data of the measurement history record used to perform the above-mentioned reproduction is also saved together. The data of the measurement history record can be recorded as data independent of the data of the observation image and the measurement result, or can be recorded by being embedded in the data of the observation image and the measurement result. For example, the data of the measurement history record can also be recorded in the metadata of the observation image.

[0148] Report button 486 is a button for outputting a report of the measurement results. When the user presses report button 486, the data of the measurement results are summarized in a report in a predetermined format and then output. The report can be output by printing on paper or as electronic data in a predetermined format such as text data.

[0149] According to the above embodiment, screens (first setting screen 421, second setting screen 422, third setting screen 423) are generated in which the setting items that can be accepted are sequentially changed according to the process (specifically, the designation process, the binarization process, and the shaping process) being executed by the analysis processing unit 303. That is, while the analysis processing unit 303 is executing a certain process, only the setting items that the analysis processing unit 303 can accept in the process are accepted from the user, so even if the user changes the setting, the change will not affect other processes.

[0150] In particular, the setting items that can be accepted in the second setting screen 422 related to the binarization process and the third setting screen 423 related to the shaping process are sequentially changed, so the binarization process and the shaping process are independent processes. As a specific example, the setting of the threshold condition for binarization (for example, the threshold of brightness) can only be accepted in the binarization process. Therefore, the re-execution of the binarization process caused by the change of the setting of the threshold condition occurs only in the second setting screen 422.

[0151] Therefore, there is no situation where, for example, the shaping process and the binarization process are executed again due to a change in the threshold condition during the shaping process, and thus the convenience of the magnification observation apparatus 10 is improved.

[0152] In addition, in the magnification observation device 10 described above, different setting screens (first setting screen 421, second setting screen 422, third setting screen 423) are generated according to the designated processing, binarization processing, and shaping processing being executed by the analysis processing unit 303, and the setting screens themselves are sequentially changed. However, it is also possible that the setting screens do not change, and only the setting items that can be accepted change.

[0153] For example, all setting items may be listed in one setting screen, and the setting items that can be accepted may be sequentially changed in accordance with the process being executed by the analysis processing unit 303. That is, preferably, only the setting items that can be accepted in the process being executed by the analysis processing unit 303 are enabled, and the other setting items are disabled (for example, grayed out). Furthermore, when the process executed by the analysis processing unit 303 is changed to another process, it is preferred that the setting items that are enabled and the setting items that are disabled are changed in accordance with the process executed by the analysis processing unit 303.

[0154] Next, refer to Fig.14 , Fig.15 , Fig.16 , Fig.17 The flow of processing performed by the magnification observation device 10 will be described. Fig.14 is a flowchart showing the flow of the designated processing. Fig.15 : is a flowchart showing the flow of binarization processing. Fig.16 : is a flowchart showing the flow of shaping processing. Fig.17 It is a flowchart which shows the flow of a measurement process.

[0155] When the magnifying observation device 10 is started, first start (START) Fig.14 The display control unit 304 of the information processing unit 71 generates the first setting screen 421 in step S11 and displays the display screen 400 obtained by synthesizing the observation image generated by the imaging element 7 and the image generating unit 301 and the first setting screen 421 on the display unit 72 .

[0156] Next, in step S12, the movement control unit 302 controls the movement of the stage 2 according to the operation of the operation unit 73 by the user. After the movement control of the stage 2 (when the stage 2 is stationary), in step S13, it is confirmed whether the user has given an instruction to import the photographic image. The user can give an import instruction by, for example, pressing the import button 440 on the first setting screen 42. In addition, a button (photographing button) for giving an import instruction may be provided on the operation unit 73. In addition, the user may specify an image captured in advance as an observation image to be measured instead of giving an import instruction.

[0157] If the import instruction is not given ("No" in step S13), the processing of the magnifying observation device 10 proceeds to step S16 described later. If the import instruction is given ("Yes" in step S13), the processing of the magnifying observation device 10 proceeds to step S14, and the imaging by the imaging element 7 and the generation (import) of the observation image by the image generation unit 301 are performed. Then, in step S15, the observation image generated in step S14 or the image captured in advance (the image specified by the user) is stored (specified) in the storage unit 311 as the observation image to be measured.

[0158] After the observation image is stored, in step S16, it is confirmed whether the "next" button 462 of the first setting screen 421 is pressed. If the "next" button 462 is not pressed ("No" in step S16), the processing of the magnification observation device 10 returns to step S12, and the movement control according to the user's instruction is repeated.

[0159] When the "Next" button 462 is pressed ("Yes" in step S16), the end of the processing (END) is specified, and the image analysis processing of the analysis processing unit 303 is changed to Fig.15 Binarization processing.

[0160] When the image analysis process is switched to the binarization process (START), the display control unit 304 generates a second setting screen 422 in step S21 , and displays a display screen 400 obtained by combining the observation image designated by the designation process and the second setting screen 422 on the display unit 72 .

[0161] Then, the analysis processing unit 303 extracts the block region according to the threshold value specified by the user in step S22. Specifically, a group of pixels having a brightness lower than the threshold value specified by the user is extracted as the block region. At this time, identification numbers may also be assigned (marked) to each extracted block region.

[0162] The display control unit 304 highlights the extracted block region in step S23. Specifically, on the observation image screen 411, a highlighting graphic (for example, a red graphic) having the same shape as the block region is displayed superimposed on the position of the block region.

[0163] Then, in step S24, it is checked whether the "next" button 462 of the second setting screen 422 has been pressed. If the "next" button 462 has not been pressed ("No" in step S24), the processing of the analysis processing unit 303 returns to step S22, and the extraction of the block area corresponding to the user's instruction is repeated.

[0164] When the "Next" button 462 is pressed ("Yes" in step S24), the binarization process ends (END), and the image analysis process of the analysis processing unit 303 changes to Fig.16 Plastic processing.

[0165] When the image analysis process is changed to the shaping process (START), the display control unit 304 generates a third setting screen 423 in step S31 , and displays a display screen 400 in which the third setting screen 423 is combined with the observation image highlighted by the binarization process on the display unit 72 .

[0166] Then, in step S32, a shaping method (separation, combination, etc.) is selected according to the user's instruction. Next, in step S33, a block area to be shaped is selected. In addition, the selection of the shaping object in step S33 may be performed before the selection of the shaping method in step S32. In addition, the shaping object may be automatically selected according to a predetermined criterion without relying on the user's instruction.

[0167] When the shaping method and shaping target are selected, the selected block region is shaped using the selected method, and the shape of the block region is updated. Specifically, the display control unit 304 updates the shape of the highlight display graphic superimposed and displayed at the position of the block region in the observation image.

[0168] When the shaping is completed, in step S35, it is confirmed whether the "next" button 462 of the third setting screen 423 is pressed. If the "next" button 462 is not pressed ("No" in step S35), the processing of the analysis processing unit 303 returns to step S32, and the shaping of the block area corresponding to the user's instruction is repeated.

[0169] When the "next" button 462 is pressed ("yes" in step S35), the shaping process ends (END), and the image analysis process of the analysis processing unit 303 changes to Fig.17 Determination processing.

[0170] When the image analysis process is switched to the measurement process (START), the display control unit 304 displays a confirmation message 412 on the display unit 72 in step S41 to indicate that the display of particles (block regions) smaller than the reference value is to be removed.

[0171] After the user confirms the confirmation message, the feature amount of the block region to be measured is measured by the analysis processing unit 303. Then, in step S42, shape filtering is performed to remove the display of particles (block regions) smaller than a reference value.

[0172] Then, in step S43, the display control unit 304 generates a result display screen (specifically, a measurement result window 413 and a histogram window 414) based on the measurement result of the analysis processing unit 303. In addition, the display control unit 304 also generates a display form selection screen 424. Then, the display control unit 304 displays the display screen 400 obtained by synthesizing the observation image after the shaping process, the result display screen, and the display form selection screen 424 on the display unit 72.

[0173] While the result display screen is displayed, in step S44, it is checked whether the display of the feature quantity of interest has been switched based on the histogram window 414. If the display has not been switched (No in step S44), the processing of the magnification observation device 10 proceeds to step S46 described later.

[0174] When the display is switched (YES in step S44 ), the display control unit 304 updates the display of the result display screen (the histogram window 414 in this case) to focus on the feature amount after the switch in step S45 .

[0175] Then, in step S46, it is checked whether the reproduce button 463 of the display mode selection screen 424 has been pressed. If the reproduce button 463 has not been pressed (No in step S46), the analysis processing unit 303 and the display control unit 304 enter a standby state (END).

[0176] When the playback button 463 is pressed (YES in step S46), in step S47, the measurement history data is read from the storage unit 311, and the designated processing, binarization processing, shaping processing, and measurement processing are executed again according to the conditions stored in the measurement history.

[0177] In addition, Fig.14 , Fig.15 , Fig.16 , Fig.17 Although not shown in the figure, when the "return" button 461 is pressed during each process, the setting information set by the user's instruction during the process being executed by the analysis processing unit 303 remains stored, and the process of the analysis processing unit 303 changes to the previous process. Since the setting information remains stored, when the process of the analysis processing unit 303 changes to the previously executed process again, the user does not need to re-specify the previously specified setting information (for example, the brightness threshold), which is highly convenient for the user.

[0178] The above embodiments are illustrative and non-restrictive in all aspects. The scope of the present invention is not indicated by the above description, but by the claims, and is intended to include all changes within the meaning and scope equivalent to the claims. Among the structures described in the embodiments, structures other than the structures described as one aspect of the present invention in the "solution for solving the problem" are arbitrary structures and can be deleted and changed as appropriate.

[0179] Industrial Applicability

[0180] The present invention provides a magnifying observation device and an image processing device for the magnifying observation device, and has industrial applicability.

[0181] Description of Reference Numerals

[0182] 1: Camera unit; 2: Loading table; 4: Observation head; 6: Z-direction table; 7: Camera element; 10: Magnification observation device; 71: Information processing unit; 72: Display unit; 73: Operation unit; 300: Calculation unit; 301: Image generation unit; 302: Movement control unit; 303: Analysis and processing unit; 304: Display control unit; 305: Illumination control unit; 306: Input / output control unit; 311: Storage unit; 400: Display screen; 401: Measurement range frame; 411: Observation image screen; 412: Confirmation message; 413: Measurement result window; 414: Histogram window; 421: First setting screen; 422: Second setting screen; 423: Third setting screen; 424: Display format selection screen; 470: Shaping tool unit.

Claims

1. A magnifying observation device, comprising: An imaging element that captures an observation object; an image generating unit that controls the imaging element to generate an observation image; a mounting table for mounting the observation object; a stage control unit that moves the stage in a horizontal direction; a movement control unit that controls the movement of the mounting table by the mounting table control unit; an analysis processing unit that performs image analysis processing on the observation image; as well as a display control unit that generates a display screen based on the observation image, The image analysis processing performed by the analysis processing unit includes designation processing, binarization processing, shaping processing and measurement processing. The display control unit generates a result display screen corresponding to the measurement process and setting screens corresponding to the designated process, the binarization process, and the shaping process, respectively. In the designation process, the analysis processing unit designates the observation image to be the target of the image analysis process. In the binarization process, the analysis processing unit binarizes the observation image based on the set condition to extract a block region satisfying the condition. In the shaping process, the analysis processing unit performs shaping of the block region extracted by the binarization process. In the measurement process, the analysis processing unit measures a feature value of the block region shaped by the shaping process. The display control unit generates a screen in which the setting items that can be accepted are sequentially changed according to the process being executed by the analysis processing unit as the setting screen corresponding to the designated process, the binarization process, and the shaping process, respectively. The display control unit generates, as the result display screen, a screen for displaying, as a measurement result, a feature amount measured by the measurement processing of the analysis processing unit.

2. The magnifying observation device according to claim 1, wherein: The display control unit generates a plurality of setting screens capable of accepting the setting items respectively corresponding to the designated processing, the binarization processing, and the shaping processing as the setting screens, and sequentially changes the setting screens according to the processing being executed by the analysis processing unit.

3. The magnifying observation device according to claim 1, wherein: The setting screen corresponding to the designation process can accept at least one of designation of an imaging range of the observation object to be imaged by the imaging element and designation of the observation image captured in advance.

4. The magnifying observation device according to claim 1, wherein: The setting screen corresponding to the designated processing can receive information for determining a method of the image analysis processing to be executed by the analysis processing unit.

5. The magnifying observation device according to claim 1, wherein: The setting screen corresponding to the binarization process accepts a threshold value for binarizing the observation image, and the display control unit displays the extracted block area in an emphasized manner on the display screen based on a comparison between data of each pixel included in the observation image and the accepted threshold value.

6. The magnifying observation device according to claim 1, wherein: The setting screen corresponding to the binarization process receives setting of a range to be measured in the measurement process.

7. The magnifying observation device according to claim 1, wherein: When the analysis processing unit executes the shaping process, the display control unit updates the shape of the block region displayed on the display screen to the shape shaped by the shaping process.

8. The magnifying observation device according to claim 1, wherein: When the analysis processing unit executes a measurement process, the display control unit removes display of the block region satisfying a set removal criterion from the display screen.

9. The magnifying observation device according to claim 8, wherein: When the analysis processing unit executes the measurement process, the display control unit displays a confirmation message regarding removal of the block region satisfying the removal criterion.

10. The magnifying observation device according to claim 1, wherein: The analysis processing unit is capable of storing, as a measurement history, an operation condition including the contents of the setting items set in the executed image analysis processing, The analysis processing unit can read the measurement history to execute the image analysis processing again under the same conditions as the stored operation conditions.

11. An image processing device for a magnifying observation device, the magnifying observation device comprising: An imaging element that captures an observation object; a stage that is used to place the observation object; and a stage control unit that moves the stage in a horizontal direction, wherein the image processing device comprises: an image generating unit that controls the imaging element to generate an observation image; a movement control unit that controls the movement of the mounting table by the mounting table control unit; an analysis processing unit that performs image analysis processing on the observation image; as well as a display control unit that generates a display screen based on the observation image, The image analysis processing performed by the analysis processing unit includes designation processing, binarization processing, shaping processing and measurement processing. The display control unit generates a result display screen corresponding to the measurement process and setting screens corresponding to the designated process, the binarization process, and the shaping process, respectively. In the designation process, the analysis processing unit designates the observation image to be the target of the image analysis process. In the binarization process, the analysis processing unit binarizes the observation image based on the set condition to extract a block region satisfying the condition. In the shaping process, the analysis processing unit performs shaping of the block region extracted by the binarization process. In the measurement process, the analysis processing unit measures a feature value of the block region shaped by the shaping process. The display control unit generates a screen in which the setting items that can be accepted are sequentially changed according to the process being executed by the analysis processing unit as the setting screen corresponding to the designated process, the binarization process, and the shaping process, respectively. The display control unit generates, as the result display screen, a screen for displaying, as the measurement result, a feature amount measured by the measurement process performed by the analysis processing unit based on the setting items received via each of the setting screens.

Images