Image processing method, image processing device, scanning probe microscope, and program

Abstract

Provided is a technique for specifying, for an image of a sample generated by measurement by a scanning probe microscope, the type of correction corresponding to the surface state of the sample. The image processing method includes a step (SA1) of generating a corrected image by performing a first correction on an object image. The first correction includes: extracting a plurality of pixels from one straight line in a prescribed direction in a prescribed plane in the subject image; and correcting the height of the target image based on the extracted brightness of each pixel. The image processing method further comprises the steps of: generating a histogram (SA3) of pixel values in the corrected image; and determining whether a second correction different from the first correction needs to be performed on the target image using the histogram (SA4).

Description

Technical Field

[0001] The present invention relates to the processing of images obtained by a scanning probe microscope. Background Art

[0002] The scanning probe microscope (SPM) brings a sufficiently sharpened probe sufficiently close to the sample to be observed, and while adjusting the height of the probe up and down so that the physical quantity acting on the tip of the probe and the surface of the sample is constant, the surface of the sample is scanned horizontally with the probe to observe the concavity and convexity of the sample surface with high resolution. SPM is a general term for microscopes that observe the concavity and convexity of the sample surface according to the above-mentioned operating principle. As representative SPMs, there are scanning tunneling microscopes (STM) that detect the current flowing between the probe and the sample as an interaction, and atomic force microscopes (AFM) that detect the atomic force acting between the probe and the sample as an interaction.

[0003] The scanning probe microscope has a high resolution in the surface height direction, and it is difficult to set the sample surface horizontally at the level of its resolution. Therefore, it is generally corrected to make the inclined surface horizontal by performing height correction on the height image (hereinafter referred to as SPM image) obtained by the scanning probe microscope. By providing the corrected image instead of the SPM image, the user can more accurately identify the surface state of the sample.

[0004] As an example of processing an image obtained by such a scanning probe microscope, Japanese Patent Gazette No. 6627903 (Patent Document 1) discloses the following technology: at least a portion of the area other than the edge in the image data is extracted as a reference plane area, and the height of the measured data is corrected based on the height information of three points belonging to the reference plane area.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent No. 6627903 Summary of the invention

[0008] Problem that the invention aims to solve

[0009] In the conventional technology, uniform image processing is performed on image data as described in Patent Document 1. However, the type of appropriate image processing may differ depending on the surface state of the sample.

[0010] The present invention has been conceived in view of the above actual situation, and an object of the present invention is to provide a technique for determining the type of correction according to the surface state of a sample, with respect to an image of the sample generated by measurement using a scanning probe microscope.

[0011] Solutions for solving problems

[0012] An image processing method according to a certain aspect of the present disclosure is a method for processing an object image generated based on the measurement of a sample performed by a scanning probe microscope, the image processing method including a step of generating a corrected image by performing a first correction on the object image, wherein the first correction includes the following processing: extracting multiple pixels on a straight line along a specified direction in a specified plane from the object image; and correcting the height of the object image based on the brightness of each extracted pixel, the image processing method also includes the following steps: generating a histogram of pixel values ​​in the corrected image; and using the histogram to determine whether a second correction different from the first correction needs to be performed on the object image.

[0013] An image processing device according to one aspect of the present disclosure includes: one or more processors; and a storage device storing a program that, when executed by the one or more processors, causes the one or more processors to implement the above-described image processing method.

[0014] A scanning probe microscope according to one aspect of the present disclosure includes the above-mentioned image processing device.

[0015] The program according to one aspect of the present disclosure is executed by one or more processors to cause the one or more processors to implement the above-mentioned image processing method.

[0016] Effects of the Invention

[0017] According to one aspect of the present disclosure, a correction image is generated by applying a first correction to an object image, and then a histogram of pixel values ​​in the correction image is used to determine whether a second correction needs to be applied to the object image. The histogram of pixel values ​​reflects the surface state of the sample. Therefore, according to one aspect of the present disclosure, whether a second correction needs to be applied is determined based on the surface state of the sample, thereby determining the type of correction corresponding to the surface state of the sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a schematic structural diagram of a scanning probe microscope according to one embodiment.

[0019] Figure 2 It is a diagram for explaining the content of the first correction.

[0020] Figure 3This is a diagram showing an example of the shape of a sample observed by a scanning probe microscope.

[0021] Figure 4 It is shown along Figure 3 A graph of the height of the Y1-Y1 line.

[0022] Figure 5 This is a diagram showing another example of the shape of a sample observed by a scanning probe microscope.

[0023] Figure 6 It is shown along Figure 5 A graph of the height of the Y2-Y2 line.

[0024] Figure 7 FIG. 1 is a diagram schematically showing the result of edge extraction performed on an image obtained as a result of observation using a scanning probe microscope.

[0025] Figure 8 FIG. 2 is a diagram schematically showing the result of a process of dilating the extracted edges.

[0026] Fig. 9 It is a diagram schematically showing the result of binarization processing.

[0027] Fig.10 is a diagram schematically showing the result of the hole filling process.

[0028] Fig.11 This is a diagram for schematically explaining the data structure of the substrate region.

[0029] Fig.12 This is a diagram schematically showing a part of the process of tilt correction in the scanning probe microscope 1 .

[0030] Fig.13 This is a diagram schematically showing a part of the process of tilt correction in the scanning probe microscope 1 .

[0031] Fig.14 This is a flowchart of an example of processing performed for image processing in the scanning probe microscope 1 .

[0032] Fig.15 : is a diagram showing an example of the histogram generated in step SA3.

[0033] Fig.16 : is a diagram showing another example of the histogram generated in step SA3.

[0034] Fig.17 This is a flowchart of a subroutine related to the execution of the second correction in step SA5.

[0035] Fig.18This is a flowchart of a first modified example of the subroutine of step SA5 in the modified example (1).

[0036] Fig.19 This is a flowchart of a second variation of the subroutine of step SA5. DETAILED DESCRIPTION

[0037] Embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, the same reference numerals are attached to the same or corresponding parts in the drawings, and their description will not be repeated.

[0038] [Schematic structure of a scanning probe microscope]

[0039] Figure 1 1 is a schematic structural diagram of a scanning probe microscope according to an embodiment. An example of a scanning probe microscope is an atomic force microscope. In addition, the scanning probe microscope may also be another type of scanning probe microscope (eg, a scanning tunneling microscope).

[0040] Reference Figure 1 The scanning probe microscope 1 includes: a sample stage 112 for placing a sample 110; a piezoelectric scanner 111 for displacing the sample stage; a cantilever 113 having a probe 114 formed at the front end; a displacement detection mechanism 120 for detecting the displacement of the cantilever 113; a feedback signal generating unit 131; a computer 132; a scanning signal generating unit 133; a storage device 134; and a display unit 135. In one implementation example, the computer 132 includes at least one processor, and the storage device 134 stores a program executed by the processor in a non-volatile manner.

[0041] The piezoelectric scanner 111 includes a Z scanner 111z that generates displacement in the Z direction based on a voltage value Vz, and an XY scanner 111xy that generates displacement in the XY direction based on voltage values ​​Vx and Vy.

[0042] The displacement detection mechanism 120 includes a laser diode 115 and a photodetector 119. In the scanning probe microscope 1, when the tip of the probe 114 is brought close to the sample 110 to observe the surface, the laser light emitted from the laser diode 115 is reflected by the back surface of the cantilever 113, and the reflected light is received by the photodetector 119. When the probe 114 is brought close to the surface of the sample 110, the cantilever 113 is bent like a leaf spring, and the amount of bending is observed at the light receiving position of the photodetector 119.

[0043] The feedback signal generating unit 131 receives the detection signal from the photodetector 119. The feedback signal generating unit 131 calculates the deflection amount of the cantilever 113 based on the detection signal. The feedback signal generating unit 131 controls the Z-direction position of the sample so that the atomic force between the probe 114 and the surface of the sample 110 is always constant. The feedback signal generating unit 131 calculates the voltage value Vz that causes the piezoelectric scanner 111 to displace along the Z-axis direction based on the deflection amount of the cantilever 113, and outputs the voltage value Vz to the Z scanner 111z.

[0044] The scanning signal generator 133 calculates the voltage value Vx in the X-axis direction and the voltage value Vy in the Y-axis direction so that the sample 110 moves relative to the probe 114 in the XY plane according to a predetermined scanning pattern, and outputs the voltage values ​​Vx and Vy to the XY scanner 111 xy.

[0045] The signal reflecting the feedback amount in the Z-axis direction (the applied voltage Vz to the scanner and the deviation signal Sd) is also sent to the computer 132 and stored in the storage device 134. The computer 132 calculates the displacement of the surface caused by the unevenness of the sample 110 according to the voltage Vz based on the relevant information representing the relationship between the voltage Vz and the displacement of the surface caused by the unevenness of the sample 110 corresponding to it, which is pre-stored in the storage device 134. The computer 132 reproduces the three-dimensional image of the sample surface by calculating the displacement for each position in the X-axis and Y-axis directions, and depicts the three-dimensional image on the screen of the display unit 135. The data of the three-dimensional image is also stored in the storage device 134. The data includes the coordinates representing the position on the XY plane and the sample height of the coordinates. The computer 132 can read the data of the three-dimensional image stored in the storage device 134 at any time and display the three-dimensional image on the display unit 135.

[0046] The computer 132 can perform height correction on the data of the three-dimensional image as necessary and display the three-dimensional image on the display unit 135 .

[0047] Regarding the altitude correction, the computer 132 can perform at least two types of corrections. In this specification, the two types of corrections may be referred to as "first correction" and "second correction", respectively.

[0048] [First Correction]

[0049] In one implementation example, as a first correction, as described in Japanese Patent Gazette No. 6631647, the computer 132 extracts the brightness of multiple pixels on a straight line along a specified direction in an image of the surface of the sample 110, which should be horizontal, and then performs a height correction based on the brightness of the extracted multiple pixels.

[0050] Figure 2 It is a diagram for explaining the content of the first correction. Figure 2 IM90 is shown as an example of a surface image of the sample 110. As a specific example of the sample 110 corresponding to the image IM90, for example, a pattern produced by a semiconductor process is given. Figure 2 It also shows that Figure 1 Same for the X and Y axes. Also, in Figure 2 , line L90 is a straight line along the Y axis, representing a straight line used to extract the brightness of the image in the first correction.

[0051] [Second Correction]

[0052] As an example of the second correction, the computer 132 generates an image for specifying the region of the sample 110 corresponding to the substrate, and performs height correction on the image. Next, the second correction will be described.

[0053] <Ideal Observation Example>

[0054] Figure 3 is a diagram showing an example of the shape of a sample observed by a scanning probe microscope. Figure 3 In the sample 110 corresponding to the image IM01 shown, 16 structures are arranged on the substrate. The 16 structures are arranged in four rows along the X-axis direction and four rows along the Y-axis direction, that is, arranged in a 4x4 state. An example of the substrate is a mica board. An example of the structure is a nanoparticle or nanofiber of a biological sample. In addition, these are only examples, and the samples (substrates, structures) used as the object of the scanning probe microscope 1 are not limited to them.

[0055] The lightness or darkness of the pixels in the image IM01 indicates the height in the Z-axis direction. A dark pixel indicates a higher position of the sample in the Z-axis direction, and a light pixel indicates a lower position of the sample in the Z-axis direction.

[0056] Figure 4 It is shown along Figure 3 The height of the Y1-Y1 line in the graph. Figure 4 In the graph of , line L30 indicates the height generated based on ideal observation by the scanning probe microscope 1. More specifically, line L30 indicates the amount of one line of image data observed in an ideal case where there is no inclination of the substrate in the sample.

[0057] In addition, Figure 3 In the image IM01, the plane boundary (the boundary between the substrate and the structure) is represented by a line, but Figure 4In order to make the correction process easier to understand, a taper is set at the plane boundary to approximate the actual sample shape. The horizontal axis represents the position in the X direction, and the Z axis represents the height at each X position. In addition, when the probe is scanned along the X direction, the X axis also corresponds to the time axis.

[0058] <Observation Example with the Sample Tilt>

[0059] Figure 5 This is a diagram showing another example of the shape of a sample observed by a scanning probe microscope. Figure 5 The image IM10 shown corresponds to the same Figure 3 The sample 110 corresponding to the image IM01 shown is the same sample. In the image IM10, shades are generated on the same plane. In particular, in the image IM10, a large amount of shades are generated in the area other than the area corresponding to the 16 structures (the area corresponding to the substrate). One of the reasons for the generation of such shades is that the sample 110 is set at an angle on the sample stage 112.

[0060] Figure 6 It is shown along Figure 5 The height of the Y2-Y2 line in the graph. Figure 6 In FIG. 1 , line L10 indicates the height of the sample 110 assumed from the line Y2 - Y2 of the image IM10 . The line L10 shows that the surface of the sample 110 is inclined such that the surface of the sample 110 becomes higher as the X coordinate increases.

[0061] In the scanning probe microscope 1, correction is applied to an image generated based on the observation result, such as the image IM10, and the corrected image is provided. Thus, the surface state of the sample 110 can be more accurately recognized based on the corrected image. Next, the content of the correction applied to the image will be described.

[0062] <Edge Extraction>

[0063] Figure 7 Schematically shows the result of edge extraction of an image obtained as a result of observation in a scanning probe microscope. In one implementation example, the image IM10 is subjected to a known process for edge extraction (Process / Find Edges in the open source ImageJ (https: / / imagej.nih.gov / ij / ) etc.) to obtain Figure 7 Image IM11 is shown.

[0064] The process of extracting edges to obtain the image IM11 typically includes performing differentiation processing on the height image. In addition, the process of extracting edges may also use a general contour extraction method such as a photo processing technique. In one example, the portion where the absolute value of the difference between adjacent pixel data exceeds a certain threshold may be extracted.

[0065] In the image IM11, pixels constituting the boundaries between the 16 structures and the substrate are detected as edges, and these pixels are shown as black pixels. At this time, the pixels detected as edges are examples of edge pixels.

[0066] <Edge Expansion>

[0067] Figure 8 FIG. 1 is a diagram schematically showing the result of the process of dilating the extracted edge. Figure 7 The image IM11 is obtained by performing maximum value processing Figure 8 Image IM12 is shown.

[0068] exist Figure 7 In the image IM11, edge pixels are shown as black pixels (pixels with relatively high pixel values). Figure 8 In the image IM12 shown, the edge composed of edge pixels is expanded. The image IM12 is an example of a "expanded image".

[0069] In addition, the processing for edge expansion is not limited to the maximum value processing. The type of processing for edge expansion can be appropriately changed according to the representation method of edge pixels, etc. For example, when edge pixels are represented as white pixels (pixels with relatively low pixel values), the minimum value processing can be used as the processing for edge expansion.

[0070] <Binarization>

[0071] Fig. 9 is a diagram schematically showing the result of binarization processing. In one implementation example, by Figure 8 The image IM12 is subjected to a known binarization process (Process / Binary / Make Binary in the open source ImageJ (https: / / imagej.nih.gov / ij / ) etc.) to obtain Fig. 9 The image IM13 is shown. The threshold used for binarization can also be adjusted appropriately by the user and / or the computer 132.

[0072] exist Fig. 9In the image IM13 shown, portions of the sample 110 corresponding to the ends of the 16 structures are represented by black pixels, and other portions are represented by white pixels.

[0073] <Connection>

[0074] The computer 132 may also perform line connection processing on the binarized image IM13. Thus, the edge pixels detected in the image IM10 more accurately represent the outer edge of the structure. The line connection processing can be implemented, for example, by a known technique (Process / Binary / Close in the open source ImageJ (https: / / imagej.nih.gov / ij / )).

[0075] <Hole filling process>

[0076] The computer 132 may perform hole filling processing on the binarized image IM13 (or an image obtained by performing line connection processing on the image IM13).

[0077] Fig.10 Schematically shows the result of the hole filling process. The hole filling process can be realized by, for example, a known technique (Process / Binary / Fill Holes in the open source ImageJ (https: / / imagej.nih.gov / ij / )).

[0078] exist Fig.10 In the image IM14 shown, the region corresponding to each of the 16 structures in the sample 110 is indicated by filling in black pixels, and the other regions are indicated by filling in white pixels.

[0079] <Substrate Area>

[0080] The computer 132 can be Fig.10 In the image IM14 shown, a region of white pixels is extracted as a region corresponding to the substrate in the sample (a region where no structure is placed). Data for specifying a region corresponding to the substrate (substrate region) is also referred to as “substrate region data”.

[0081] Fig.11 This is a diagram for schematically explaining the data structure of the substrate region. Fig.11 The representation in the region containing the substrate is along Fig.10 The heights of the Y3-Y3 line are lines L11, L12, L13, L14, and L15.

[0082] As a comparison, Fig.11 It also shows Figure 6The end of each of the lines L11 to L15 is located at a position corresponding to the length D1 on the inside of the portion of the line L10 corresponding to the substrate. This corresponds to the following situation: after the edge detected for the structure is expanded, the substrate area is determined as an area other than the area determined by the expanded edge. That is, in the scanning probe microscope 1, the area of ​​the structure is widely determined by expanding the edge of the structure, thereby reliably avoiding the situation where the structure is included in the substrate area.

[0083] <Tilt Correction>

[0084] Fig.12 FIG. 1 is a diagram schematically showing a part of the process of tilt correction in the scanning probe microscope 1. Fig.12 In the figure, the dotted line indicates that Fig.11 The lines L11 to L15 shown are data obtained by filling (interpolating) one line. The line formed by data interpolation is shown as line A10.

[0085] In one implementation example, the computer 132 generates a straight line (line A10) estimated from the points constituting the lines L11 to L15 in data interpolation. The generation of a straight line is an example of the generation of data for filling. Then, the computer 132 implements data interpolation by filling the corresponding part of the generated straight line with the data of the part other than the five lines L11 to L15.

[0086] The computer 132 stores the entire data of the substrate area (not only Fig.10 Y3-Y3 line, and other areas) Fig.12 The same data interpolation as shown in the data interpolation (generation of data for filling and filling of the data). Thus, data corresponding to the area of ​​the entire sample, that is, data representing the surface of the substrate corresponding to the area of ​​the entire sample is generated. The image represented by the data generated here virtually represents the substrate in a state where the structure is removed from the sample, which is an example of a "substrate display image". It is expected that the inclination of the surface generated here represents the inclination of the sample 110 in the sample stage 112.

[0087] Fig.13 1 is a diagram schematically showing a part of the process of tilt correction in the scanning probe microscope 1. The computer 132 calculates the Fig.12 The image IM10 is corrected so as to correct the calculated inclination of the surface of the data generated in the described steps with respect to the ideal surface.

[0088] exist Fig.13 In the reference Fig.12The orientation of the surface of the data generated in the described process is shown as line A10. The ideal orientation of the surface is shown as line A20. The computer 132 calculates the inclination of line A10 relative to line A20. This inclination is equivalent to the inclination of the sample 110. Then, the computer 132 corrects the image IM10 ( Figure 5 ). Through this correction, Fig.13 Line L20 is generated from line L10 as shown. Similarly, the entire area of ​​image IM10 is corrected to generate a corrected image. It is expected that this correction will make image IM10 close to image IM01 ( Figure 3 ). In this specification, the image after the second correction is also referred to as a "corrected image".

[0089] [Processing flow]

[0090] Fig.14 FIG. 1 is a flowchart of an example of processing performed for image processing in the scanning probe microscope 1. In one implementation example, the processing is performed by executing a given program by the processor of the computer 132. Fig.14 In this sense, the computer 132 is an example of an image processing device.

[0091] In step SA1, the scanning probe microscope 1 acquires image data as an observation result. An example of the image data acquired here is similar to Figure 5 In this specification, an image that is an observation result is an example of a "target image".

[0092] In step SA2 , the scanning probe microscope 1 performs the above-mentioned “first correction” on the target image.

[0093] In step SA3 , the scanning probe microscope 1 generates a histogram of pixel values ​​for the target image after the first correction performed in step SA2 .

[0094] Fig.15 is a diagram showing an example of a histogram generated in step SA3. Fig.15 As shown in , the horizontal axis of the histogram represents the pixel value, and the vertical axis of the histogram represents the number of pixels. Fig.15 In the example of , in the target image, the number of each pixel value is represented by line L91. Line L91 has a peak at pixel value V11.

[0095] Back to Fig.14 In step SA4 , the scanning probe microscope 1 determines whether the number of peaks in the histogram generated in step SA3 is plural.

[0096] For example, the number and positions of peaks in the histogram can be determined by performing known peak detection processing on the histogram. In addition, the scanning probe microscope 1 may perform smoothing processing on a graph such as that indicated by the line L91 before the peak detection processing.

[0097] The number and position of the peaks in the histogram may also be determined by the user. That is, the scanning probe microscope 1 may display the histogram generated in step SA3 on the display unit 135. Alternatively, the user observes the histogram displayed on the display unit 135, determines the number (and position) of the peaks, and inputs it to the scanning probe microscope 1. Alternatively, the scanning probe microscope 1 may be provided with an input device such as a keyboard, and the user inputs the number (and position) of the peaks using the input device. The scanning probe microscope 1 may also use the input number of peaks to implement the determination of step SA4.

[0098] When the scanning probe microscope 1 determines that the number of peaks in the above-mentioned histogram is plural ("Yes" in step SA4), the control enters step SA6. If it is determined that there are not plural peaks, that is, it is determined that there is a single peak ("No" in step SA4), the control enters step SA6. Here, an example of a hypothetical correlation between the number of peaks in the histogram and the morphology of the surface of the sample 110 is described.

[0099] Fig.16 FIG. 4 is a diagram showing another example of a histogram generated in step SA3. Fig.16 In the example of , the number of each pixel value in the target image is represented by line L92. Line L92 has peaks at pixel value V21 and pixel value V22. That is, line L92 has two peaks.

[0100] Whether there are multiple peaks in the histogram reflects whether the surface of the sample 110 has a concavo-convex structure. More specifically, it is assumed that the histogram of the image of the surface has multiple peaks when the surface of the sample 110 has a concavo-convex structure. For example, as shown in FIG. Figure 3 In the case where a structure is arranged on a substrate in the sample as described in , the plurality of peaks include a peak corresponding to the height of the substrate and a peak corresponding to the height of the structure. On the other hand, the following case is assumed: Figure 2 As described above, when the surface of the sample 110 has no (macroscopic) concavoconvex structure, the histogram of the image of the surface has a single peak.

[0101] Back to Fig.14 In step SA5, the scanning probe microscope 1 performs a second correction on the object image. Fig.17 The implementation of the second correction will be described.

[0102] In step SA6, the scanning probe microscope 1 displays the processing result on the display unit 135. The displayed result may include an image generated by the first correction or the second correction as a corrected image. The image displayed as the corrected image is an image after the second correction is performed in step SA6, and an image after the first correction is performed in step SA6. Thereafter, the scanning probe microscope 1 uses Fig.14 The processing is finished.

[0103] Fig.17 This is a flowchart of a subroutine related to the execution of the second calibration in step SA5.

[0104] In step S10 , the scanning probe microscope 1 acquires a target image. When the target image acquired in step SA1 is stored in the storage device 134 , the target image may be read from the storage device 134 in step S10 .

[0105] In step S12, the scanning probe microscope 1 extracts edges from the image acquired in step S10. The edges are extracted, for example, by referring to Figure 7 The method described is implemented.

[0106] In step S14, the scanning probe microscope 1 expands the edge extracted in step S12. The expansion of the edge is performed by, for example, referring to Figure 8 The method described is implemented.

[0107] In step S16, the scanning probe microscope 1 binarizes the image after edge dilation in step S14. Binarization is performed by referring to, for example, Fig. 9 The method described is implemented.

[0108] In step S18, the scanning probe microscope 1 performs line connection processing on the image binarized in step S16. The line connection processing is performed by referring to Fig.10 The method described is implemented.

[0109] In step S20, the scanning probe microscope 1 performs hole filling processing on the image that has been subjected to the line connection processing in step S18. The hole filling processing is performed by referring to Fig.10 The method described is implemented.

[0110] In step S22, the scanning probe microscope 1 generates substrate region data using the image after the hole filling process in step S20. The substrate region data is data for specifying the substrate region.

[0111] In step S24, the scanning probe microscope 1 uses the second data to perform correction (height correction) of the image data acquired in step S10. The correction of the image data is performed, for example, by referring to Fig.12 and Fig.13 The scanning probe microscope 1 returns the control to the control of step S24. Fig.14 .

[0112] In reference Fig.12 and Fig.13 In the method described, for data used to determine the substrate area, such as Fig.12 The area corresponding to the structure is filled with data as indicated by the dotted line (line A10) in FIG. 1. In addition, in the correction of the image data, such filling of data may be omitted. The scanning probe microscope 1 may also calculate the tilt (relative to the ideal surface) of the sample 110 based only on the substrate area.

[0113] In the present embodiment described above, the first correction is applied to the target image, and a histogram of the image after the first correction is applied is generated. Furthermore, if the number of peaks in the histogram is plural, the image after the second correction is applied is displayed as the "corrected image" in step SA6. On the other hand, if the number of peaks in the histogram is not plural, the image after the first correction is displayed as the "corrected image" in step SA6.

[0114] In the second correction, a "dilated image" is generated by performing dilation processing on the edge of the structure, and then the "dilated image" is used to generate "substrate region data" for specifying a region corresponding to the substrate in the sample (a region on the substrate where the structure does not exist). Thus, the region specified by the "substrate region data" is reliably prevented from including an image of the structure. In addition, the region other than the region specified by the "substrate region data" is specified as a region that reliably includes the entire structure.

[0115] When the tilt in the object image caused by the tilt of the sample is corrected using the area determined by the "substrate area data", more points (or lines or areas) can be used in the correction. As a result, the tilt can be corrected more accurately in the correction. Therefore, the surface state of the sample can be accurately provided to the user.

[0116] [Modification (1)]

[0117] In the modification (1), the scanning probe microscope 1 corrects the target image using the substrate region data in the second correction, and specifies the region corresponding to the structure again based on the corrected image data.

[0118] Fig.18 This is a flowchart of a first modified example of the subroutine of step SA5 in the modified example (1). Fig.18 Processing and Fig.17 The processing of also includes the control of steps S10 to S24. Fig.18 In the process of , the scanning probe microscope 1 advances control to step S30 after step S24.

[0119] In step S30, the scanning probe microscope 1 identifies a region corresponding to the structure in the corrected image generated in step S24. The corrected image generated in step S24 is an example of a “corrected image”.

[0120] The determination of the region corresponding to the structure in step S30 may also include, for example, the same control as steps S12 to S22. That is, the scanning probe microscope 1 extracts edges from the corrected image, expands the extracted edges, binarizes the image after the edge expansion, performs line connection processing on the binarized image, performs hole filling processing on the image after the line connection processing, and then generates substrate region data of the image after the hole filling processing. Then, the scanning probe microscope 1 determines the region other than the region determined by the substrate region data (i.e., the region other than the substrate region) as the region corresponding to the structure.

[0121] In step S32, the scanning probe microscope 1 generates a "structure image" by extracting the region corresponding to the structure determined in step S30 from the image data corrected in step S24. The structure image includes an image corresponding to the structure. After that, the scanning probe microscope 1 returns control to Fig.14 In the modification (1), the result displayed in step SA6 may include the image of the structure image generated in step S30, that is, the image corresponding to the structure.

[0122] In the above-described modification (1), an area other than the area determined by the substrate area data (i.e., an area other than the substrate area) is extracted from the object image and displayed as a structure image. The substrate area data is generated using an expansion image, which is generated by expansion processing. Thus, the area other than the area determined by the substrate area data reliably includes the entire area of ​​the structure. Therefore, the state of the structure is accurately provided to the user, thereby being able to accurately provide the surface state of the sample.

[0123] [Variation (2)]

[0124] In variant (2), the scanning probe microscope 1 generates an image of the simulated sample as a whole by combining the structure image of variant (1) with the background data of the region embedded in the substrate. In variant (2), the "image of the simulated sample as a whole" generated in this way is an image after the second correction is applied.

[0125] Fig.19 4 is a flowchart of a second variant of the subroutine of step SA5. Fig.19 In the process of , the scanning probe microscope 1 advances the control to step S40 after step S22.

[0126] In step S40, the scanning probe microscope 1 generates a structure image using the object image acquired in step S10 and the substrate region data generated in step S22. The structure image generated in step S40 is generated by extracting a region other than the region of the substrate determined by the substrate region data generated in step S22 from the object image acquired in step S10.

[0127] In step S42, the scanning probe microscope 1 fills the background data in the area other than the area determined by the structure image generated in step S40, thereby generating browsing data. The browsing data represents the image of the simulated sample as a whole. In the image of the simulated sample as a whole, the structure in the object image acquired in step S10 and the background image are combined.

[0128] After that, the scanning probe microscope 1 returns control to Fig.14 In the modification (2), the result displayed in step SA6 may include the image of the browsing data generated in step S42, that is, the image of the entire simulated sample.

[0129] In the image of the browsing data of modification (2) (the image of the entire simulated sample), the area corresponding to the substrate area has appropriate brightness, so it is not necessary to adjust the contrast when displaying the image of the browsing data. Therefore, the user can visually recognize the result of the processing without having to perform complicated operations such as adjusting the contrast. In addition, in the prior art, sometimes a wave of pixel concentration is generated in the background during the adjustment of the contrast. Since the adjustment of the contrast can be omitted in modification (2), it is possible to avoid the generation of the above-mentioned wave of pixel concentration in the image displayed to the user. In addition, the browsing data can also be generated by combining the structure image (S30) generated in modification (1) with the background data (S42).

[0130] The simulated image of the entire sample includes an image of a region other than the region determined by the substrate region data based on the object image. The substrate region data is generated using an expansion image, which is generated by expansion processing. Thus, the region other than the region determined by the substrate region data reliably includes the entire region of the structure. Therefore, by providing the simulated image of the entire sample, the state of the structure is accurately provided to the user, thereby accurately providing the surface state of the sample.

[0131] [Way]

[0132] It will be understood by those skilled in the art that the above-mentioned multiple exemplary embodiments are specific examples of the following aspects.

[0133] (Item 1) An image processing method involved in one embodiment may be a method for processing an object image generated based on the measurement of a sample performed by a scanning probe microscope, the image processing method including the step of generating a corrected image by performing a first correction on the object image, wherein the first correction includes the following processing: extracting a plurality of pixels on a straight line along a specified direction in a specified plane from the object image; and correcting the height of the object image based on the brightness of each extracted pixel, the image processing method also includes the following steps: generating a histogram of pixel values ​​in the corrected image; and using the histogram to determine whether a second correction different from the first correction needs to be performed on the object image.

[0134] According to the image processing method described in the first aspect, the type of correction can be determined according to the surface state of the sample.

[0135] (Second item) In the image processing method described in the first item, the step of determining whether the second correction needs to be implemented may also include the following processing: determining that the second correction needs to be implemented when the histogram contains multiple peaks; and determining that the second correction does not need to be implemented when the histogram does not contain multiple peaks.

[0136] According to the image processing method described in the second aspect, it is possible to determine the type of correction that reliably corresponds to the surface state of the sample.

[0137] (Item 3) In the image processing method described in Item 1 or Item 2, it may also be that the sample includes a substrate and a structure on the substrate, and the second correction includes the following steps: generating substrate area data for determining an area corresponding to the substrate in the object image; and performing height correction based on pixels of the area determined by the substrate area data.

[0138] According to the image processing method described in the third aspect, it is possible to provide an accurate surface state of the sample through the second correction.

[0139] (Item 4) In the image processing method described in Item 3, the second correction may also include the following steps: extracting pixels from the object image that satisfy the condition of being an edge as a comparison result with adjacent pixels as edge pixels; and generating an expanded image by performing a dilation process on the object image to dilate an edge formed by the edge pixels, and the step of generating the substrate area data includes using the expanded image to generate the substrate area data.

[0140] According to the image processing method described in the fourth aspect, a more accurate surface state of the sample can be provided through the second correction.

[0141] (Item 5) In the image processing method described in Item 4, the dilation processing may also include maximum value processing.

[0142] According to the image processing method described in the fifth item, the structure is reliably expanded by the edge pixels in the expansion process.

[0143] (Item 6) In the image processing method described in any one of Items 3 to 5, the second correction may also include the following processing: filling the filling data into the area in the object image other than the area determined by the substrate area data, thereby generating a substrate display image representing an image of only the substrate.

[0144] According to the image processing method described in the sixth aspect, it is possible to more accurately perform height correction of original image data.

[0145] (Item 7) In the image processing method described in Item 6, generating the substrate display image may also include the following processing: using the pixel value of the area specified by the substrate area data in the target image to generate the data for filling.

[0146] According to the image processing method described in the seventh item, it is possible to generate padding data as padding data more suitable for original image data.

[0147] (Item 8) In the image processing method described in Item 6 or Item 7, generating the substrate display image may also include the following processing: generating a corrected image by performing a tilt correction on the object image using the substrate display image.

[0148] According to the image processing method described in the eighth aspect, an image in which the tilt is corrected more reliably can be generated as a corrected image.

[0149] (Item 9) The image processing method described in Item 8 may also include the following steps: determining the area in the object image corresponding to the structure in the corrected image; and generating a structure image obtained by extracting the determined area corresponding to the structure from the object image.

[0150] According to the image processing method described in item 9, data that more reliably includes the entire structure can be generated as a structure image.

[0151] (Item 10) The image processing method described in any one of Items 3 to 9 may further include the step of generating a structure image by extracting a region specified by the substrate region data from the target image.

[0152] According to the image processing method described in the tenth item, it is possible to generate an image that more reliably includes the entire structure as a structure image.

[0153] (Item 11) The image processing method described in Item 9 or Item 10 may also include the following step: combining the structure data with background data having background pixels in the area determined by the substrate area data, thereby generating browsing data.

[0154] According to the image processing method described in item 11, it is not necessary to adjust the contrast when displaying the browsing data.

[0155] (Item 12) The image processing method described in Item 4 or Item 5 may also include the following step: performing binarization processing on the expanded image or data resulting from the expanded image to generate the substrate region data.

[0156] According to the image processing method described in the twelfth item, edges can be more prominently expressed in data.

[0157] (Item 13) The image processing method described in Item 12 may also include the following steps: performing line connection processing on the image after the binarization processing is performed to generate the substrate area data.

[0158] According to the image processing method described in the thirteenth item, the area corresponding to the substrate can be more reliably determined in the data.

[0159] (Item 14) An image processing device according to one embodiment may include: one or more processors; and a storage device storing a program that is executed by the one or more processors to cause the one or more processors to implement the image processing method described in any one of Items 1 to 13.

[0160] According to the image processing device described in item 14, it is possible to determine the type of correction according to the surface state of the sample.

[0161] (Item 15) A scanning probe microscope according to one embodiment may include the image processing device according to Item 14.

[0162] According to the scanning probe microscope described in the fifteenth item, it is possible to determine the type of correction according to the surface state of the sample.

[0163] (Item 16) The program according to one aspect may be executed by one or more processors to cause the one or more processors to implement the image processing method according to any one of Items 1 to 13.

[0164] According to the program described in the sixteenth item, the type of correction according to the surface state of the sample can be determined.

[0165] It should be considered that the embodiments disclosed this time are illustrative in all aspects and are not restrictive. The scope of the present invention is not shown by the description of the above-mentioned embodiments, but by the claims, and is intended to include all changes within the meaning and scope equivalent to the claims. In addition, it is intended that each technology in the embodiments can be implemented alone or in combination with other technologies in the embodiments as much as possible as needed.

[0166] Description of Reference Numerals

[0167] 1: scanning probe microscope; 110: sample; 111: piezoelectric scanner; 111xy: XY scanner; 111z: Z scanner; 112: sample stage; 113: cantilever; 114: needle; 115: laser diode; 119: photodetector; 120: displacement detection mechanism; 131: feedback signal generating unit; 132: computer; 133: scanning signal generating unit; 134: storage device; 135: display unit.

Claims

1. An image processing method for processing an object image generated based on measurement of a sample by a scanning probe microscope, The image processing method comprises the step of generating a corrected image by performing a first correction on the target image. in, The first correction includes the following processing: extracting a plurality of pixels on a straight line along a specified direction in a specified plane from the object image; as well as Correcting the height of the object image based on the extracted brightness of each pixel, The image processing method further comprises the following steps: generating a histogram of pixel values ​​in the corrected image; and The histogram is used to determine whether a second correction different from the first correction needs to be performed on the target image.

2. The image processing method according to claim 1, wherein: The step of determining whether the second correction needs to be performed includes the following processing: determining that the second correction needs to be performed when the histogram includes a plurality of peaks; and If the histogram does not include a plurality of peaks, it is determined that the second correction does not need to be performed.

3. The image processing method according to claim 1 or 2, wherein: The sample includes a substrate and a structure on the substrate. The second correction comprises the following steps: generating substrate region data for determining a region corresponding to the substrate in the object image; and Height correction is performed based on pixels in a region specified by the substrate region data.

4. The image processing method according to claim 3, wherein: The second correction further comprises the following steps: Extracting pixels that satisfy the edge condition as a result of comparison with adjacent pixels from the target image as edge pixels; and generating a dilated image by performing a dilation process on the target image to dilate an edge formed by the edge pixels, The step of generating the substrate region data includes generating the substrate region data using the dilated image.

5. The image processing method according to claim 4, wherein: The expansion process includes maximum value processing.

6. The image processing method according to claim 3, wherein: The second correction includes a process of filling in data for filling in a region other than a region specified by the substrate region data in the target image, thereby generating a substrate display image representing an image of only the substrate.

7. The image processing method according to claim 6, wherein: Generating the substrate display image includes a process of generating the filling data using pixel values ​​of a region specified by the substrate region data in the target image.

8. The image processing method according to claim 6, wherein: Generating the substrate display image includes a process of generating a corrected image by performing tilt correction on the object image using the substrate display image.

9. The image processing method according to claim 8, wherein: The following steps are also included: determining, in the corrected image, a region in the object image corresponding to the structure; as well as A structure image is generated by extracting the identified region corresponding to the structure from the target image.

10. The image processing method according to claim 3, wherein: The following steps are also included: A structure image is generated by extracting a region specified by the substrate region data from the target image.

11. The image processing method according to claim 9, wherein: The following steps are also included: The browsing data is generated by combining the structure image with background data having background pixels in the area specified by the substrate area data.

12. The image processing method according to claim 4, wherein: The following steps are also included: Binarization processing is performed on the expansion image or data resulting from the expansion image to generate the substrate region data.

13. The image processing method according to claim 12, wherein: The following steps are also included: A line connection process is performed on the image subjected to the binarization process to generate the substrate region data.

14. An image processing device comprising: More than one processor; and A storage device storing a program which, when executed by the one or more processors, causes the one or more processors to implement the image processing method according to claim 1 or 2. 15 . A scanning probe microscope comprising the image processing device according to claim 14 . 16 . A program that, when executed by one or more processors, causes the one or more processors to implement the image processing method according to claim 1 or 2 .

Images