Image processing method, image processing device, scanning probe microscope, and program

Abstract

Provided is a technology for identifying the type of a correction according to the state of the surface of a sample, an image of which has been generated on the basis of the measurement using a scanning probe microscope An image processing method according to the present invention comprises a step (SA1) for generating a corrected image by performing a first correction on a target image. The first correction includes extracting, from the target image, a plurality of pixels on a straight line along a prescribed direction on a prescribed plane and correcting the height of the target image on the basis of the luminance of each of the pixels extracted. The image processing method further comprises: a step (SA3) for generating a histogram of the pixel values in the corrected image; and a step (SA4) for using the histogram to determine whether or not a second correction, which is different from the first correction, is necessary on the target image.

Description

[Technical Field]

[0001] The present invention relates to the processing of images obtained by scanning probe microscopes. [Background technology]

[0002] A scanning probe microscope (SPM) observes the topography of a sample surface with high resolution by bringing a sharpened probe close enough to the sample to be observed and scanning the sample surface horizontally with the probe while raising or lowering the probe height so that the physical quantities acting on the tip of the probe and the sample surface remain constant. SPM is a general term for microscopes that observe the topography of a sample surface using the above operating principle. Representative SPMs include the scanning tunneling microscope (STM), which detects the current flowing between the probe and sample as an interaction, and the atomic force microscope (AFM), which detects the atomic force acting between the probe and sample as an interaction.

[0003] Scanning probe microscopes have high resolution in the surface height direction, and it is difficult to set the sample surface horizontally at that resolution level. Therefore, height correction is generally performed on the height image acquired by the scanning probe microscope (hereinafter referred to as SPM image) to correct the inclined surface so that it is horizontal. By providing the corrected image instead of the SPM image, the user can more accurately recognize the surface condition of the sample.

[0004] As an example of processing images obtained by such a scanning probe microscope, Japanese Patent No. 6627903 (Patent Document 1) discloses a technology in which at least a portion of the area other than the edge in the image data is extracted as a reference plane area, and the height of the measurement data is corrected based on height information of three points belonging to the reference plane area. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6627903 Summary of the Invention [Problem to be solved by the invention]

[0006] In conventional techniques, uniform image processing is performed on image data, as described in Patent Document 1. However, there may be cases where the appropriate type of image processing differs depending on the surface condition of the sample.

[0007] The present invention has been devised in view of the above-described circumstances, and its purpose is to provide a technique for identifying the type of correction to be applied to an image of a sample generated based on measurements using a scanning probe microscope, in accordance with the surface condition of the sample. [Means for solving the problem]

[0008] An image processing method according to one aspect of the present disclosure is a method for processing an object image generated based on measurement of a sample using a scanning probe microscope, comprising the steps of generating a corrected image by performing a first correction on the object image, the first correction including extracting a plurality of pixels from the object image in a straight line along a predetermined direction in a predetermined plane, and correcting the height of the object image based on the brightness of each extracted pixel, and further comprising the steps of generating a histogram of pixel values ​​in the corrected image, and using the histogram to determine whether a second correction different from the first correction is required for the object image.

[0009] An image processing device according to an aspect of the present disclosure includes one or more processors and a storage device storing a program that, when executed by the one or more processors, causes the one or more processors to perform the image processing method described above.

[0010] A scanning probe microscope according to an aspect of the present disclosure includes the image processing device described above. A program according to an aspect of the present disclosure, when executed by one or more processors, causes the one or more processors to perform the image processing method described above. [Effects of the Invention]

[0011] According to an aspect of the present disclosure, a corrected image is generated by performing a first correction on a target image, and a histogram of pixel values ​​in the corrected image is used to determine whether a second correction is required for the target image. The histogram of pixel values ​​reflects the surface condition of the sample. Therefore, according to an aspect of the present disclosure, whether the second correction is required is determined based on the surface condition of the sample, and the type of correction depending on the surface condition of the sample can be identified. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic configuration diagram of a scanning probe microscope according to an embodiment. [Figure 2] FIG. 10 is a diagram for explaining the contents of a first correction. [Figure 3] FIG. 1 is a diagram showing an example of the shape of a sample observed with a scanning probe microscope. [Figure 4] 4 is a graph showing the height along the Y1-Y1 line in FIG. 3. [Figure 5] 10A and 10B are diagrams showing other examples of the shape of a sample observed with a scanning probe microscope. [Figure 6] 6 is a graph showing the height along the Y2-Y2 line in FIG. 5. [Figure 7] FIG. 10 is a diagram schematically showing the results of edge extraction for an image obtained as an observation result using a scanning probe microscope. [Figure 8] FIG. 10 is a diagram illustrating the results of a process of expanding extracted edges. [Figure 9] FIG. 10 is a diagram schematically illustrating the results of binarization processing. [Figure 10] FIG. 10 is a diagram schematically illustrating the results of filling processing. [Figure 11]FIG. 2 is a diagram for schematically explaining the data configuration of a board area. [Figure 12] 3A to 3C are diagrams showing a part of the process of tilt correction in the scanning probe microscope 1. [Figure 13] 3A to 3C are diagrams showing a part of the process of tilt correction in the scanning probe microscope 1. [Figure 14] 10 is a flowchart of an example of processing carried out for image processing in the scanning probe microscope 1. [Figure 15] FIG. 10 is a diagram showing an example of a histogram generated in step SA3. [Figure 16] FIG. 10 is a diagram showing another example of the histogram generated in step SA3. [Figure 17] 10 is a flowchart of a subroutine related to the execution of the second correction in step SA5. [Figure 18] 10 is a flowchart of a first modified example of the subroutine of step SA5 in the modified example (1). [Figure 19] 10 is a flowchart of a second modified example of the subroutine of step SA5. DETAILED DESCRIPTION OF THE INVENTION

[0013] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail with reference to the accompanying drawings, in which the same or corresponding parts are designated by the same reference numerals and will not be described repeatedly.

[0014] [Schematic configuration of a scanning probe microscope] 1 is a schematic diagram of a scanning probe microscope according to one embodiment. An example of a scanning probe microscope is an atomic force microscope. However, the scanning probe microscope may be another type of scanning probe microscope (for example, a scanning tunneling microscope).

[0015] 1, the scanning probe microscope 1 includes a sample stage 112 on which a sample 110 is placed, a piezoelectric scanner 111 that displaces the sample stage, a cantilever 113 having a probe 114 formed at the tip thereof, a displacement detection mechanism 120 that detects the displacement of the cantilever 113, a feedback signal generator 131, a computer 132, a scanning signal generator 133, a storage device 134, and a display device 135. In one implementation example, the computer 132 includes at least one processor, and the storage device 134 stores in a non-volatile manner a program executed by the processor.

[0016] The piezo scanner 111 includes a Z scanner 111z that generates a displacement in the Z direction based on a voltage value Vz, and an XY scanner 111xy that generates a displacement in the X and Y directions based on voltage values ​​Vx and Vy.

[0017] Displacement detection mechanism 120 includes laser diode 115 and photodetector 119. In scanning probe microscope 1, when the tip of probe 114 is brought close to sample 110 to observe the surface, laser light emitted from laser diode 115 is reflected by the back surface of cantilever 113, and the reflected light is received by photodetector 119. As probe 114 is brought closer to the surface of sample 110, cantilever 113 bends like a leaf spring, and the amount of bending is observed at the light receiving position of photodetector 119.

[0018] The feedback signal generating unit 131 receives a detection signal from the photodetector 119. The feedback signal generating unit 131 calculates the amount of deflection of the cantilever 113 based on the detection signal. The feedback signal generating unit 131 controls the Z-direction position of the sample so that the atomic force between the probe 114 and the surface of the sample 110 is always constant. The feedback signal generating unit 131 calculates a voltage value Vz for displacing the piezo scanner 111 in the Z-axis direction based on the amount of deflection of the cantilever 113, and outputs the voltage value Vz to the Z scanner 111z.

[0019] The scanning signal generating unit 133 calculates voltage values ​​Vx and Vy in the X-axis and Y-axis directions so that the sample 110 moves relative to the probe 114 in the XY plane according to a predetermined scanning pattern, and outputs them to the XY scanner 111xy.

[0020] A signal reflecting the feedback amount in the Z-axis direction (the voltage Vz applied to the scanner and the deviation signal Sd) is also sent to the computer 132 and stored in the storage device 134. The computer 132 calculates the amount of surface displacement due to the unevenness of the sample 110 from the voltage Vz based on correlation information previously stored in the storage device 134, which indicates the relationship between the voltage Vz and the corresponding amount of surface displacement due to the unevenness of the sample 110. The computer 132 reconstructs a three-dimensional image of the sample surface by calculating the amount of displacement at each position in the X-axis and Y-axis directions, and displays this on the screen of the display unit 135. This three-dimensional image data is also stored in the storage device 134. The data includes coordinates indicating the position on the XY plane and the sample height at those coordinates. The computer 132 can read out the three-dimensional image data stored in the storage device 134 at any time and display it on the display unit 135.

[0021] The computer 132 can correct the height of the three-dimensional image data as needed and display it on the display unit 135 .

[0022] The computer 132 can perform at least two types of height correction. In this specification, the two types of height correction may be referred to as a "first correction" and a "second correction," respectively.

[0023] [First Amendment] In one implementation example, as a first correction, the computer 132 extracts the brightness of multiple pixels on a straight line along a predetermined direction in an image of the surface of the sample 110, which should be horizontal, as described in Japanese Patent No. 6631647, and then performs height correction based on the brightness of the extracted multiple pixels.

[0024] FIG. 2 is a diagram for explaining the contents of the first correction. FIG. 2 shows image IM90 as an example of a surface image of sample 110. A specific example of sample 110 corresponding to image IM90 is a pattern produced by a semiconductor process. FIG. 2 also shows the same X-axis and Y-axis as FIG. 1. In FIG. 2, line L90 is a straight line along the Y-axis, and represents the line from which the brightness of the image is extracted in the first correction.

[0025] [Second Amendment] As an example of the second correction, the computer 132 generates an image that identifies the region of the sample 110 that corresponds to the substrate, and performs height correction on the image. The second correction will be described below.

[0026] <Ideal observation example> FIG. 3 is a diagram showing an example of the shape of a sample observed with a scanning probe microscope. In sample 110 corresponding to image IM01 shown in FIG. 3, 16 structures are arranged on a substrate. The 16 structures are arranged in four rows in the X-axis direction and four rows in the Y-axis direction, i.e., a 4x4 configuration. An example of a substrate is a mica plate. An example of a structure is nanoparticles or nanofibers of a biological sample. Note that these are merely examples, and the samples (substrates, structures) that the scanning probe microscope 1 targets are not limited to these.

[0027] The shade of the pixels in image IM01 represents the height along the Z axis. Darker pixels represent higher positions in the sample along the Z axis. Lighter pixels represent lower positions in the sample along the Z axis.

[0028] Fig. 4 is a graph showing the height along the Y1-Y1 line in Fig. 3. In the graph of Fig. 4, line L30 represents the height generated based on ideal observation by the scanning probe microscope 1. More specifically, line L30 represents one line of image data observed in an ideal case where there is no tilt of the substrate in the sample.

[0029] In image IM01 in Figure 3, the planar boundary (the boundary between the substrate and the structure) is shown by a line, but in Figure 4, to make it easier to understand the correction process, the planar boundary is tapered to resemble the actual sample shape. The horizontal axis indicates the position in the X direction, and the Z axis indicates the height at each X position. When the probe is scanned along the X direction, the X axis also corresponds to the time axis.

[0030] <Example of observation with a tilted sample> FIG. 5 shows another example of the shape of a sample observed with a scanning probe microscope. Image IM10 shown in FIG. 5 corresponds to the same sample as sample 110 corresponding to image IM01 shown in FIG. 3. Image IM10 shows shading variations on the same plane. In particular, image IM10 shows large shading variations in the areas other than the areas corresponding to the 16 structures (the areas corresponding to the substrate). One of the reasons for this shading variation is that sample 110 was placed at an angle on sample stage 112.

[0031] Fig. 6 is a graph showing the height along the Y2-Y2 line in Fig. 5. In Fig. 6, line L10 represents the height of the sample 110 estimated from the Y2-Y2 line in image IM10. Line L10 indicates that the surface of the sample 110 is inclined such that the surface of the sample 110 becomes higher as the X coordinate increases.

[0032] In the scanning probe microscope 1, corrections are made to an image generated based on observation results such as image IM10, and a corrected image is provided. This allows the surface state of the sample 110 to be recognized more accurately from the corrected image. The details of the corrections made to the image are explained below.

[0033] <Edge extraction> Fig. 7 is a diagram showing the result of edge extraction for an image obtained as an observation result with a scanning probe microscope. In one implementation example, image IM11 shown in Fig. 7 is obtained by performing a known process for edge extraction (such as Process / Find Edges in the open source ImageJ (https: / / imagej.nih.gov / ij / )) on image IM10.

[0034] The process of extracting edges to obtain image IM11 typically involves differential processing of the height image. Alternatively, the process of extracting edges may use a contour extraction method commonly used in photo processing techniques. For example, the part where the absolute value of the difference between adjacent pixel data exceeds a certain threshold may be extracted.

[0035] In image IM11, the pixels that form the boundaries between each of the 16 structures and the substrate are detected as edges and are shown as black pixels. Each of the pixels detected as an edge is an example of an edge pixel.

[0036] <Edge expansion> 8 is a diagram showing the result of the process of dilating the extracted edges. In one implementation example, the image IM12 shown in FIG. 8 is obtained by performing maximum value processing on the image IM11 of FIG.

[0037] In image IM11 in Figure 7, edge pixels are shown as black pixels (pixels having relatively high pixel values). Therefore, by performing maximum value processing on image IM11, edges formed by edge pixels are dilated in image IM12 shown in Figure 8. Image IM12 is an example of a "dilated image."

[0038] The process for edge expansion is not limited to maximum value processing. The type of process for edge expansion can be changed as appropriate depending on the representation of edge pixels, etc. For example, if edge pixels are represented as white pixels (pixels having relatively low pixel values), minimum value processing may be adopted as the process for edge expansion.

[0039] <Binarization> Fig. 9 is a diagram schematically illustrating the result of a binarization process. In one implementation example, image IM13 shown in Fig. 9 is obtained by performing a known binarization process (such as Process / Binary / Make Binary in the open source ImageJ (https: / / imagej.nih.gov / ij / )) on image IM12 of Fig. 8. The threshold for binarization may be adjusted as appropriate by the user and / or the computer 132.

[0040] In the image IM13 shown in FIG. 9, portions of the sample 110 corresponding to the ends of the 16 structures are represented by black pixels, and the other portions are represented by white pixels.

[0041] <Line connection> The computer 132 may perform a line-connecting process on the binarized image IM13, so that the edge pixels detected in the image IM10 more accurately represent the outer edges of the structures. The line-connecting process can be realized, for example, by a known technique (such as Process / Binary / Close in the open source ImageJ (https: / / imagej.nih.gov / ij / )).

[0042] <Filling process> The computer 132 may perform a hole filling process on the binarized image IM13 (or an image obtained by performing a line connection process on the image IM13).

[0043] 10 is a diagram schematically illustrating the results of the hole filling process. The hole filling process can be realized by, for example, a known technique (such as Process / Binary / Fill Holes in the open source ImageJ (https: / / imagej.nih.gov / ij / )).

[0044] In the image IM14 shown in FIG. 10, the areas of the sample 110 corresponding to each of the 16 structures are represented by being filled with black pixels, and the other areas are represented by being filled with white pixels.

[0045] <Substrate area> The computer 132 can extract the region of white pixels from the image IM14 shown in Fig. 10 as a region of the sample corresponding to the substrate (a region on which no structure is placed). Data specifying the region corresponding to the substrate (substrate region) is also referred to as "substrate region data."

[0046] Fig. 11 is a diagram for schematically explaining the data configuration of the board area, and includes lines L11, L12, L13, L14, and L15 that indicate the height of the board area along the Y3-Y3 line in Fig. 10.

[0047] 11 also shows the line L10 shown in FIG. 6 for comparison. The ends of each of the lines L11 to L15 are located a length D1 inward from the portion of the line L10 that corresponds to the substrate. This corresponds to the substrate region being identified as a region other than the region identified by the expanded edge after the detected edge of the structure is expanded. That is, in the scanning probe microscope 1, by expanding the edge of the structure, the region of the structure is identified broadly, thereby reliably avoiding the structure from being included in the substrate region.

[0048] <Tilt correction> Fig. 12 is a diagram schematically showing part of the process of tilt correction in the scanning probe microscope 1. In Fig. 12, the data filled in (interpolated) to combine lines L11 to L15 shown in Fig. 11 into a single line is shown by a dashed line. The line formed by data interpolation is shown as line A10.

[0049] In one implementation example, the computer 132 generates a straight line (line A10) inferred from the points that make up lines L11 to L15 during data interpolation. The generation of the straight line is an example of generating data for filling. The computer 132 then performs data interpolation by filling the parts other than the five lines L11 to L15 with data from the corresponding parts of the generated straight line.

[0050] The computer 132 performs data interpolation (generation of data to be filled and filling of the data) similar to that shown in FIG. 12 on the entire data of the substrate area (not only the Y3-Y3 line in FIG. 10 but also other areas). This generates data corresponding to the entire area of ​​the sample, i.e., data representing the surface of the substrate corresponding to the entire area of ​​the sample. The image represented by the data generated here virtually represents the substrate in a state where the structure has been removed from the sample, and is an example of a "substrate display image." The tilt of the surface generated here is expected to represent the tilt of the sample 110 on the sample stage 112.

[0051] Fig. 13 is a diagram schematically illustrating a part of the process of tilt correction in the scanning probe microscope 1. Computer 132 calculates the tilt of the data plane generated in the process described with reference to Fig. 12 relative to an ideal plane, and corrects image IM10 so as to correct the calculated tilt.

[0052] In FIG. 13, the surface orientation of the data generated in the process described with reference to FIG. 12 is shown as line A10. The ideal surface orientation is shown as line A20. The computer 132 calculates the inclination of line A10 relative to line A20. This inclination corresponds to the inclination of the sample 110. The computer 132 then corrects image IM10 ( FIG. 5 ) to offset the calculated inclination. This correction generates line L20 from line L10, as shown in FIG. 13 . Similarly, the entire area of ​​image IM10 is corrected to generate a corrected image. This correction is expected to bring image IM10 closer to image IM01 ( FIG. 3 ). In this specification, the image that has undergone the second correction is also referred to as the “corrected image.”

[0053] [Processing flow] Fig. 14 is a flowchart of an example of a process performed for image processing in the scanning probe microscope 1. In one implementation example, the process in Fig. 14 is performed by the processor of the computer 132 executing a given program. In this sense, the scanning probe microscope 1 is an example of an image processing device.

[0054] In step SA1, the scanning probe microscope 1 acquires image data that is the observation result. An example of the image acquired here corresponds to image IM10 in Figure 5. In this specification, the image that is the observation result is an example of a "target image."

[0055] In step SA2, the scanning probe microscope 1 performs the above-mentioned "first correction" on the target image.

[0056] In step SA3, the scanning probe microscope 1 generates a histogram of pixel values ​​for the target image that has been subjected to the first correction performed in step SA2.

[0057] FIG. 15 is a diagram showing an example of a histogram generated in step SA3. As shown in FIG. 15, the horizontal axis of the histogram represents pixel values, and the vertical axis of the histogram represents the number of pixels. In the example of FIG. 15, the number of each pixel value in the target image is represented by line L91. Line L91 has a peak at pixel value V11.

[0058] Returning to FIG. 14, in step SA4, the scanning probe microscope 1 determines whether or not the histogram generated in step SA3 has a plurality of peaks.

[0059] The number and positions of peaks in the histogram are identified, for example, by performing a known peak detection process on the histogram. Note that the scanning probe microscope 1 may perform a smoothing process on the graph as indicated by line L91 before the peak detection process.

[0060] The number and positions of peaks in the histogram may be specified by the user. That is, the scanning probe microscope 1 may display the histogram generated in step SA3 on the display unit 135. The user may look at the histogram displayed on the display unit 135, specify the number (and positions) of peaks, and input this to the scanning probe microscope 1. The scanning probe microscope 1 may be equipped with an input device such as a keyboard, and the user may input the number (and positions) of peaks using this input device. The scanning probe microscope 1 may use the input number of peaks to make the determination in step SA4.

[0061] If the scanning probe microscope 1 determines that the number of peaks in the histogram is multiple (YES in step SA4), it proceeds to step SA6, but if it determines that this is not the case, that is, if it determines that there is a single peak (NO in step SA4), it proceeds to step SA6. Here, an example of a possible correlation between the number of peaks in the histogram and the surface morphology of the sample 110 will be described.

[0062] Fig. 16 is a diagram showing another example of the histogram generated in step SA3. In the example of Fig. 16, the number of each pixel value in the target image is represented by line L92. Line L92 has peaks at pixel values ​​V21 and V22. In other words, line L92 has two peaks.

[0063] Whether or not there are multiple peaks in the histogram reflects whether or not the surface of the sample 110 has a concave-convex structure. More specifically, if the surface of the sample 110 has a concave-convex structure, the histogram of the image of the surface is expected to have multiple peaks. For example, if structures are arranged on a substrate in the sample as described with reference to FIG. 3, etc., the multiple peaks include a peak corresponding to the height of the substrate and a peak corresponding to the height of the structure. On the other hand, if the surface of the sample 110 does not have a concave-convex structure (in a macroscopic sense), as described with reference to FIG. 2, the histogram of the image of the surface is expected to have a single peak.

[0064] 14, in step SA5, the scanning probe microscope 1 performs a second correction on the target image. The execution of the second correction will be described later with reference to FIG.

[0065] In step SA6, the scanning probe microscope 1 displays the results of the processing on the display unit 135. The displayed results may include an image generated by the first correction or the second correction as a corrected image. The image displayed as a corrected image is an image that has been subjected to the second correction if the second correction has been performed in step SA6, and is an image that has been subjected to the first correction if the second correction has not been performed in step SA6. The scanning probe microscope 1 then terminates the processing of FIG. 14.

[0066] FIG. 17 is a flowchart of a subroutine related to the execution of the second correction in step SA5.

[0067] In step S10, the scanning probe microscope 1 acquires a target image. If the target image acquired in step SA1 is stored in the storage device 134, the target image may be read out from the storage device 134 in step S10.

[0068] In step S12, the scanning probe microscope 1 extracts edges from the image acquired in step S10. The edge extraction is realized by, for example, the method described with reference to FIG.

[0069] In step S14, the scanning probe microscope 1 dilates the edges extracted in step S12. The dilation of the edges is achieved by, for example, the method described with reference to FIG.

[0070] In step S16, the scanning probe microscope 1 binarizes the image whose edges have been dilated in step S14. The binarization is achieved by, for example, the method described with reference to FIG.

[0071] In step S18, the scanning probe microscope 1 performs line connection processing on the image binarized in step S16. The line connection processing is realized by the method described with reference to FIG.

[0072] In step S20, the scanning probe microscope 1 performs a hole filling process on the image that has been subjected to the line connection process in step S18. The hole filling process is realized by, for example, the method described with reference to FIG.

[0073] In step S22, the scanning probe microscope 1 generates substrate area data using the image that has been subjected to the hole filling process in step S20. The substrate area data is data that identifies the substrate area.

[0074] In step S24, the scanning probe microscope 1 uses the second data to correct (correct height) the image data acquired in step S10. The image data correction is realized, for example, by the method described with reference to Figs. 12 and 13. By the control of step S24, an image is generated in which the "second correction" has been applied to the "target image." Then, the scanning probe microscope 1 returns control to Fig. 14.

[0075] 12 and 13, data specifying the substrate region was filled into the region corresponding to the structure, as shown by the dashed line (line A10) in Fig. 12. Note that such data filling may be omitted when correcting the image data. The scanning probe microscope 1 may calculate the tilt of the sample 110 (with respect to the ideal plane) from only the substrate region.

[0076] In the present embodiment described above, a first correction is applied to a target image, and a histogram of the image that has undergone the first correction is generated. If the histogram has multiple peaks, the image that has undergone the second correction is displayed as the "corrected image" in step SA6. On the other hand, if the histogram does not have multiple peaks, the image that has undergone the first correction is displayed as the "corrected image" in step SA6.

[0077] In the second correction, a dilation process is performed on the edges of the structures to generate a "dilated image," and the "dilated image" is then used to generate "substrate region data" that identifies the region of the sample corresponding to the substrate (the region on the substrate where no structures exist). This reliably prevents the region identified by the "substrate region data" from including an image of the structure. Furthermore, regions other than the region identified by the "substrate region data" are reliably identified as regions that include the entire structure.

[0078] When the tilt in the target image caused by the tilt of the sample is corrected using the area specified by the "substrate area data," more points (or lines or areas) can be used for the correction. This allows the tilt to be corrected more accurately. Therefore, the surface condition of the sample can be accurately provided to the user.

[0079] [Variation (1)] In the modification (1), the scanning probe microscope 1 corrects the target image using the substrate region data in the second correction, and identifies the region corresponding to the structure again from the corrected image data.

[0080] Fig. 18 is a flowchart of a first modified example of the subroutine of step SA5 in modified example (1). The processing in Fig. 18 includes the control of steps S10 to S24, similar to the processing in Fig. 17. In the processing in Fig. 18, the scanning probe microscope 1 advances control to step S30 after step S24.

[0081] In step S30, the scanning probe microscope 1 identifies an area corresponding to the structure in the corrected image generated in step S24. The corrected image generated in step S24 is an example of a "corrected image."

[0082] The identification of the region corresponding to the structure in step S30 may include, for example, the same control as steps S12 to S22. That is, the scanning probe microscope 1 extracts edges from the corrected image, expands the extracted edges, binarizes the edge-expanded image, performs line connection processing on the binarized image, performs hole filling processing on the image that has been subjected to line connection processing, and generates substrate region data for the image that has been subjected to hole filling processing. Then, the scanning probe microscope 1 identifies regions other than those identified by the substrate region data (i.e., regions other than the substrate region) as regions corresponding to the structure.

[0083] In step S32, the scanning probe microscope 1 generates a "structure image" by extracting an area corresponding to the structure identified in step S30 from the image data corrected in step S24. The structure image includes an image corresponding to the structure. The scanning probe microscope 1 then returns control to FIG. 14. In modification example (1), the result displayed in step SA6 may include an image of the structure image generated in step S30, i.e., an image corresponding to the structure.

[0084] In the above-described variant (1), an area other than the area specified by the substrate area data (i.e., an area other than the substrate area) is extracted from the target image and displayed as a structure image. The substrate area data is generated using a dilated image, and the dilated image is generated through a dilation process. This ensures that the area other than the area specified by the substrate area data includes the entire structure. Therefore, the user can be provided with an accurate view of the state of the structure, and thereby the surface state of the sample can be provided with an accurate view.

[0085] [Variation (2)] In the modified example (2), the scanning probe microscope 1 generates a pseudo image of the entire sample by combining the structure image of the modified example (1) with background data that fills in the substrate area. In the modified example (2), the "pseudo image of the entire sample" generated in this way is an image that has been subjected to the second correction.

[0086] Fig. 19 is a flowchart of a second modified example of the subroutine of step SA5. In the processing of Fig. 19, the scanning probe microscope 1 advances control to step S40 after step S22.

[0087] In step S40, the scanning probe microscope 1 generates a structure image using the target image acquired in step S10 and the substrate area data generated in step S22. The structure image generated in step S40 is generated by extracting, from the target image acquired in step S10, an area other than the area of ​​the substrate identified by the substrate area data generated in step S22.

[0088] In step S42, the scanning probe microscope 1 generates viewing data by filling in areas other than those identified by the structure image generated in step S40 with background data. The viewing data represents a pseudo image of the entire sample. The pseudo image of the entire sample combines the structure in the target image acquired in step S10 with the background image.

[0089] Thereafter, the scanning probe microscope 1 returns control to Fig. 14. In variant example (2), the results displayed in step SA6 may include an image of the viewing data generated in step S42, i.e., an image of the entire pseudo sample.

[0090] In the image of the viewing data of the modification (2) (a pseudo image of the entire sample), the area corresponding to the substrate region has appropriate brightness, which eliminates the need for contrast adjustment when displaying the image of the viewing data. Therefore, the user can visually confirm the processing results without the need for tedious tasks such as adjusting the contrast. Furthermore, in conventional techniques, contrast adjustment can sometimes result in fluctuations in pixel density in the background. In the modification (2), contrast adjustment can be omitted, thereby avoiding the occurrence of such fluctuations in pixel density in the image displayed to the user. The viewing data may also be generated by combining the structure image (S30) generated in the modification (1) with the background data (S42).

[0091] The pseudo-image of the entire sample includes an image of a region other than the region specified by the substrate region data from the target image. The substrate region data is generated using a dilated image, and the dilated image is generated through a dilation process. This ensures that the region other than the region specified by the substrate region data includes the entire structure. Therefore, by providing an image of the pseudo-image of the entire sample, the user can be provided with an accurate representation of the state of the structure, and thereby the surface state of the sample can be provided with an accurate representation.

[0092] [Aspect] It will be appreciated by those skilled in the art that the exemplary embodiments described above are examples of the following aspects.

[0093] (Item 1) An image processing method according to one embodiment is a method for processing an object image generated based on measurement of a sample using a scanning probe microscope, and includes a step of generating a corrected image by performing a first correction on the object image, the first correction including extracting a plurality of pixels from the object image on a straight line along a predetermined direction in a predetermined plane, and correcting the height of the object image based on the brightness of each extracted pixel, and may also include a step of generating a histogram of pixel values ​​in the corrected image, and a step of determining, using the histogram, whether a second correction different from the first correction is required for the object image.

[0094] According to the image processing method described in item 1, the type of correction can be specified according to the surface condition of the sample.

[0095] (Clause 2) In the image processing method described in clause 1, the step of determining whether the second correction is necessary may include determining that the second correction is necessary if the histogram includes multiple peaks, and determining that the second correction is not necessary if the histogram does not include multiple peaks.

[0096] According to the image processing method described in the second aspect, the type of correction that is reliably suited to the surface condition of the sample can be identified.

[0097] (Clause 3) In the image processing method described in paragraph 1 or 2, the sample may include a substrate and a structure on the substrate, and the second correction may include a step of generating substrate area data that identifies an area in the target image corresponding to the substrate, and a step of performing height correction based on pixels in the area identified by the substrate area data.

[0098] According to the image processing method described in item 3, the second correction can provide an accurate surface state of the sample.

[0099] (4) In the image processing method described in 3, the second correction may further include a step of extracting pixels from the target image that satisfy the condition of being an edge as a result of comparison with adjacent pixels, and a step of generating an expanded image by performing an expansion process on the target image to expand the edges formed by the edge pixels, and the step of generating the substrate area data may include generating the substrate area data using the expanded image.

[0100] According to the image processing method described in item 4, the second correction can provide a more accurate surface state of the sample.

[0101] (Item 5) In the image processing method described in Item 4, the expansion process may include maximum value processing.

[0102] According to the image processing method described in paragraph 5, the structure is reliably expanded by edge pixels in the expansion process.

[0103] (Item 6) In the image processing method described in any one of Items 3 to 5, the second correction may include generating a substrate display image representing an image of only the substrate by filling areas of the target image other than the area identified by the substrate area data with filling data.

[0104] According to the image processing method described in paragraph 6, the height of the original image data can be corrected more accurately.

[0105] (Item 7) In the image processing method described in Item 6, generating the substrate display image may include generating the filling data using pixel values ​​of an area in the target image identified by the substrate area data.

[0106] According to the image processing method described in paragraph 7, data for filling can be generated that is more compatible with the original image data.

[0107] (Item 8) In the image processing method described in Item 6 or Item 7, generating the substrate display image may include generating a corrected image by performing tilt correction on the target image using the substrate display image.

[0108] According to the image processing method described in paragraph 8, an image whose tilt has been corrected more reliably can be generated as the corrected image.

[0109] (Clause 9) The image processing method described in clause 8 may further include the steps of identifying an area in the corrected image that corresponds to the structure in the target image, and generating a structure image from the target image by extracting the area that corresponds to the identified structure.

[0110] According to the image processing method described in paragraph 9, data that includes the entire structure more reliably can be generated as a structure image.

[0111] (Item 10) The image processing method described in any one of items 3 to 9 may further include a step of generating a structure image by extracting an area identified by the board area data from the target image.

[0112] According to the image processing method described in paragraph 10, an image that includes the entire structure more reliably can be generated as a structure image.

[0113] (Clause 11) The image processing method described in clause 9 or 10 may further include a step of generating viewing data by combining the structure data with background data having background pixels in an area specified by the substrate area data.

[0114] According to the image processing method described in paragraph 11, contrast adjustment is not required when displaying the viewing data.

[0115] (Item 12) The image processing method described in item 4 or 5 may further include a step of performing a binarization process on the dilated image or data resulting from the dilated image in order to generate the substrate area data.

[0116] According to the image processing method described in paragraph 12, edges can be more clearly expressed in the data.

[0117] (Item 13) The image processing method described in Item 12 may further include a step of performing a line connection process on the image that has been subjected to the binarization process in order to generate the board area data.

[0118] According to the image processing method described in item 13, the area corresponding to the substrate in the data can be more reliably identified.

[0119] (Clause 14) An image processing device according to one embodiment may include one or more processors and a storage device that stores a program that, when executed by the one or more processors, causes the one or more processors to implement the image processing method described in any one of clauses 1 to 13.

[0120] According to the image processing device described in item 14, the type of correction can be specified according to the surface condition of the sample.

[0121] (Item 15) A scanning probe microscope according to one aspect may include the image processing device according to item 14.

[0122] According to the scanning probe microscope described in Item 15, the type of correction can be specified according to the surface condition of the sample.

[0123] (Item 16) A program according to one aspect may be executed by one or more processors to cause the one or more processors to perform the image processing method according to any one of items 1 to 13.

[0124] According to the program described in paragraph 16, the type of correction can be specified according to the surface condition of the sample.

[0125] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present disclosure is defined by the claims, not by the description of the above-described embodiments, and is intended to include all modifications within the meaning and scope of the claims. Furthermore, it is intended that each technique in the embodiments can be implemented alone or, if necessary, in combination with other techniques in the embodiments to the extent possible. [Explanation of symbols]

[0126] 1 scanning probe microscope, 110 sample, 111 piezo scanner, 111xy XY scanner, 111z Z scanner, 112 sample stage, 113 cantilever, 114 needle, 115 laser diode, 119 photodetector, 120 displacement detection mechanism, 131 feedback signal generator, 132 computer, 133 scanning signal generator, 134 storage device, 135 display unit.

Claims

1. 1. A method for processing an image of an object generated based on measurement of a sample by a scanning probe microscope, comprising: generating a corrected image by performing a first correction on the target image; The first correction is extracting a plurality of pixels on a straight line along a predetermined direction on a predetermined plane from the target image; correcting the height of the target image based on the luminance of each extracted pixel; generating a histogram of pixel values ​​in the corrected image; and determining whether a second correction different from the first correction is required for the target image using the histogram; The step of determining whether the second correction is necessary includes: determining that the second correction is necessary if the histogram includes multiple peaks; determining that the second correction is unnecessary if the histogram does not include multiple peaks.

2. the sample includes a substrate and a structure on the substrate; The second correction is generating substrate area data that identifies an area in the target image corresponding to the substrate; and performing height correction based on pixels of the area identified by the substrate area data.

3. The image processing method of claim 2, wherein the second correction includes generating a substrate display image representing an image of only the substrate by filling areas of the target image other than the area identified by the substrate area data with filling data.

4. 4. The image processing method according to claim 3, wherein generating the board display image includes generating the filling data using pixel values ​​of an area in the target image specified by the board area data.

5. The image processing method according to claim 3 , wherein generating the board display image includes generating a corrected image by performing tilt correction on the target image using the board display image.

6. identifying, in the corrected image, an area corresponding to the structure in the target image; The image processing method according to claim 5 , further comprising: generating a structure image by extracting an area corresponding to the identified structure from the target image.

7. The image processing method according to claim 6, further comprising a step of generating viewing data by combining the structure image with background data having background pixels in the area specified by the substrate area data.

8. The image processing method according to claim 2 , further comprising the step of generating a structure image by extracting an area specified by the board area data from the target image.

9. 1. A method for processing an image of an object generated based on measurement of a sample by a scanning probe microscope, comprising: generating a corrected image by performing a first correction on the target image; The first correction is extracting a plurality of pixels on a straight line along a predetermined direction on a predetermined plane from the target image; correcting the height of the target image based on the luminance of each extracted pixel; generating a histogram of pixel values ​​in the corrected image; and determining whether a second correction different from the first correction is required for the target image using the histogram; the sample includes a substrate and a structure on the substrate; The second correction is generating substrate area data that identifies an area in the target image corresponding to the substrate; performing height correction based on pixels of an area identified by the substrate area data; extracting, from the target image, pixels that satisfy a condition that they are edges as a result of comparison with adjacent pixels, as edge pixels; generating a dilated image by performing a dilation process on the target image to dilate edges formed by the edge pixels; An image processing method, wherein the step of generating the substrate region data includes generating the substrate region data using the dilated image.

10. The image processing method according to claim 9 , wherein the expansion processing includes maximum value processing.

11. 10. The image processing method according to claim 9, further comprising the step of performing a binarization process on the dilated image or data resulting from the dilated image to generate the substrate region data.

12. The image processing method according to claim 11, further comprising the step of performing a line connection process on the image that has been subjected to the binarization process in order to generate the board area data.

13. one or more processors; 10. An image processing apparatus comprising: a storage device that stores a program that, when executed by the one or more processors, causes the one or more processors to perform the image processing method according to claim 1.

14. A scanning probe microscope comprising the image processing device according to claim 13.

15. A program that, when executed by one or more processors, causes the one or more processors to perform the image processing method according to claim 1 or 9.

Images