Image processing method, image processing device, scanning probe microscope, and program

Abstract

Provided is a technology for identifying the type of a correction according to the state of the surface of a sample, an image of which has been generated on the basis of the measurement using a scanning probe microscope An image processing method according to the present invention comprises a step (SA1) for generating a corrected image by performing a first correction on a target image. The first correction includes extracting, from the target image, a plurality of pixels on a straight line along a prescribed direction on a prescribed plane and correcting the height of the target image on the basis of the luminance of each of the pixels extracted. The image processing method further comprises: a step (SA3) for generating a histogram of the pixel values in the corrected image; and a step (SA4) for using the histogram to determine whether or not a second correction, which is different from the first correction, is necessary on the target image.

Description

Image processing method, image processing device, scanning probe microscope, and program

[0001] The present invention relates to the processing of images obtained by scanning probe microscopes.

[0002] A scanning probe microscope (SPM) observes the topography of a sample surface with high resolution by bringing a sufficiently sharpened probe close enough to the sample to be observed and scanning the sample surface horizontally with the probe while raising and lowering the height of the probe so that the physical quantities acting on the tip of the probe and the surface of the sample remain constant. SPM is a general term for microscopes that observe the topography of a sample surface using the above-mentioned operating principle. Representative SPMs include a scanning tunneling microscope (STM), which detects the current flowing between the probe and the sample as an interaction, and an atomic force microscope (AFM), which detects the atomic force acting between the probe and the sample as an interaction.

[0003] Scanning probe microscopes have high resolution in the surface height direction, and it is difficult to set the sample surface horizontally at that level of resolution. Therefore, height correction is generally performed on the height image acquired by the scanning probe microscope (hereinafter referred to as SPM image) to correct the inclined surface so that it is horizontal. By providing the corrected image instead of the SPM image, the user can more accurately recognize the surface condition of the sample.

[0004] As an example of processing images obtained by such a scanning probe microscope, Japanese Patent No. 6627903 (Patent Document 1) discloses a technique in which at least a portion of the area other than the edge in the image data is extracted as a reference plane area, and the height of the measurement data is corrected based on height information of three points belonging to the reference plane area.

[0005] Patent No. 6627903

[0006] In conventional techniques, uniform image processing is performed on image data, as described in Patent Document 1. However, there may be cases where the appropriate type of image processing differs depending on the surface condition of the sample.

[0007] The present invention has been devised in view of the above-described circumstances, and its purpose is to provide a technique for identifying the type of correction to be applied to an image of a sample generated based on measurements using a scanning probe microscope, in accordance with the surface condition of the sample.

[0008] An image processing method according to one aspect of the present disclosure is a method for processing an object image generated based on measurement of a sample using a scanning probe microscope, comprising the steps of generating a corrected image by performing a first correction on the object image, the first correction including extracting a plurality of pixels from the object image on a straight line along a predetermined direction in a predetermined plane, and correcting the height of the object image based on the brightness of each extracted pixel, and further comprising the steps of generating a histogram of pixel values ​​in the corrected image, and using the histogram to determine whether a second correction different from the first correction is required for the object image.

[0009] An image processing device according to one aspect of the present disclosure includes one or more processors and a storage device storing a program that, when executed by the one or more processors, causes the one or more processors to perform the image processing method described above.

[0010] A scanning probe microscope according to an aspect of the present disclosure includes the image processing device described above. A program according to an aspect of the present disclosure, when executed by one or more processors, causes the one or more processors to perform the image processing method described above.

[0011] According to one aspect of the present disclosure, a corrected image is generated by performing a first correction on a target image, and a histogram of pixel values ​​in the corrected image is used to determine whether a second correction is required for the target image. The histogram of pixel values ​​reflects the surface condition of the sample. Therefore, according to one aspect of the present disclosure, whether the second correction is required is determined based on the surface condition of the sample, and thereby the type of correction based on the surface condition of the sample can be identified.

[0012] 5 is a schematic configuration diagram of a scanning probe microscope according to an embodiment. FIG. 6 is a diagram for explaining the contents of a first correction. FIG. 7 is a diagram showing an example of the shape of a sample observed with a scanning probe microscope. FIG. 8 is a graph showing height along the Y1-Y1 line in FIG. 3. FIG. 9 is a diagram showing another example of the shape of a sample observed with a scanning probe microscope. FIG. 10 is a graph showing height along the Y2-Y2 line in FIG. 11. FIG. 12 is a diagram showing a schematic result of edge extraction for an image obtained as an observation result with a scanning probe microscope. FIG. 13 is a diagram showing a schematic result of a process for expanding extracted edges. FIG. 14 is a diagram showing a schematic result of a binarization process. FIG. 15 is a diagram showing a schematic result of a hole filling process. FIG. 16 is a diagram for explaining a data configuration of a substrate region. FIG. 17 is a diagram showing a schematic part of the process of tilt correction in a scanning probe microscope 1. FIG. 18 is a flowchart of an example of a process performed for image processing in a scanning probe microscope 1. FIG. 19 is a diagram showing an example of a histogram generated in step SA3. FIG. 19 is a diagram showing another example of a histogram generated in step SA3. 10 is a flowchart of a subroutine relating to the implementation of the second correction in step SA5. FIG. 11 is a flowchart of a first modified example of the subroutine in step SA5 in modification (1). FIG. 12 is a flowchart of a second modified example of the subroutine in step SA5.

[0013] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail with reference to the accompanying drawings, in which the same or corresponding parts are designated by the same reference numerals and will not be described repeatedly.

[0014] [Schematic Configuration of Scanning Probe Microscope] Fig. 1 is a schematic configuration diagram of a scanning probe microscope according to one embodiment. An example of a scanning probe microscope is an atomic force microscope. However, the scanning probe microscope may be another type of scanning probe microscope (for example, a scanning tunneling microscope).

[0015] 1, the scanning probe microscope 1 includes a sample stage 112 on which a sample 110 is placed, a piezoelectric scanner 111 that displaces the sample stage, a cantilever 113 having a probe 114 formed at the tip thereof, a displacement detection mechanism 120 that detects the displacement of the cantilever 113, a feedback signal generator 131, a computer 132, a scanning signal generator 133, a storage device 134, and a display device 135. In one implementation example, the computer 132 includes at least one processor, and the storage device 134 stores in a non-volatile manner a program executed by the processor.

[0016] The piezo scanner 111 includes a Z scanner 111z that generates displacement in the Z direction based on a voltage value Vz, and an XY scanner 111xy that generates displacement in the X and Y directions based on voltage values ​​Vx and Vy.

[0017] Displacement detection mechanism 120 includes laser diode 115 and photodetector 119. In scanning probe microscope 1, when the tip of probe 114 is brought close to sample 110 to observe the surface, laser light emitted from laser diode 115 is reflected by the back surface of cantilever 113, and the reflected light is received by photodetector 119. As probe 114 is brought closer to the surface of sample 110, cantilever 113 bends like a leaf spring, and the amount of bending is observed at the light receiving position of photodetector 119.

[0018] The feedback signal generating unit 131 receives a detection signal from the photodetector 119. The feedback signal generating unit 131 calculates the amount of deflection of the cantilever 113 based on the detection signal. The feedback signal generating unit 131 controls the Z-direction position of the sample so that the atomic force between the probe 114 and the surface of the sample 110 is always constant. The feedback signal generating unit 131 calculates a voltage value Vz for displacing the piezo scanner 111 in the Z-axis direction based on the amount of deflection of the cantilever 113, and outputs the voltage value Vz to the Z scanner 111z.

[0019] The scanning signal generating unit 133 calculates voltage values ​​Vx and Vy in the X-axis and Y-axis directions so that the sample 110 moves relative to the probe 114 in the XY plane according to a predetermined scanning pattern, and outputs them to the XY scanner 111xy.

[0020] A signal reflecting the feedback amount in the Z-axis direction (the voltage Vz applied to the scanner and the deviation signal Sd) is also sent to the computer 132 and stored in the storage device 134. The computer 132 calculates the amount of surface displacement of the sample 110 due to the unevenness from the voltage Vz based on correlation information previously stored in the storage device 134, which indicates the relationship between the voltage Vz and the corresponding amount of surface displacement due to the unevenness of the sample 110. The computer 132 reconstructs a three-dimensional image of the sample surface by calculating the amount of displacement at each position in the X-axis and Y-axis directions, and displays this on the screen of the display unit 135. This three-dimensional image data is also stored in the storage device 134. The data includes coordinates indicating the position on the X-Y plane and the sample height at those coordinates. The computer 132 can read out the three-dimensional image data stored in the storage device 134 at any time and display it on the display unit 135.

[0021] The computer 132 can correct the height of the three-dimensional image data as needed and display it on the display unit 135 .

[0022] The computer 132 can perform at least two types of height correction. In this specification, the two types of height correction may be referred to as a "first correction" and a "second correction."

[0023] [First Correction] In one implementation example, as a first correction, the computer 132 extracts the brightness of a plurality of pixels on a straight line along a predetermined direction in an image of the surface of the sample 110, which should be horizontal, as described in Japanese Patent No. 6631647, and then performs height correction based on the brightness of the extracted plurality of pixels.

[0024] FIG. 2 is a diagram for explaining the contents of the first correction. FIG. 2 shows an image IM90 as an example of a surface image of a sample 110. A specific example of the sample 110 corresponding to image IM90 is a pattern produced by a semiconductor process. FIG. 2 also shows the X-axis and Y-axis similar to those in FIG. 1. In FIG. 2, line L90 is a straight line along the Y-axis, and represents the line from which the brightness of the image is extracted in the first correction.

[0025] [Second Correction] As an example of the second correction, the computer 132 generates an image that identifies the region of the sample 110 that corresponds to the substrate, and performs height correction on the image. The second correction will be described below.

[0026] <Ideal Observation Example> Figure 3 is a diagram showing an example of the shape of a sample observed with a scanning probe microscope. In sample 110 corresponding to image IM01 shown in Figure 3, 16 structures are arranged on a substrate. The 16 structures are arranged in four rows in the X-axis direction and four rows in the Y-axis direction, i.e., a 4x4 configuration. An example of a substrate is a mica plate. An example of a structure is nanoparticles or nanofibers of a biological sample. Note that these are merely examples, and the samples (substrates, structures) that the scanning probe microscope 1 targets are not limited to these.

[0027] The shade of the pixels in image IM01 represents the height along the Z axis. Darker pixels represent higher positions on the sample along the Z axis. Lighter pixels represent lower positions on the sample along the Z axis.

[0028] Fig. 4 is a graph showing the height along the Y1-Y1 line in Fig. 3. In the graph of Fig. 4, line L30 represents the height generated based on ideal observation by the scanning probe microscope 1. More specifically, line L30 represents one line of image data observed in an ideal case where there is no tilt of the substrate in the sample.

[0029] In image IM01 in Figure 3, the planar boundary (the boundary between the substrate and the structure) is shown by a line, but in Figure 4, the planar boundary is tapered to resemble the actual sample shape, in order to make the correction process easier to understand. The horizontal axis indicates the position in the X direction, and the Z axis indicates the height at each X position. When the probe is scanned along the X direction, the X axis also corresponds to the time axis.

[0030] <Example of Observation with a Tilt Sample> Figure 5 shows another example of the shape of a sample observed with a scanning probe microscope. Image IM10 shown in Figure 5 corresponds to the same sample as sample 110 corresponding to image IM01 shown in Figure 3. Image IM10 shows shading variations on the same plane. In particular, image IM10 shows large shading variations in the area other than the area corresponding to the 16 structures (the area corresponding to the substrate). One of the reasons for this shading variation is that sample 110 is placed at an angle on sample stage 112.

[0031] Fig. 6 is a graph showing the height along the Y2-Y2 line in Fig. 5. In Fig. 6, line L10 represents the height of the sample 110 estimated from the Y2-Y2 line in image IM10. Line L10 indicates that the surface of the sample 110 is inclined such that the surface of the sample 110 becomes higher as the X coordinate increases.

[0032] In the scanning probe microscope 1, corrections are made to an image generated based on observation results such as image IM10, and a corrected image is provided. This allows the surface state of the sample 110 to be recognized more accurately from the corrected image. The details of the corrections made to the image are described below.

[0033] <Edge Extraction> Fig. 7 is a diagram showing the result of edge extraction for an image obtained as an observation result with a scanning probe microscope. In one implementation example, image IM11 shown in Fig. 7 is obtained by performing a known process for edge extraction (such as Process / Find Edges in the open source ImageJ (https: / / imagej.nih.gov / ij / )) on image IM10.

[0034] The process of extracting edges to obtain image IM11 typically involves differential processing of the height image. Alternatively, the process of extracting edges may use a contour extraction method commonly used in photo processing techniques. For example, the part where the absolute value of the difference between adjacent pixel data exceeds a certain threshold may be extracted.

[0035] In image IM11, pixels that form the boundaries between each of the 16 structures and the substrate are detected as edges and are shown as black pixels. Each of the pixels detected as an edge is an example of an edge pixel.

[0036] <Edge Dilation> Fig. 8 is a diagram showing the result of a process of dilating extracted edges. In one implementation example, the image IM12 shown in Fig. 8 is obtained by performing maximum value processing on the image IM11 of Fig. 7.

[0037] In image IM11 of Fig. 7, edge pixels are shown as black pixels (pixels having relatively high pixel values). Therefore, by performing maximum value processing on image IM11, edges formed by edge pixels are dilated in image IM12 shown in Fig. 8. Image IM12 is an example of a "dilated image."

[0038] The process for edge expansion is not limited to maximum value processing. The type of process for edge expansion can be changed as appropriate depending on the representation of edge pixels, etc. For example, if edge pixels are represented as white pixels (pixels having relatively low pixel values), minimum value processing may be adopted as the process for edge expansion.

[0039] <Binarization> Fig. 9 is a diagram schematically illustrating the result of binarization processing. In one implementation example, image IM13 shown in Fig. 9 is obtained by performing a known binarization process (such as Process / Binary / Make Binary in the open source ImageJ (https: / / imagej.nih.gov / ij / )) on image IM12 of Fig. 8. The threshold for binarization may be adjusted as appropriate by the user and / or the computer 132.

[0040] In the image IM13 shown in FIG. 9, portions of the sample 110 corresponding to the ends of the 16 structures are represented by black pixels, and the other portions are represented by white pixels.

[0041] <Line Joining> The computer 132 may perform line joining processing on the binarized image IM13. This allows the edge pixels detected in the image IM10 to more accurately represent the outer edges of the structures. The line joining processing can be achieved, for example, by a known technique (such as Process / Binary / Close in the open source ImageJ (https: / / imagej.nih.gov / ij / )).

[0042] <Hole Filling Process> The computer 132 may perform hole filling process on the binarized image IM13 (or an image obtained by performing line connecting process on the image IM13).

[0043] 10 is a diagram illustrating the results of the hole filling process. The hole filling process can be realized by, for example, a known technique (such as Process / Binary / Fill Holes in the open source ImageJ (https: / / imagej.nih.gov / ij / )).

[0044] In the image IM14 shown in FIG. 10, the areas of the sample 110 corresponding to each of the 16 structures are represented by being filled with black pixels, and the other areas are represented by being filled with white pixels.

[0045] <Substrate Region> The computer 132 can extract the region of white pixels from the image IM14 shown in Fig. 10 as a region of the sample corresponding to the substrate (a region on which no structure is placed). Data specifying the region corresponding to the substrate (substrate region) is also referred to as "substrate region data."

[0046] Fig. 11 is a diagram for schematically explaining the data configuration of a board area, including lines L11, L12, L13, L14, and L15 that indicate the height of the board area along the Y3-Y3 line in Fig. 10.

[0047] 11 also shows, for comparison, the line L10 shown in FIG. 6. The ends of each of the lines L11 to L15 are located a length D1 inward from the portion of the line L10 that corresponds to the substrate. This corresponds to the fact that, after the edges detected for the structures are dilated, the substrate region is identified as a region other than the region identified by the dilated edges. In other words, in the scanning probe microscope 1, by dilating the edges of the structures, the region of the structures is identified broadly, thereby reliably avoiding the structures from being included in the substrate region.

[0048] <Tilt Correction> Figure 12 is a diagram showing a schematic representation of part of the process of tilt correction in the scanning probe microscope 1. In Figure 12, the dashed lines show data that has been filled in (interpolated) to combine lines L11 to L15 shown in Figure 11 into a single line. The line formed by data interpolation is shown as line A10.

[0049] In one implementation example, the computer 132 generates a straight line (line A10) estimated from the points that make up lines L11 to L15 during data interpolation. Generating a straight line is an example of generating fill data. The computer 132 then performs data interpolation by filling the portions other than the five lines L11 to L15 with data from the corresponding portions of the generated straight line.

[0050] The computer 132 performs data interpolation (generation of filling data and filling of the data) similar to that shown in FIG. 12 on the entire data of the substrate area (not only the Y3-Y3 line in FIG. 10 but also other areas). This generates data corresponding to the entire area of ​​the sample, i.e., data representing the surface of the substrate corresponding to the entire area of ​​the sample. The image represented by the data generated here virtually represents the substrate in a state where the structure has been removed from the sample, and is an example of a "substrate display image." The tilt of the surface generated here is expected to represent the tilt of the sample 110 on the sample stage 112.

[0051] Fig. 13 is a diagram schematically illustrating a part of the process of tilt correction in the scanning probe microscope 1. The computer 132 calculates the tilt of the data surface generated in the process described with reference to Fig. 12 relative to an ideal surface, and corrects the image IM10 so as to correct the calculated tilt.

[0052] In FIG. 13 , the surface orientation of the data generated in the process described with reference to FIG. 12 is shown as line A10. The ideal surface orientation is shown as line A20. The computer 132 calculates the tilt of line A10 relative to line A20. This tilt corresponds to the tilt of the sample 110. The computer 132 then corrects image IM10 ( FIG. 5 ) to offset the calculated tilt. This correction generates line L20 from line L10, as shown in FIG. 13 . Similarly, the entire area of ​​image IM10 is corrected to generate a corrected image. This correction is expected to bring image IM10 closer to image IM01 ( FIG. 3 ). In this specification, the image that has undergone the second correction is also referred to as a “corrected image.”

[0053] [Processing Flow] Figure 14 is a flowchart of an example of processing performed for image processing in the scanning probe microscope 1. In one implementation example, the processing in Figure 14 is performed by the processor of the computer 132 executing a given program. In this sense, the scanning probe microscope 1 is an example of an image processing device.

[0054] In step SA1, the scanning probe microscope 1 acquires image data that is the observation result. An example of the image acquired here corresponds to image IM10 in Fig. 5. In this specification, the image that is the observation result is an example of a "target image."

[0055] In step SA2, the scanning probe microscope 1 performs the above-mentioned "first correction" on the target image.

[0056] In step SA3, the scanning probe microscope 1 generates a histogram of pixel values ​​for the target image that has been subjected to the first correction performed in step SA2.

[0057] 15 is a diagram showing an example of a histogram generated in step SA3. As shown in FIG. 15, the horizontal axis of the histogram represents pixel values, and the vertical axis of the histogram represents the number of pixels. In the example of FIG. 15, the number of each pixel value in the target image is represented by line L91. Line L91 has a peak at pixel value V11.

[0058] Returning to FIG. 14, in step SA4, the scanning probe microscope 1 determines whether or not the histogram generated in step SA3 has a plurality of peaks.

[0059] The number and positions of peaks in the histogram are identified, for example, by performing a known peak detection process on the histogram. Note that the scanning probe microscope 1 may perform a smoothing process on the graph as shown by line L91 before the peak detection process.

[0060] The number and positions of peaks in the histogram may be specified by the user. That is, the scanning probe microscope 1 may display the histogram generated in step SA3 on the display unit 135. The user may look at the histogram displayed on the display unit 135, specify the number (and positions) of peaks, and input this to the scanning probe microscope 1. The scanning probe microscope 1 may be equipped with an input device such as a keyboard, and the user may input the number (and positions) of peaks using this input device. The scanning probe microscope 1 may use the input number of peaks to make the determination of step SA4.

[0061] If the scanning probe microscope 1 determines that the number of peaks in the histogram is multiple (YES in step SA4), it proceeds to step SA6, but if it determines that this is not the case, that is, if it determines that there is a single peak (NO in step SA4), it proceeds to step SA6. Here, an example of a possible correlation between the number of peaks in the histogram and the surface morphology of the sample 110 will be described.

[0062] Fig. 16 shows another example of the histogram generated in step SA3. In the example of Fig. 16, the number of each pixel value in the target image is represented by line L92. Line L92 has peaks at pixel values ​​V21 and V22. In other words, line L92 has two peaks.

[0063] Whether or not there are multiple peaks in the histogram reflects whether or not the surface of the sample 110 has a concave-convex structure. More specifically, if the surface of the sample 110 has a concave-convex structure, the histogram of the image of the surface is expected to have multiple peaks. For example, if structures are arranged on a substrate in the sample as described with reference to FIG. 3 etc., the multiple peaks include a peak corresponding to the height of the substrate and a peak corresponding to the height of the structure. On the other hand, if the surface of the sample 110 does not have a concave-convex structure (in a macroscopic sense), as described with reference to FIG. 2, the histogram of the image of the surface is expected to have a single peak.

[0064] 14, in step SA5, the scanning probe microscope 1 performs a second correction on the target image. The execution of the second correction will be described later with reference to FIG.

[0065] In step SA6, the scanning probe microscope 1 displays the results of the processing on the display unit 135. The displayed results may include an image generated by the first correction or the second correction as a corrected image. The image displayed as a corrected image is an image that has been subjected to the second correction if the second correction has been performed in step SA6, and is an image that has been subjected to the first correction if the second correction has not been performed in step SA6. The scanning probe microscope 1 then terminates the processing of FIG. 14 .

[0066] FIG. 17 is a flowchart of a subroutine for performing the second correction in step SA5.

[0067] In step S10, the scanning probe microscope 1 acquires a target image. If the target image acquired in step SA1 is stored in the storage device 134, the target image may be read out from the storage device 134 in step S10.

[0068] In step S12, the scanning probe microscope 1 extracts edges from the image acquired in step S10. The edge extraction is realized by, for example, the method described with reference to FIG.

[0069] In step S14, the scanning probe microscope 1 dilates the edges extracted in step S12. The dilation of the edges is realized by, for example, the method described with reference to FIG.

[0070] In step S16, the scanning probe microscope 1 binarizes the image whose edges have been dilated in step S14. The binarization is realized by, for example, the method described with reference to FIG.

[0071] In step S18, the scanning probe microscope 1 performs line connection processing on the image binarized in step S16. The line connection processing is realized by, for example, the method described with reference to FIG.

[0072] In step S20, the scanning probe microscope 1 performs a hole filling process on the image that has been subjected to the line connection process in step S18. The hole filling process is realized by, for example, the method described with reference to FIG.

[0073] In step S22, the scanning probe microscope 1 generates substrate area data using the image that has been subjected to the hole filling process in step S20. The substrate area data is data that identifies the substrate area.

[0074] In step S24, the scanning probe microscope 1 uses the second data to correct the image data acquired in step S10 (correcting the height). The image data correction is realized, for example, by the method described with reference to Figures 12 and 13. By the control of step S24, an image is generated in which the "second correction" has been applied to the "target image." Then, the scanning probe microscope 1 returns control to Figure 14.

[0075] 12 and 13, data specifying the substrate region was filled into the region corresponding to the structure, as shown by the dashed line (line A10) in Fig. 12. Note that such data filling may be omitted when correcting the image data. The scanning probe microscope 1 may calculate the tilt of the sample 110 (with respect to the ideal plane) from only the substrate region.

[0076] In the present embodiment described above, a first correction is applied to a target image, and a histogram of the image subjected to the first correction is generated. If the histogram has multiple peaks, the image subjected to the second correction is displayed as the "corrected image" in step SA6. On the other hand, if the histogram does not have multiple peaks, the image subjected to the first correction is displayed as the "corrected image" in step SA6.

[0077] In the second correction, a dilation process is performed on the edges of the structure to generate a "dilated image," and the "dilated image" is then used to generate "substrate region data" that identifies the region of the sample corresponding to the substrate (the region on the substrate where no structure exists). This reliably prevents the region identified by the "substrate region data" from including an image of the structure. Furthermore, regions other than the region identified by the "substrate region data" are reliably identified as regions that include the entire structure.

[0078] When the tilt in the target image caused by the tilt of the sample is corrected using the area specified by the "substrate area data," more points (or lines or areas) can be used for the correction. This allows the tilt to be corrected more accurately. Therefore, the surface condition of the sample can be accurately provided to the user.

[0079] [Variation (1)] In variation (1), the scanning probe microscope 1 corrects the target image using the substrate area data in the second correction, and again identifies the area corresponding to the structure from the corrected image data.

[0080] Fig. 18 is a flowchart of a first modified example of the subroutine of step SA5 in modified example (1). The processing in Fig. 18 includes the control of steps S10 to S24, similar to the processing in Fig. 17. In the processing in Fig. 18, the scanning probe microscope 1 advances control to step S30 after step S24.

[0081] In step S30, the scanning probe microscope 1 identifies an area corresponding to the structure in the corrected image generated in step S24. The corrected image generated in step S24 is an example of a "corrected image."

[0082] The identification of the region corresponding to the structure in step S30 may include, for example, control similar to steps S12 to S22. That is, the scanning probe microscope 1 extracts edges from the corrected image, expands the extracted edges, binarizes the edge-expanded image, performs line connection processing on the binarized image, performs hole filling processing on the image that has been subjected to line connection processing, and generates substrate region data for the image that has been subjected to hole filling processing. The scanning probe microscope 1 then identifies regions other than those identified by the substrate region data (i.e., regions other than the substrate region) as regions corresponding to the structure.

[0083] In step S32, the scanning probe microscope 1 generates a "structure image" by extracting an area corresponding to the structure identified in step S30 from the image data corrected in step S24. The structure image includes an image corresponding to the structure. The scanning probe microscope 1 then returns control to Fig. 14. In modification example (1), the result displayed in step SA6 may include the structure image generated in step S30, i.e., an image corresponding to the structure.

[0084] In the above-described variant (1), an area other than the area specified by the substrate area data (i.e., an area other than the substrate area) is extracted from the target image and displayed as a structure image. The substrate area data is generated using a dilated image, and the dilated image is generated through a dilation process. This ensures that the area other than the area specified by the substrate area data includes the entire structure. Therefore, the user can be provided with an accurate view of the state of the structure, and thereby the surface state of the sample can be provided with an accurate view.

[0085] [Variation (2)] In Variation (2), the scanning probe microscope 1 generates a pseudo image of the entire sample by combining the structure image of Variation (1) with background data that fills in the substrate area. In Variation (2), the "pseudo image of the entire sample" generated in this way is an image that has been subjected to the second correction.

[0086] Fig. 19 is a flowchart of a second modified example of the subroutine of step SA5. In the process of Fig. 19, the scanning probe microscope 1 advances control to step S40 after step S22.

[0087] In step S40, the scanning probe microscope 1 generates a structure image using the target image acquired in step S10 and the substrate area data generated in step S22. The structure image generated in step S40 is generated by extracting, from the target image acquired in step S10, an area other than the area of ​​the substrate specified by the substrate area data generated in step S22.

[0088] In step S42, the scanning probe microscope 1 generates viewing data by filling in areas other than those identified by the structure image generated in step S40 with background data. The viewing data represents a pseudo image of the entire sample. The pseudo image of the entire sample combines the structure in the target image acquired in step S10 with the background image.

[0089] Thereafter, the scanning probe microscope 1 returns control to Fig. 14. In modification (2), the results displayed in step SA6 may include an image of the viewing data generated in step S42, i.e., an image of the entire pseudo sample.

[0090] In the image of the viewing data of the modification (2) (a pseudo image of the entire sample), the area corresponding to the substrate region has appropriate brightness, which eliminates the need for contrast adjustment when displaying the image of the viewing data. Therefore, the user can visually confirm the processing results without the need for tedious tasks such as adjusting the contrast. Furthermore, in conventional techniques, contrast adjustment can sometimes result in fluctuations in pixel density in the background. In the modification (2), contrast adjustment can be omitted, thereby avoiding the occurrence of such fluctuations in pixel density in the image displayed to the user. The viewing data may also be generated by combining the structure image (S30) generated in the modification (1) with the background data (S42).

[0091] The pseudo-image of the entire sample includes an image of a region other than the region specified by the substrate region data from the target image. The substrate region data is generated using a dilated image, and the dilated image is generated through a dilation process. This ensures that the region other than the region specified by the substrate region data includes the entire structure. Therefore, by providing an image of the pseudo-image of the entire sample, the user can be provided with an accurate representation of the state of the structure, and thereby the surface state of the sample can be provided with an accurate representation.

[0092] Aspects It will be understood by those skilled in the art that the exemplary embodiments described above are examples of the following aspects.

[0093] (Item 1) An image processing method according to one aspect is a method for processing an object image generated based on measurement of a sample using a scanning probe microscope, comprising the steps of: generating a corrected image by performing a first correction on the object image, wherein the first correction includes extracting a plurality of pixels from the object image on a straight line along a predetermined direction in a predetermined plane, and correcting a height of the object image based on the luminance of each extracted pixel; and may also comprise the steps of generating a histogram of pixel values ​​in the corrected image; and using the histogram to determine whether a second correction different from the first correction is necessary for the object image.

[0094] According to the image processing method described in paragraph 1, the type of correction can be specified according to the surface condition of the sample.

[0095] (Clause 2) In the image processing method described in paragraph 1, the step of determining whether the second correction is necessary may include determining that the second correction is necessary if the histogram includes multiple peaks, and determining that the second correction is not necessary if the histogram does not include multiple peaks.

[0096] According to the image processing method described in the second paragraph, the type of correction can be identified in accordance with the surface condition of the sample.

[0097] (Clause 3) In the image processing method described in paragraph 1 or 2, the sample may include a substrate and a structure on the substrate, and the second correction may include a step of generating substrate area data that identifies an area in the target image corresponding to the substrate, and a step of performing height correction based on pixels in the area identified by the substrate area data.

[0098] According to the image processing method described in the third aspect, the second correction can provide an accurate surface state of the sample.

[0099] (4) In the image processing method described in 3, the second correction further includes a step of extracting pixels from the target image that satisfy the condition that they are edges as a result of comparison with adjacent pixels as edge pixels, and a step of generating an expanded image by performing an expansion process on the target image to expand the edges formed by the edge pixels, and the step of generating the substrate area data may include generating the substrate area data using the expanded image.

[0100] According to the image processing method described in item 4, the second correction can provide a more accurate surface state of the sample.

[0101] (Item 5) In the image processing method described in Item 4, the expansion processing may include maximum value processing.

[0102] According to the image processing method described in paragraph 5, the structure is reliably expanded by edge pixels in the expansion process.

[0103] (Clause 6) In the image processing method described in any one of clauses 3 to 5, the second correction may include generating a substrate display image representing an image of only the substrate by filling areas of the target image other than the area specified by the substrate area data with filling data.

[0104] According to the image processing method described in paragraph 6, the height of the original image data can be corrected more accurately.

[0105] (Clause 7) In the image processing method described in Clause 6, generating the substrate display image may include generating the filling data using pixel values ​​of an area in the target image identified by the substrate area data.

[0106] According to the image processing method described in Section 7, data for filling can be generated that is more compatible with the original image data.

[0107] (Clause 8) In the image processing method described in clause 6 or clause 7, generating the substrate display image may include generating a corrected image by performing tilt correction on the target image using the substrate display image.

[0108] According to the image processing method described in paragraph 8, an image whose tilt has been corrected more reliably can be generated as the corrected image.

[0109] (Clause 9) The image processing method described in Clause 8 may further include a step of identifying an area in the corrected image that corresponds to the structure in the target image, and a step of generating a structure image from the target image by extracting the area that corresponds to the identified structure.

[0110] According to the image processing method described in paragraph 9, data that includes the entire structure more reliably can be generated as a structure image.

[0111] (Clause 10) The image processing method described in any one of clauses 3 to 9 may further include a step of generating a structure image by extracting an area identified by the board area data from the target image.

[0112] According to the image processing method described in paragraph 10, an image including the entire structure can be generated more reliably as a structure image.

[0113] (Clause 11) The image processing method described in clause 9 or clause 10 may further include a step of generating viewing data by combining the structure data with background data having background pixels in an area specified by the substrate area data.

[0114] According to the image processing method described in paragraph 11, contrast adjustment is not required when displaying the viewing data.

[0115] (Clause 12) The image processing method described in clause 4 or clause 5 may further include a step of performing a binarization process on the dilated image or data resulting from the dilated image in order to generate the substrate area data.

[0116] According to the image processing method described in paragraph 12, edges can be more clearly expressed in the data.

[0117] (Item 13) The image processing method described in Item 12 may further include a step of performing a line connection process on the image that has been subjected to the binarization process in order to generate the board area data.

[0118] According to the image processing method described in item 13, the area corresponding to the substrate in the data can be more reliably identified.

[0119] (Clause 14) An image processing device according to one aspect may include one or more processors and a storage device that stores a program that, when executed by the one or more processors, causes the one or more processors to implement the image processing method described in any one of clauses 1 to 13.

[0120] According to the image processing device described in item 14, the type of correction can be specified according to the surface condition of the sample.

[0121] (15th Item) A scanning probe microscope according to one aspect may include the image processing device according to the 14th item.

[0122] According to the scanning probe microscope described in Item 15, the type of correction can be specified according to the surface condition of the sample.

[0123] (16th paragraph) A program according to one aspect may be executed by one or more processors to cause the one or more processors to perform the image processing method according to any one of paragraphs 1 to 13.

[0124] According to the program described in paragraph 16, the type of correction can be specified according to the surface condition of the sample.

[0125] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present disclosure is defined by the claims, not by the description of the above-described embodiments, and is intended to include all modifications within the meaning and scope of the claims. Furthermore, it is intended that each technique in the embodiments can be implemented alone or, if necessary, in combination with other techniques in the embodiments to the extent possible.

[0126] 1 scanning probe microscope, 110 sample, 111 piezo scanner, 111xy XY scanner, 111z Z scanner, 112 sample stage, 113 cantilever, 114 needle, 115 laser diode, 119 photodetector, 120 displacement detection mechanism, 131 feedback signal generator, 132 computer, 133 scanning signal generator, 134 storage device, 135 display unit.

Claims

1. A method for processing an image of an object generated based on measurement of a sample by a scanning probe microscope, comprising: generating a corrected image by performing a first correction on the target image; The first correction is Extracting a plurality of pixels on a straight line along a predetermined direction in a predetermined plane from the target image; and correcting the height of the target image based on the luminance of each extracted pixel; generating a histogram of pixel values ​​in the corrected image; and determining whether or not the target image requires a second correction different from the first correction, using the histogram; The step of determining whether the second correction is necessary includes: determining that the second correction is necessary if the histogram includes multiple peaks; determining that the second correction is unnecessary if the histogram does not include multiple peaks.

2. The sample includes a substrate and a structure on the substrate; The second correction is generating substrate area data that identifies an area in the target image that corresponds to the substrate; and performing a height correction based on pixels of an area identified by said substrate area data.

3. The image processing method of claim 2, wherein the second correction includes generating a substrate display image representing an image of only the substrate by filling areas of the target image other than an area identified by the substrate area data with filling data.

4. 4. The image processing method according to claim 3, wherein generating the board display image includes generating the filling data using pixel values ​​of an area in the target image specified by the board area data.

5. The image processing method according to claim 3 , wherein generating the board display image includes generating a corrected image by performing tilt correction on the target image using the board display image.

6. identifying, in the corrected image, an area corresponding to the structure in the target image; The image processing method according to claim 5 , further comprising: generating a structure image by extracting an area corresponding to the identified structure from the target image.

7. The image processing method according to claim 6 , further comprising the step of generating viewing data by combining the structure image with background data having background pixels in an area specified by the board area data.

8. The image processing method according to claim 2 , further comprising the step of generating a structure image by extracting a region specified by the board region data from the target image.

9. A method for processing an image of an object generated based on measurement of a sample by a scanning probe microscope, comprising: generating a corrected image by performing a first correction on the target image; The first correction is Extracting a plurality of pixels on a straight line along a predetermined direction in a predetermined plane from the target image; and correcting the height of the target image based on the luminance of each extracted pixel; generating a histogram of pixel values ​​in the corrected image; and determining whether the target image requires a second correction different from the first correction using the histogram, wherein the sample includes a substrate and a structure on the substrate, The second correction is generating substrate area data that identifies an area in the target image that corresponds to the substrate; performing a height correction based on pixels of an area identified by said substrate area data; extracting, from the target image, pixels that satisfy a condition that they are edges as a result of comparison with adjacent pixels as edge pixels; generating a dilated image by performing a dilation process on the target image to dilate edges formed by the edge pixels; A method of image processing, wherein the step of generating the substrate region data includes generating the substrate region data using the dilated image.

10. The image processing method according to claim 9 , wherein the expansion process includes a maximum value process.

11. 10. The image processing method of claim 9, further comprising the step of performing a binarization process on the dilated image or data resulting from the dilated image to generate the board region data.

12. The image processing method according to claim 11 , further comprising the step of performing a line connection process on the image that has been subjected to the binarization process in order to generate the board area data.

13. one or more processors; 10. An image processing apparatus comprising: a storage device that stores a program that, when executed by the one or more processors, causes the one or more processors to perform the image processing method according to claim 1.

14. A scanning probe microscope comprising the image processing device according to claim 13.

15. A program that, when executed by one or more processors, causes the one or more processors to perform the image processing method according to claim 1 or 9.