Image processing method, device and computer readable storage medium

Abstract

Embodiments of the present disclosure relate to image processing methods, devices, and computer-readable storage media. The image processing method comprises: determining the contour lines and corner points of a figure in an image, wherein the figure has a cross-section of stacked parallel layers; determining characteristic corner points among the corner points based on the contour lines, wherein the characteristic corner points are related to the boundary lines of the parallel layers in the cross-section; determining the angle for rotating the image based on the characteristic corner points; and rotating the image within the plane of the image based on the angle so that the boundary line extends in a direction parallel to or perpendicular to the horizontal line within the plane where the image is located. The technical solution of the present disclosure can effectively solve the inconvenience caused by the manual leveling method and improve efficiency and accuracy.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to integrated circuits, and more particularly, to image processing methods, apparatus, and computer-readable storage media. Background Art

[0002] Transmission Electron Microscope (TEM) is an electronic optical instrument with high resolution and high magnification characteristics. TEM can be used as a failure analysis tool in semiconductor manufacturing. TEM uses a very short wavelength electron beam as an electron light source, and uses a high-speed and concentrated electron beam emitted by an electron gun to irradiate the sample, collect transmitted electrons and perform multi-stage magnification imaging through an electromagnetic lens. Before TEM shooting, samples need to be made. The sample can be a slice of a finished silicon wafer. In order to find a better shooting position during the shooting process, the platform on which the sample is placed is usually constantly moved, which may cause the sample to tilt. In addition, if the sample preparation technique is not good, the sample may also tilt, resulting in a tilted picture, which increases the difficulty of quantitative measurement and may cause large deviations.

[0003] Usually, the image needs to be leveled. The traditional solution is for the operator to level it during the process. The disadvantage is that it is time-consuming and the accuracy of leveling is greatly affected by subjective factors. Summary of the invention

[0004] According to an exemplary embodiment of the present disclosure, an image processing scheme is provided to at least partially overcome the above or other potential drawbacks.

[0005] According to one aspect of the present disclosure, an image processing method is provided. The method includes: determining contour lines and corner points of a figure in an image, wherein the figure has a cross section of stacked parallel layers; determining characteristic corner points among the corner points based on the contour lines, wherein the characteristic corner points are related to boundary lines of the parallel layers in the cross section; determining an angle by which the image needs to be rotated based on the characteristic corner points; and rotating the image within the plane of the image based on the angle so that the boundary line extends in a direction parallel to or perpendicular to a horizontal line within the plane where the image is located.

[0006] In a second aspect of the present disclosure, an electronic device is provided. The electronic device includes a processor; and a memory coupled to the processor, the memory having instructions stored therein, and when the instructions are executed by the processor, the device performs an action, the action including: determining contour lines and corner points of a figure in an image, wherein the figure has a cross section of stacked parallel layers; determining characteristic corner points among the corner points based on the contour lines, wherein the characteristic corner points are related to boundary lines of the parallel layers in the cross section; determining an angle to be rotated for the image based on the characteristic corner points; and rotating the image in a plane of the image based on the angle so that the boundary line extends in a direction parallel to or perpendicular to a horizontal line in the plane where the image is located.

[0007] In some embodiments, determining a contour line of a graphic in an image includes: determining edge points in the image; determining continuous edge points from the edge points; and determining the continuous edge points as the contour line of the graphic.

[0008] In some embodiments, determining the corner points of a graphic in an image includes: traversing the image to determine the grayscale value of each pixel point respectively; determining the difference between the first grayscale value of the central pixel point and the second grayscale value of the non-central pixel point in each area centered on each pixel point respectively; and determining each corner point based on each difference.

[0009] In some embodiments, determining each corner point based on each difference value includes: in response to the absolute value of the difference between the central pixel point in the region and the corresponding non-central pixel point in the region being less than a first threshold, determining the corresponding non-central pixel point in the region as a homogeneous point; accumulating the number of homogeneous points in each region to obtain respective accumulated values; and determining the corner point in each region based on comparison of the accumulated value with a second threshold value.

[0010] In some embodiments, determining corner points in each region based on comparison of the accumulated value with the second threshold includes: in response to the corresponding accumulated value of the region being less than the second threshold, determining each response value of the region, the response value being equal to the difference between the second threshold and the corresponding accumulated value of the region; and determining the pixel point corresponding to the maximum response value of each region, and determining the pixel point corresponding to the maximum response value as the corner point in the corresponding region.

[0011] In some embodiments, determining a corner point of a figure in an image includes: determining a contour line of the figure; and determining a point in the contour line with the maximum local curvature as a corner point.

[0012] In some embodiments, determining feature corner points among corner points based on contour lines includes determining corner points that meet the following conditions as feature corner points: there are adjacent corner points; the adjacent corner points are connected by continuous contour lines; and the adjacent corner points are connected by contour lines with an inclination angle less than a predetermined threshold.

[0013] In some embodiments, determining the angle at which an image needs to be rotated based on feature corner points includes: determining a fitting curve for each point on each contour line connecting each adjacent feature corner point; determining an angle between the fitting curve and a horizontal line in a plane where the image is located or a vertical line perpendicular to the horizontal line; and determining the angle as an angle.

[0014] In some embodiments, determining the fitting curves for each point on each contour line connected to each adjacent feature corner point includes: taking the contour line within a specified reference sub-region including the feature corner point as a reference, respectively aggregating the points on the contour lines within each sub-region including the feature corner point into the reference sub-region; and fitting the points on the aggregated contour line to generate a fitting curve.

[0015] In some embodiments, respectively aggregating points on the contour line within the sub-region of each feature corner point to the reference sub-region includes: determining a first point corresponding to the average value of the horizontal coordinate and the average value of the vertical coordinate of the points on the contour line within the reference sub-region; determining a second point corresponding to the average value of the horizontal coordinate and the average value of the vertical coordinate of the contour line within other sub-regions including the feature corner point; and moving each point on the contour line within the other sub-regions parallel to the reference sub-region along the direction from the second point to the first point so that the second point coincides with the first point.

[0016] In some embodiments, determining the slope of the contour line based on the corner points includes: determining the perpendicular bisector of the contour line connecting two adjacent feature corner points based on two adjacent feature corner points; and determining the angle based on the slope of the perpendicular bisector relative to a horizontal line or a vertical line perpendicular to the horizontal line.

[0017] In a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0018] It will be understood from the following description that the technical solution of the present disclosure can effectively solve the inconvenience caused by the manual leveling method and improve the efficiency and accuracy.

[0019] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the disclosure, nor is it intended to limit the scope of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0021] Figure 2 A flowchart of an image processing method according to some embodiments of the present disclosure is shown;

[0022] Figure 3 A schematic diagram showing structural features contained in a picture to be processed according to an embodiment of the present disclosure;

[0023] Figure 4 A schematic diagram showing a picture to be processed according to some embodiments of the present disclosure;

[0024] Figure 5 A schematic diagram showing contour lines in a picture of some embodiments of the present disclosure;

[0025] Figure 6 A schematic diagram showing contour lines and corner points in a picture according to some embodiments of the present disclosure;

[0026] Figure 7 A schematic diagram showing selected feature corner points according to some embodiments of the present disclosure;

[0027] Figure 8 A schematic diagram showing a moving process of points on a contour line according to some embodiments of the present disclosure is shown;

[0028] Fig. 9 A schematic diagram showing a fitting curve according to some embodiments of the present disclosure;

[0029] Fig.10 A schematic diagram showing a graph showing a perpendicular midline according to some embodiments of the present disclosure is shown;

[0030] Fig.11 A schematic diagram showing a leveled picture according to some embodiments of the present disclosure;

[0031] Fig.12 A block diagram of a computing device capable of implementing various embodiments of the present disclosure is shown.

[0032] In the various drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0033] The principles of the present disclosure will be described below with reference to the various exemplary embodiments shown in the accompanying drawings. It should be understood that the description of these embodiments is only to enable those skilled in the art to better understand and further implement the present disclosure, and is not intended to limit the scope of the present disclosure in any way. It should be noted that similar or identical reference numerals may be used in the figures where feasible, and similar or identical reference numerals may represent similar or identical functions. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods described herein may be adopted without departing from the principles of the present invention described herein.

[0034] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects.

[0035] As mentioned earlier, before TEM photography, the sample may be tilted for various reasons. TEM images need to capture the depth and width of the pattern. If the image is not flat, it will be difficult to measure and may cause large deviations. Specifically, if the image is tilted, the corresponding figure is tilted. When measuring the depth and width, it is necessary to obtain the results when the figure is horizontal. In other words, the measurement will take the depth and width of the sample when it is horizontal as the final result. If the image is tilted, the same tilt needs to be maintained when measuring the result to obtain an accurate result. This is difficult to operate and the accuracy cannot be guaranteed. After adjusting the level, the depth and width measurements will be less difficult. Usually, the tilt angle is not too large, and the image can be adjusted to make it flat relative to the observer. In fact, in other words, leveling is to adjust the image. After leveling, the width can be measured along the horizontal direction of the figure and the depth can be measured in the vertical direction. Measuring along its correct structural direction will improve accuracy and also conform to the operating habits of engineers.

[0036] The traditional solution is that the operator will level the image during the process. The leveling result is closely related to the operator. Because human judgment often differs, and different people understand the image differently. For example, everyone has a different sense of what angle is horizontal. The adjustment will be different depending on the operator, and the operating habits are different, which will lead to different results for different operators for the same image. Therefore, it is necessary to understand the structure of the sample in the image to determine in which direction to adjust.

[0037] In view of this, the present disclosure provides an improved solution.

[0038] The embodiments of the present disclosure provide an improved image processing method. The method includes: determining the contour line and corner points of a figure in an image, wherein the figure has a cross section of stacked parallel layers; determining characteristic corner points among the corner points based on the contour line, wherein the characteristic corner points are related to the boundary lines of the parallel layers in the cross section; determining the angle at which the image needs to be rotated based on the characteristic corner points; and rotating the image in the plane of the image based on the angle so that the boundary line extends in a direction parallel to or perpendicular to the horizontal line in the plane where the image is located. The embodiments of the present disclosure can effectively solve the inconvenience caused by the manual leveling method, can automatically perform leveling processing on tilted images, and improve efficiency and accuracy.

[0039] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0040] Figure 1 1 is a schematic diagram of an example environment 100 in which embodiments according to the present disclosure can be implemented. Figure 1 As shown, the example environment 100 includes a computing device 110 and a client 120 .

[0041] In some embodiments, the computing device 110 may interact with the client 120. For example, the computing device 110 may receive an input message from the client 120 and output a feedback message to the client 120. In some embodiments, the input message from the client 120 may be a TEM image. The computing device 110 may process the TEM image and output the corresponding operation result to the client 120.

[0042] In some embodiments, computing device 110 may include, but is not limited to, a personal computer, a server computer, a handheld or laptop device, a mobile device (such as a mobile phone, a personal digital assistant (PDA), a media player, etc.), consumer electronics, a minicomputer, a mainframe computer, cloud computing resources, etc.

[0043] It should be understood that the structure and function of the example environment 100 are described for exemplary purposes only and are not intended to limit the scope of the subject matter described herein. The subject matter described herein can be implemented in different structures and / or functions. The environment is merely illustrative and is not intended to limit the application environment of the embodiments of the present disclosure.

[0044] In order to explain the principle of the present disclosure more clearly, the following will refer to Figure 2 Let's describe it in more detail.

[0045] Figure 2 A flowchart of an image processing method 200 according to some embodiments of the present disclosure is shown.

[0046] At box 202, the contours and corners of the graphics in the image are determined. A corner point is the junction of two contours. In mathematical terms, a corner point is usually an extreme point. The contours and corners of the graphics can be determined in a variety of ways. In some embodiments, the graphics of the image have a cross-section of stacked parallel layers. The TEM image (or picture) is usually a photograph of a slice. The photograph includes a cross-section of multiple layers. The layers are usually parallel. The size of the relevant characteristic parameters, such as depth, width, etc., can be determined through the TEM picture.

[0047] In some embodiments, the solution of the present disclosure is particularly suitable for Figure 3 The image of the structural features shown is processed. Figure 3 Give a description. Figure 3 , specifically, three parallel layers are included, namely, layer A, layer B and layer C. The three layers are parallel structures and are stacked adjacent to each other. Figure 3 The boundary lines 302 between the various layers are clearly shown. Figure 3 The layers shown in the figure can be layers on a silicon wafer in a semiconductor process. During the wafer manufacturing process, layers are stacked one after another on the silicon wafer, and the result is mostly Figure 3 In the structure shown, each layer is substantially horizontal. Figure 3 After the graphic of the structure shown is photographed, the photographed image can be measured using common software in the industry to determine the parameters of the graphic therein. The present disclosure does not provide a detailed description of the specific measurement process.

[0048] Combine the following Figure 4 Give a description. Figure 4 A schematic diagram of a picture to be processed according to an embodiment of the present disclosure is shown. Figure 4 , which shows the contour line 402 of the figure. There are many ways to obtain the contour line. In some embodiments, the contour line in the image can be determined in the following way: first, the edge points in the image can be determined; then, continuous edge points can be determined from the edge points; finally, the continuous edge points can be determined as the contour line of the figure.

[0049] In some embodiments, edge points can be obtained by algorithms such as Sobel or Canny operators, in other words, contour extraction can be performed. Contour extraction can obtain contour lines by finding continuous edge points in the edge image. Figure 5 Describe. Figure 5 , Figure 5 The figure shows the outline diagram of the embodiment of the present disclosure. Figure 5 As shown, an acquired contour line 402 is shown.

[0050] There are many ways to obtain corner points. In some embodiments, the corner points of the graphics in the image can be directly obtained, that is, the corner points can be directly found through the original image. For example, the SUSAN algorithm can be used. The original image is, for example, an original image taken by a TEM machine.

[0051] The SUSAN algorithm detects edges by comparing the grayscale values ​​of a pixel with those of its neighbors. It is insensitive to noise and does not require the calculation of gradients, which makes it perform well when processing noisy images. In addition, the algorithm is computationally efficient, requiring only a small amount of addition operations for each point, making it suitable for real-time processing. In some embodiments, the SUSAN algorithm searches for the contour edge points of a graphic, forms contour lines, and finds corner points at the same time.

[0052] In some embodiments, determining the corner points of a graphic in an image may include: traversing the image to determine the grayscale value of each pixel point respectively. Determine the difference between the first grayscale value of the central pixel point and the second grayscale value of the non-central pixel point in the area (or local area) centered on each pixel point respectively; and determine each corner point based on each difference. The specific process is further described below. Each area is a range of a predetermined distance from each central pixel point, for example, the edge of the area is the same distance from the central pixel point, that is, a circle with the central pixel point as the center. The embodiments of the present disclosure are not limited to this, and the area may also be an area of ​​other shapes, for example, a square or other regular polygon centered on the central pixel point.

[0053] Similarity judgment :

[0054] In some embodiments, the corner points in the image can be determined by similarity judgment. Specifically, the following methods can be used:

[0055] Calculate the grayscale difference between the pixel grayscale value I (x, y) of the non-central point (x, y) of the local area and the pixel grayscale value I (u, v) of the central point (u, v). The local area can be selected within a predetermined distance from the central pixel point as required.

[0056] Specifically, the entire image is traversed to determine the grayscale value of each pixel. The difference between the grayscale value of the central pixel centered on each pixel (called the first grayscale value) and the grayscale value of the non-central pixel (called the second grayscale value) is determined for each local area. In the process of traversing the entire image, the pixel that is traversed is taken as the center point, and the difference between each non-central pixel and the central pixel in the local area centered on the pixel is calculated. The corner points can then be determined based on the differences, because the difference in grayscale value between the pixel at the corner point and the pixel at the non-corner point and their respective adjacent pixels is different.

[0057] In some embodiments, when each pixel is taken as the center point, the grayscale difference between the grayscale I (x, y) of the non-center pixel in the local area and the grayscale point I (u, v) of the center pixel is calculated. When the absolute value of the difference between the center pixel in each local area and the corresponding non-center pixel in the area is less than a first threshold, the corresponding non-center pixel in the local area is determined as a homogeneous point; otherwise, when the difference between the center pixel in each local area and the corresponding non-center pixel in the area is not less than the first threshold, the corresponding non-center pixel in the local area is determined as a heterogeneous point.

[0058] In some embodiments, determining non-central pixel points in the local area as homogeneous points may include assigning values ​​to the homogeneous points using the following formula: ;

[0059] Among them, w(x,y) indicates that the point is a homogeneous point; (x,y) indicates a non-central point; I (x,y) indicates the grayscale of the non-central point; (u,v) indicates the central point, and I (u,v) indicates the grayscale of the central point; w(x,y)=1 indicates that a homogeneous point is assigned a value of 1; represents the grayscale difference between the center point and the non-center point; t0 represents the first threshold; con(x, y) represents the assignment, which assigns 1 to homogeneous points and 0 to heterogeneous points.

[0060] Homogeneous point accumulation:

[0061] In some embodiments, the homogeneous points determined above may be accumulated to obtain respective accumulated values, that is, the number of homogeneous points in each local area may be accumulated respectively. For example, it may be expressed by the following formula:

[0062] ;

[0063] Wherein N(u,v) represents the accumulated value, that is, the value obtained by accumulating the homogeneous points of the pixel point centered at (u,v). For each local area, the accumulated value of the homogeneous points in the area is calculated.

[0064] Corner point determination :

[0065] In some embodiments, the corner points may be determined in each local area based on the comparison between the accumulated value and the second threshold value. In some embodiments, the corner points may be determined based on N(u,v). Specifically, the response value, for example, the SUSAN response value, may be obtained by the following formula: R(u,v).

[0066] ;

[0067] Wherein G is a predetermined threshold, that is, the second threshold mentioned above.

[0068] It can be seen from the expression that when N(u,v) is less than G, the value of the response value R(u,v) is GN(u,v). Otherwise, when N(u,v) is greater than or equal to G, the value of the response value R(u,v) is 0. It should be understood that the values ​​shown are only exemplary, and the embodiments of the present disclosure are not limited thereto, and other values ​​can be set as needed.

[0069] When the accumulated value is less than the second threshold, the response value R(u,v) of the local area is determined. At this time, the response value is equal to the difference between the second threshold and the accumulated value of the local area. Obviously, this value is greater than 0. The pixel point corresponding to the maximum response value of each area is determined, and the pixel point is determined as the corner point in the corresponding area.

[0070] In the above embodiment, each of the accumulated values ​​is compared with the second threshold value; based on the result of the comparison, part of the local area in the image is filtered out; the value of the maximum response value indicating the comparison result in the local area not filtered out is determined; and the pixel point corresponding to the maximum value is determined as the corner point. It can be seen that by setting the second threshold value G, most of the local areas can be filtered out. After filtering out some local areas, only the local areas not filtered out can be operated, which helps to quickly determine the corner points of the graphics, thereby significantly improving the efficiency of calculation.

[0071] The second threshold G can be adjusted according to the picture quality of the customer. For example, if there are 38 pixels in an area, G can be set to 19. It should be understood that the value of the second threshold G shown is only for illustration, and the embodiments of the present disclosure are not limited thereto, and other values ​​can be set as needed. Because the SUSAN response value of the points near the corner point is relatively large, the point with the maximum response value can be selected from the points with the SUSAN response value in the local area as the corner point.

[0072] In some embodiments, corner points may also be obtained based on contour lines. Determining the corner points of a graphic in an image may include: determining the contour line of the graphic; and determining the point with the maximum local curvature in the contour line as the corner point. In fact, there are many ways to obtain corner points based on contour lines. For example, a CSS (Curvature Scale Space) algorithm may be used to find the point with the maximum local curvature as the corner point.

[0073] See also Figure 6 , Figure 6 Schematic diagram of contour lines and corner points in a picture according to some embodiments of the present disclosure is shown. Figure 6 As shown, a plurality of corner points 404 and a plurality of contour lines 402 are shown.

[0074] Back to Figure 2, at box 204, feature corner points in the corner points are determined based on the contour lines. Feature corner points are corner points that meet predetermined conditions. Feature corner points are related to the boundary lines of parallel layers in the profile. In other words, the selection of feature corner points needs to be based on the features of the boundary lines of parallel layers, so that the contour lines connected by adjacent feature corner points are parallel to the boundary lines or the angle between the two is within a predetermined threshold range. By selecting feature corner points, preparations can be made for subsequent leveling. Usually, in the leveled image, the boundary lines of parallel layers are in a horizontal position, or according to actual needs, they can also be in a position perpendicular to the horizontal line. In some embodiments, the selected feature corner points need to meet the following conditions:

[0075] 1) There are adjacent corner points;

[0076] 2) The contour lines connecting adjacent corner points are continuous;

[0077] 3) The inclination angle of the contour line connecting adjacent corner points meets a certain value (which can be a threshold set manually), for example, 30°, that is, the angle between the contour line and the horizontal line (or vertical line, depending on the placement direction of the silicon wafer) in the image plane is 30°. It should be understood that the embodiments of the present disclosure are not limited to this, and different thresholds can be set according to actual needs.

[0078] After finding the characteristic corner points, you may find that the contour line connecting the characteristic corner points may be close to the horizontal direction (forming a small angle with the horizontal direction) or close to the vertical direction (forming a small angle with the horizontal direction). At this time, you can determine which direction to adjust according to the predetermined rules, that is, to adjust (rotate) in the horizontal direction or the vertical direction, which should be convenient for subsequent measurements in principle.

[0079] Refer to the following Figure 7 Give a description. Figure 7 FIG. 1 is a schematic diagram showing selected feature corner points according to some embodiments of the present disclosure. Figure 7 As can be seen from the figure, there are 12 continuous corner points that meet the conditions, which are indicated by circles with numbers 1 to 12. Figure 7 It can be seen that there are two adjacent corner points in the area where each circle is located. The selected feature corner points can be used for subsequent image leveling processing.

[0080] Back to Figure 2 Continuing the description, at block 206, the angle at which the image needs to be rotated is determined based on the feature corner points. In some embodiments, determining the angle at which the image needs to be rotated based on the feature corner points may include: determining a fitting curve for each point on each contour line connected by each adjacent feature corner point; determining an angle between the fitting curve and a horizontal line in a plane where the image is located; and determining the angle as the angle, that is, the angle at which the image needs to be rotated.

[0081] In some embodiments, determining the fitting curves of the points on the contour lines connected by the adjacent feature corner points may include: taking a specified reference sub-region (eg, Figure 7 Taking the contour line in region 5 in the image as a reference, the points on the contour lines in each sub-region including the feature corner points (for example, other regions except region 5) are aggregated into the reference sub-region; and the points on the aggregated contour lines are fitted to generate a fitting curve.

[0082] In some embodiments, taking the contour line in the designated reference sub-region including the feature corner point as the reference, respectively aggregating the points on the contour lines in the sub-regions of each feature corner point into the reference sub-region may include: determining a first point corresponding to the average value of the horizontal coordinates and the average value of the vertical coordinates of the points on the contour line in the reference sub-region, in short, taking the average of the horizontal coordinate and the vertical coordinate of the contour line and determining the corresponding point; determining a second point corresponding to the average value of the horizontal coordinate and the average value of the vertical coordinate of the contour lines in other sub-regions including the feature corner point, taking the average of the horizontal coordinate and the vertical coordinate of these contour lines and determining the corresponding points respectively; and moving each point on the contour line in the other sub-regions parallel to the reference sub-region along the direction from the second point to the first point, so that the second point coincides with the first point respectively.

[0083] like Figure 3 In the structure shown in the figure, each layer is horizontal. The figure will also tilt due to the tilt caused during the shooting process. There are obvious grayscale features between the layers. The contour line obtained based on the feature is horizontal when there is no tilt, so the angle between the contour line and the horizontal direction is the tilt angle. The figure can be leveled according to the determined tilt angle. The solution in the above embodiment can determine the tilt angle.

[0084] The method for determining the angle for rotating an image is described in detail below with reference to a formula.

[0085] In some embodiments, the regions of the characteristic corner points that meet the conditions are taken and aggregated into the same region. Specifically, one of the regions is used as the reference, that is, the points on the contour line in the region are used as the reference points. The points on the contour lines of other regions (also called coordinate points) are translated over. The curve can be fitted based on the X and Y coordinates of the coordinate points. In this embodiment, the coordinate points of each region are actually aggregated, and the coordinate points of each region are aggregated into one piece to facilitate the subsequent fitting of the curve.

[0086] For example, Figure 7 Area 5 in is used as the reference.

[0087] ;

[0088] ;

[0089] Where n represents the number of points selected on the contour line of the local area; x1 represents the x-coordinate (horizontal coordinate) of the first point, y1 represents the y-coordinate (vertical coordinate) of the first point, and x n Indicates the x-coordinate of the nth point, y n Indicates the y-coordinate of the nth point, parameter represents the average value of the x-direction of the coordinates of the fifth region (region 5), Indicates the average value of the coordinates of the fifth region (region 5) in the y direction.

[0090] The average values ​​of the coordinates of other regions 1 to 12 are … arrive , … arrive ; For example, for the first area (area 1), the distances required to move to area 5 are:

[0091]

[0092]

[0093] and They represent the average x-direction and y-direction coordinates of the first region, Δx1 represents the distance the first region needs to move along the x-direction, and Δy1 represents the distance the first region needs to move along the y-direction. In addition, as mentioned above, the parameters represents the average value of the x-direction of the coordinates of the fifth region (region 5), Indicates the average value in the y direction of the coordinates of the fifth region (region 5).

[0094] Move the points in area 1 parallel to area 5, and adjust the other areas in the same way. Fit its slope (fit the curves that are aggregated together to obtain its slope), which is the angle that needs to be leveled. For the method of parallel movement, see Figure 8 .

[0095] Figure 8 FIG. 2 shows a schematic diagram of the movement process of points on a contour line according to some embodiments of the present disclosure. Figure 8As shown, the dotted line L5 represents the point on the contour line in area 5. Several points on the contour line can be selected as needed. Obviously, the more points selected, the higher the accuracy. Appropriate selection can be made taking into account both efficiency and accuracy; the dotted line L1 represents the point on the contour line in area 1. The arrow shown in the figure indicates the moving direction. The moving distance is determined by Δx1 and Δy1. During the movement, the point corresponding to the center value of the horizontal coordinate and the vertical coordinate of the dotted line L1 moves to coincide with the point corresponding to the center value of the horizontal coordinate and the vertical coordinate of the dotted line L5. During the movement, the dotted line L1 maintains the same angle, that is, it moves in parallel. For the contour lines in area 1 and the contour lines in area 5, if both are straight line segments, the points corresponding to the center values ​​of their horizontal coordinates and vertical coordinates mentioned above are the midpoints of the two line segments respectively. The same treatment can be performed for the contour lines in other areas.

[0096] See also Fig. 9 , Fig. 9 A schematic diagram of a fitting curve according to some embodiments of the present disclosure is shown.

[0097] Fig. 9 The small dots shown in the figure represent the results of moving the points on the contour lines in each area to the specified area. By fitting the aggregated curve, its slope can be determined, and thus the inclination angle of the graph can be determined. Specifically, the slope of the curve is the inclination angle of the graph. In addition Fig. 9 The expression of the fitting curve is shown in : y=1.0303x+69.734, where x is the horizontal coordinate and y is the vertical coordinate.

[0098] By fitting, the influence of abnormal points on the curve can be reduced, so that the inclination angle of the graph can be accurately determined.

[0099] In some embodiments, determining the slope of the contour line based on the corner points may include: determining the perpendicular bisector of the contour line connecting the two adjacent feature corner points based on the two adjacent feature corner points; and determining the angle based on the slope of the perpendicular bisector relative to the horizontal line. Specifically, the tilt angle of the image may be determined by finding the perpendicular bisector.

[0100] Combine the following Fig.10 Give a description. Fig.10 FIG. 2 shows a schematic diagram of a graph showing a perpendicular midline according to some embodiments of the present disclosure. Fig.10 As described above, the perpendicular bisector 1002 and two points P1 and P2 on the perpendicular bisector, as well as the inclination angle θ are shown.

[0101] In some embodiments, the contour line that satisfies a certain inclination angle can be determined by the following formula. The points on the perpendicular midline 1002 satisfy:

[0102] |f(x,y)-G1(x,y)| = |f(x,y)-G2(x,y)|;

[0103] Among them, G1(x,y) is the coordinate of the first corner point, G2(x,y) is the coordinate of the second corner point; |f(x,y)-G1(x,y)| is the distance from the point in the contour area to the corner point. When the distance from the point in the contour area to the corner point is equal, it is the perpendicular bisector of the contour line.

[0104] The slope of the perpendicular bisector is the angle of inclination of the image. The image can be leveled according to the inclination angle. For example, if the image is tilted by 20°, that is, θ is 20°, then the image can be rotated so that its inclination angle is 0°, which means that it is leveled.

[0105] In the above embodiment, the slope is determined by solving the expression of f(x, y) according to the parameters in the expression. For example, f(x, y) is a one-variable equation, and f(x, y)=ax+by is set, then the slope can be determined according to a and b, and the slope is the inclination angle.

[0106] It should be pointed out that, for the characteristic corner points determined above, any two adjacent characteristic corner points can be selected as G1 and G2. Thus, the corresponding perpendicular bisector can be determined. This method is based on the recognition that the line connecting any two adjacent characteristic corner points has basically the same direction. In fact, as mentioned above, the characteristic corner points are related to the boundary lines of the parallel layers in the section, and the line connecting the adjacent characteristic corner points is basically parallel to the boundary lines of the parallel layers in the section. Therefore, the inclination angle of the boundary lines of the parallel layers can be basically determined by the adjacent characteristic corner points, thereby obtaining the inclination angle of the figure.

[0107] At block 208 , the image is rotated within the plane of the image based on the determined angle so that the boundary line extends in a direction that is parallel or perpendicular to a horizontal line within the plane of the image.

[0108] See also Fig.11 , Fig.11 A schematic diagram of an image after leveling according to some embodiments of the present disclosure is shown. By the method of the embodiment of the present disclosure, the processed image is rotated. Fig.10 The shown changes to Fig.11 To the observer or measuring tool, the image is flat (positive), thus facilitating subsequent measurements.

[0109] It should be noted that the shooting is usually performed after the platform is adjusted. The image after shooting can be leveled. In some embodiments of the present disclosure, the leveling of the image is to adjust (rotate) the image only in one plane.

[0110] In some embodiments of the present disclosure, by extracting TEM image features, leveling is automatically performed, so that when performing subsequent measurements, the problem of reduced accuracy due to image tilt can be effectively solved.

[0111] Some embodiments of the present disclosure provide a graphics processing method. It should be noted that the examples given in the above embodiments are only for illustrating the solutions of the embodiments of the present disclosure, and are not intended to limit the solutions of the present disclosure.

[0112] It should be understood that the embodiments shown in the drawings are only for schematically illustrating the solutions of some embodiments of the present disclosure and are not intended to limit the present disclosure. The embodiments of the present disclosure may also have various other forms.

[0113] An electronic device is also disclosed in an embodiment of the present disclosure. The electronic device includes: a processor; and a memory coupled to the processor, the memory having instructions stored therein, and when the instructions are executed by the processor, the device performs actions, the actions including: determining contour lines and corner points of a figure in an image, wherein the figure has a cross section of stacked parallel layers; determining characteristic corner points among the corner points based on the contour lines, wherein the characteristic corner points are related to boundary lines of the parallel layers in the cross section; determining an angle to be rotated for the image based on the characteristic corner points; and rotating the image in a plane of the image based on the angle so that the boundary line extends in a direction parallel to or perpendicular to a horizontal line in the plane where the image is located.

[0114] An embodiment of the present disclosure further discloses a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the image processing method according to the embodiment of the present disclosure is implemented.

[0115] In some embodiments of the present disclosure, TEM images are used as examples for explanation. Those skilled in the art should understand that the embodiments of the present disclosure are not limited thereto, but can be applied to a variety of images. As long as the images contain a film stack structure, the methods of the embodiments of the present disclosure can be adopted.

[0116] The solutions of some embodiments of the present disclosure adopt an improved image processing method, which can effectively solve the inconvenience caused by manual leveling and improve efficiency and accuracy.

[0117] Fig.12A schematic block diagram of an electronic device according to some exemplary embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0118] like Fig.12 As shown, the device 1200 includes a CPU 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the device 1200 can also be stored. The CPU 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0119] Multiple components in the device 1200 are connected to the I / O interface 1205, including: an input unit 1206, such as a keyboard, a mouse, etc.; an output unit 1207, such as various types of displays, speakers, etc.; a storage unit 1208, such as a disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0120] The various processes and processing described above, such as method 200, may be executed by CPU 1201. For example, in some embodiments, method 200 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by CPU 1201, one or more steps in method 200 described above may be performed.

[0121] The scheme according to the embodiment of the present disclosure may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium on which computer-readable program instructions for executing various aspects of the present disclosure are loaded. The computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. The computer-readable program instructions may be downloaded from the computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network and / or a wireless network.

[0122] Various embodiments of the present disclosure have been described above, and the above descriptions are exemplary and are only optional embodiments of the present disclosure, not exhaustive, and are not intended to limit the present disclosure. Although the claims in this application have been formulated for specific combinations of features, it should be understood that the scope of the present disclosure also includes any novel features or any novel combination of features disclosed herein, whether or not it relates to the same scheme in any claim currently claimed for protection. The applicant hereby informs that new claims may be formulated into these features and / or combinations of these features during the examination of this application or in any further application derived therefrom.

[0123] The terms used in this article are selected to best explain the principles of each embodiment, practical application or technical improvement in the market, or to enable other ordinary technicians in the field to understand the embodiments disclosed herein. For those skilled in the art, the present disclosure may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. An image processing method, comprising: determining contours and corner points of a pattern in an image, wherein the pattern has a cross-section of stacked parallel layers; Determining a characteristic corner point among the corner points based on the contour line, wherein the characteristic corner point is a corner point among the corner points that meets a predetermined condition and is related to a boundary line of the parallel layers in the cross section; Determining the angle at which the image needs to be rotated based on the feature corner points specifically includes: determining a fitting curve of each point on each contour line connected to each adjacent feature corner point; determining an angle between the fitting curve and a horizontal line in a plane where the image is located or a vertical line perpendicular to the horizontal line; and determining the angle as the angle; and The image is rotated within the plane of the image based on the angle so that the boundary line extends in a direction parallel to or perpendicular to a horizontal line within the plane where the image is located.

2. The method according to claim 1, wherein determining the contour of the figure in the image comprises: determining edge points in the image; Determining continuous edge points from the edge points; as well as The continuous edge points are determined as the contour line of the graphic.

3. The method according to claim 1, wherein determining the corner points of the figure in the image comprises: Determine the gray value of each pixel; Determine the difference between the first grayscale value of the central pixel point in each area centered on each pixel point and the second grayscale value of the non-central pixel point; as well as The corner points are determined based on the difference values.

4. The method according to claim 3, wherein determining each of the corner points based on each of the difference values ​​comprises: In response to an absolute value of a difference between the central pixel point in the region and a corresponding non-central pixel point in the region being less than a first threshold, determining the corresponding non-central pixel point as a homogeneous point; Accumulate the number of homogeneous points in each area to obtain their respective cumulative values; as well as Corner points are determined in each area based on the comparison between each of the accumulated values ​​and the second threshold value.

5. The method according to claim 4, wherein determining the corner point in each area based on the comparison between each of the accumulated values ​​and the second threshold value comprises: In response to the corresponding accumulated value of the region being less than the second threshold, determining each response value of the region, the response value being equal to the difference between the second threshold and the corresponding accumulated value of the region; as well as The pixel point corresponding to the maximum response value of each region is determined, and the pixel point corresponding to the maximum response value is determined as the corner point in the corresponding region.

6. The method according to claim 1, wherein determining the corner points of the figure in the image comprises: determining an outline of the graphic; as well as The point with the maximum local curvature in the contour line is determined as the corner point.

7. The method according to claim 1, wherein determining the characteristic corner points among the corner points based on the contour line comprises determining the corner points satisfying the following conditions as the characteristic corner points: There are adjacent corner points; Adjacent corner points are connected by continuous contour lines; and Adjacent corner points are connected by contour lines whose inclination angle is less than a predetermined threshold.

8. According to the method of claim 1, determining the fitting curves of the points on the contour lines connected by the adjacent feature corner points comprises: Taking the contour line in the designated reference sub-region including the feature corner point as a reference, respectively aggregating the points on the contour line in each sub-region including the feature corner point into the reference sub-region; as well as The points on the aggregated contour line are fitted to generate a fitting curve.

9. The method according to claim 8, wherein respectively aggregating points on the contour lines in the sub-regions of the feature corner points into the reference sub-region comprises: Determine a first point corresponding to an average value of the abscissas and an average value of the ordinates of the points on the contour line within the reference sub-region; Determine a second point corresponding to an average value of the abscissa and an average value of the ordinate of the contour lines in other sub-regions including the feature corner point; as well as Each point on the contour line in the other sub-region is moved parallel to the reference sub-region along the direction from the second point to the first point, so that the second point coincides with the first point.

10. The method of claim 1, wherein determining the slope of the contour line based on the corner point comprises: Determine, based on two adjacent feature corner points, a perpendicular bisector of a contour line connecting the two adjacent feature corner points; as well as The angle is determined based on a slope of the perpendicular bisector relative to the horizontal line or a vertical line perpendicular to the horizontal line.

11. An electronic device, comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, the instructions causing the device to perform actions when executed by the processor, the actions comprising: determining contours and corner points of a pattern in an image, wherein the pattern has a cross-section of stacked parallel layers; Determining a characteristic corner point among the corner points based on the contour line, wherein the characteristic corner point is a corner point among the corner points that meets a predetermined condition and is related to a boundary line of the parallel layers in the cross section; Determining the angle at which the image needs to be rotated based on the feature corner points specifically includes: determining a fitting curve of each point on each contour line connected to each adjacent feature corner point; determining an angle between the fitting curve and a horizontal line in a plane where the image is located or a vertical line perpendicular to the horizontal line; and determining the angle as the angle; and The image is rotated within the plane of the image based on the angle so that the boundary line extends in a direction parallel to or perpendicular to a horizontal line within the plane where the image is located.

12. A computer-readable storage medium having machine-executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 10.

Images