Image processing method, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention relates to an image processing method, electronic equipment and a computer readable storage medium. The method comprises the steps that a first feature point of a graph contour in an image is determined, the first feature point is a point with an extreme value in a preset range in the graph contour, and the graph contour in the image comprises section line contours of parallel layers; determining a first feature region based on the first feature point, wherein the first feature region surrounds a first graph contour including the first feature point; performing feature matching in the image by using the first feature region to determine a second feature region, so that the first graphic contour is matched with a second graphic contour in the second feature region; determining an angle at which the image needs to be rotated based on the first graph contour and the second graph contour; and rotating the image in the plane of the image based on the angle so that the section line extends in a direction parallel to or perpendicular to a horizontal line in the plane where the image is located. According to the technical scheme, the leveling efficiency and precision can be improved.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to integrated circuits, and more particularly, to image processing methods, electronic devices, and computer-readable storage media. Background Art

[0002] Transmission Electron Microscope (TEM) is an electron optical instrument with high resolution and high magnification. TEM can be used as a failure analysis tool in semiconductor manufacturing.

[0003] In the manufacturing process of semiconductor chips, it is sometimes necessary to perform TEM measurement on the chips in the process. During the measurement process, due to sample preparation conditions and the angle rotation of the TEM shooting platform, the sample may not reach the optimal horizontal angle. In this way, the pattern or film to be measured in the TEM image will be tilted, which is not conducive to subsequent pattern measurement. Therefore, in most cases, it is necessary to manually adjust the level of the image according to the state of the pattern in the image before measuring. In the process of manual leveling, the results will vary from person to person due to factors such as the operator's habits and state.

[0004] The disadvantage of the traditional manual leveling solution is that it is time-consuming and the accuracy of leveling is greatly affected by subjective factors. Summary of the invention

[0005] According to an exemplary embodiment of the present disclosure, an image processing scheme is provided to at least partially overcome the above or other potential drawbacks.

[0006] According to one aspect of the present disclosure, an image processing method is provided. The method includes: determining a first feature point of a graphic contour in an image, wherein the first feature point is a point with an extreme value within a predetermined range in the graphic contour, and the graphic contour in the image includes a profile line contour of a parallel layer; determining a first feature area based on the first feature point, wherein the first feature area surrounds the first graphic contour including the first feature point; performing feature matching in the image using the first feature area to determine a second feature area, so that the first graphic contour matches the second graphic contour within the second feature area; determining the angle at which the image needs to be rotated based on the first graphic contour and the second graphic contour; and rotating the image in the plane of the image based on the angle so that the profile line extends in a direction parallel to or perpendicular to the horizontal line in the plane where the image is located. The technical solution disclosed in the present disclosure can effectively solve the inconvenience caused by the manual leveling method and improve the leveling efficiency and leveling accuracy.

[0007] In a second aspect of the present disclosure, an electronic device is provided. The electronic device includes a processor; and a memory coupled to the processor, the memory having instructions stored therein, and when the instructions are executed by the processor, the device performs an action, the action including: determining a first feature point of a graphic contour in an image, wherein the first feature point is a point with an extreme value within a predetermined range in the graphic contour, and the graphic contour in the image includes a profile line contour of a parallel layer; determining a first feature area based on the first feature point, wherein the first feature area surrounds the first graphic contour including the first feature point; performing feature matching in the image using the first feature area to determine a second feature area, so that the first graphic contour matches the second graphic contour in the second feature area; determining an angle to be rotated for the image based on the first graphic contour and the second graphic contour; and rotating the image in the plane of the image based on the angle so that the profile line extends in a direction parallel to or perpendicular to a horizontal line in the plane where the image is located.

[0008] In some embodiments, determining the first feature point of the contour of a graphic in an image includes: moving a detection line parallel to a first boundary of the image in a direction perpendicular to the first boundary; respectively determining the coordinates of the intersection points on the contour of the graphic that intersect with the detection line; and taking the intersection points whose coordinates in a predetermined range along the direction of the detection line on the contour of the graphic as the first feature points.

[0009] In some embodiments, determining the first feature area based on the first feature point includes: extending from the first feature point along the first direction to a first auxiliary point on a second contour opposite to the first graphic contour; selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point, so that the second auxiliary point intersects with the graphic contour of the graphic at a third auxiliary point and a fourth auxiliary point in a direction perpendicular to the line segment; and determining the first feature area based on the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point.

[0010] In some embodiments, determining the first feature area based on the first feature point includes: extending a predetermined distance from the first feature point along the first direction to the first auxiliary point; selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point, so that the second auxiliary point intersects with the graphic contour of the figure at a third auxiliary point and a fourth auxiliary point in a direction perpendicular to the line segment; and determining the first feature area based on the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point.

[0011] In some embodiments, determining the first feature area based on the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point includes: selecting an area surrounding the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point as the first feature area.

[0012] In some embodiments, selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point includes: selecting the second auxiliary point between the line segment connecting the first feature point and the first auxiliary point so that the distance between the third auxiliary point and the fourth auxiliary point is maximized; or selecting the midpoint of the line segment connecting the first feature point and the first auxiliary point as the second auxiliary point.

[0013] In some embodiments, features are matched in images using one of the following methods: brute force matching; and fast approximate neighbor matching.

[0014] In some embodiments, determining the angle at which the image needs to be rotated based on the first graphic contour and the second graphic contour includes: determining a fitting straight line of the midpoints of the first feature area and the second feature area; determining the angle between the fitting straight line and a horizontal line in the plane where the image is located or a vertical line perpendicular to the horizontal line; and determining the angle as the angle at which the image needs to be rotated.

[0015] In a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0016] It will be understood from the following description that the technical solution of the present disclosure can effectively solve the inconvenience caused by the manual leveling method and improve the leveling efficiency and leveling accuracy.

[0017] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the disclosure, nor is it intended to limit the scope of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented; Figure 2 A flowchart of an image processing method according to some embodiments of the present disclosure is shown; Figure 3 A schematic diagram showing a TEM image to be processed according to an embodiment of the present disclosure is shown; Figure 4 Shows Figure 3 The schematic diagram of the outline of the image to be processed is shown; Figure 5 A schematic diagram showing detection of feature points in a continuous contour along a first direction in an image in some embodiments of the present disclosure is shown; Figure 6 A schematic diagram showing feature points in each continuous contour within a given range in an image of some embodiments of the present disclosure; Figure 7 A schematic diagram showing two points in an image used to determine a first feature area in some embodiments of the present disclosure; Figure 8 A schematic diagram showing four points in an image used to determine a first feature area in some embodiments of the present disclosure; Fig. 9 A schematic diagram showing a contour map and a first feature region in an original image according to some embodiments of the present disclosure; Fig.10 A schematic diagram showing various characteristic regions according to some embodiments of the present disclosure; Fig.11 A schematic diagram showing slope fitting of characteristic graphs of some embodiments of the present disclosure is shown; Fig.12 A schematic diagram showing a leveled image according to some embodiments of the present disclosure; Fig.13 A schematic diagram showing a contour line that is unbounded downwards according to some embodiments of the present disclosure; Fig.14 A schematic diagram showing a contour line that is unbounded downward according to other embodiments of the present disclosure; Fig.15 A schematic diagram showing detection of feature points in a continuous contour along a second direction in an image according to some embodiments of the present disclosure is shown; Fig.16 A schematic diagram showing two points in an image used to determine a feature area in some embodiments of the present disclosure is shown; Fig.17 A schematic diagram showing four points in an image used to determine a feature area in some embodiments of the present disclosure; Fig.18 A schematic diagram showing a TEM image to be processed according to another embodiment of the present disclosure is shown; Fig.19 Shows Fig.18 The schematic diagram of the outline of the image to be processed is shown; Fig. 20 The first direction is shown Fig.19 Schematic diagram of detecting feature points in a continuous contour as shown in; Fig.21 A schematic diagram showing four points for determining a first feature domain according to some embodiments of the present disclosure is shown; Fig. 22 Shows Fig.18 A schematic diagram of a graphic outline within a first feature region of an original image shown in ; Fig.23 Shows Fig.18 Schematic diagram of the graphic outlines within each feature area of ​​the original image shown in ; Fig.24 Shows Fig.23 A schematic diagram of the slope fit of the graphical profile shown in ; and Fig.25 A block diagram of a computing device capable of implementing various embodiments of the present disclosure is shown.

[0019] In the various drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0020] The principles of the present disclosure will be described below with reference to the various exemplary embodiments shown in the accompanying drawings. It should be understood that the description of these embodiments is only to enable those skilled in the art to better understand and further implement the present disclosure, and is not intended to limit the scope of the present disclosure in any way. It should be noted that similar or identical reference numerals may be used in the figures where feasible, and similar or identical reference numerals may represent similar or identical functions. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods described herein may be adopted without departing from the principles of the present invention described herein.

[0021] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects.

[0022] As mentioned earlier, before TEM photography, the sample may be tilted for various reasons. TEM images (or images) need to capture the depth and width of the pattern. If the image is not flat, it will be difficult to measure and may cause large deviations. Specifically, if the image is tilted, the corresponding figure is tilted. When measuring the depth and width, it is necessary to obtain the results when the figure is horizontal. In other words, the measurement will take the depth and width of the sample when it is horizontal as the final result. If the image is tilted, the same tilt needs to be maintained when measuring the result to obtain an accurate result. This is difficult to operate and the accuracy cannot be guaranteed. After adjusting the level, the depth and width measurements will be less difficult. Usually, the tilt angle is not too large, and the image can be adjusted to make it flat relative to the observer. In fact, in other words, leveling is to adjust the image. After leveling, the width can be measured along the horizontal direction of the figure and the depth can be measured in the vertical direction. Measuring along its correct structural direction will improve accuracy and also conform to the operating habits of engineers.

[0023] The traditional solution is that the operator will level the image during the process. The leveling result is closely related to the operator. Because human judgment often differs, and different people understand the image differently. For example, everyone has a different sense of what angle is horizontal. The adjustment will be different depending on the operator, and the operating habits are different, which will lead to different results for different operators for the same image. Therefore, it is necessary to understand the structure of the sample in the image to determine in which direction to adjust.

[0024] Currently, wafer factories (Fab) mainly use manual leveling, which is greatly affected by human factors. Some semi-automatic methods require manual setting of templates and then matching to achieve semi-automatic leveling. This method still requires manual participation in setting templates, which is not completely automatic leveling, and template production is still affected by human factors and varies.

[0025] In view of this, the present disclosure provides an improved solution.

[0026] The embodiment of the present disclosure provides an improved image processing method. The method includes: determining a first feature point of a graphic contour in an image, wherein the first feature point is a point with an extreme value within a predetermined range in the graphic contour, and the graphic contour in the image includes a profile line contour of a parallel layer; determining a first feature area based on the first feature point, wherein the first feature area surrounds the first graphic contour including the first feature point; using the first feature area to perform feature matching in the image to determine a second feature area, so that the first graphic contour matches the second graphic contour in the second feature area; determining the angle at which the image needs to be rotated based on the first graphic contour and the second graphic contour; and rotating the image in the plane of the image based on the angle so that the profile line extends in a direction parallel to or perpendicular to the horizontal line in the plane where the image is located. The technical solution disclosed in the present disclosure can effectively solve the inconvenience caused by the manual leveling method and improve the leveling efficiency and leveling accuracy.

[0027] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0028] Figure 1 1 is a schematic diagram of an example environment 100 in which embodiments according to the present disclosure can be implemented. Figure 1 As shown, the example environment 100 includes a computing device 110 and a client 120 .

[0029] In some embodiments, the computing device 110 may interact with the client 120. For example, the computing device 110 may receive an input message from the client 120 and output a feedback message to the client 120. In some embodiments, the input message from the client 120 may be image data, such as a TEM image. The computing device 110 may perform corresponding mathematical operations on the image data and output corresponding operation results to the client 120.

[0030] In some embodiments, computing device 110 may include, but is not limited to, a personal computer, a server computer, a handheld or laptop device, a mobile device (such as a mobile phone, a personal digital assistant (PDA), a media player, etc.), consumer electronics, a minicomputer, a mainframe computer, cloud computing resources, etc.

[0031] It should be understood that the structure and function of the example environment 100 are described for exemplary purposes only and are not intended to limit the scope of the subject matter described herein. The subject matter described herein can be implemented in different structures and / or functions. The environment is merely illustrative and is not intended to limit the application environment of the embodiments of the present disclosure.

[0032] In order to explain the principle of the present disclosure more clearly, the following will refer to Figure 4 Let's describe it in more detail.

[0033] Figure 2 A flowchart for an image processing method 200 according to some embodiments of the present disclosure is shown.

[0034] At block 202 , a first feature point of a graphic contour in an image is determined, wherein the first feature point is a point having an extreme value within a predetermined range in the graphic contour, and the graphic contour in the image includes profiles of cross-sections of parallel layers.

[0035] TEM images (or pictures) are usually photos of slices. The photos include cross-sections of multiple layers. The layers are usually parallel. The size of relevant characteristic parameters, such as depth, width, etc., can be determined through TEM pictures. After the slices are photographed, the photographed images can be measured using software commonly used in the industry to determine the parameters of the graphics therein. The specific measurement process is not described in detail in this disclosure.

[0036] In some embodiments, the solution of the embodiment of the present disclosure is particularly suitable for Figure 3 The TEM images shown were processed.

[0037] See also Figure 3 and Figure 4 . Figure 3 A schematic diagram showing a TEM image to be processed according to an embodiment of the present disclosure is shown; Figure 4 Shows Figure 3The schematic diagram of the outline of the image to be processed is shown in FIG. Figure 3 and Figure 4 , where an outline 302 of a graphic is shown.

[0038] There are many ways to obtain contours. For example, commonly used binarization edge detection algorithms can be used to extract contours. Specifically, commonly used algorithms include the Sobel operator, the Prewitt operator, the Canny edge detection algorithm, and the like. The Sobel operator is an important processing method in the field of computer vision and is often used for edge detection. The Prewitt operator is an edge detection of a first-order differential operator. The Canny edge detection algorithm is a multi-level edge detection algorithm. For example, in some embodiments, the contour lines in the image can be determined by any of the above algorithms in the following manner: first, the edge points in the image can be determined; then, continuous edge points can be determined from the edge points; and finally, the continuous edge points can be determined as the contour of the graphic.

[0039] Figure 3 and Figure 4 304 shown in the figure may represent the parallel section line profile in the above-mentioned layer. 300 represents a substrate, such as a silicon substrate. In fact, the TEM image is characterized by a silicon-based substrate at the bottom and various structures grown on the top during the chip manufacturing process. Therefore, the upper part of the image is the direction of the pattern, and the upper edge profile of the pattern is the uppermost boundary of the pattern. Usually, the graphics in the TEM image mainly refer to the graphics of the same layer in the chip manufacturing process. This layer can also be regarded as a thin film (such as various oxide layer films, metal layer films, etc.). For example, there will be different AA graphics in the active layer AA (Active Area Layer, referred to as AA). However, the TEM image is a cross-section of the chip, so it will contain multiple layers or films.

[0040] In some embodiments, a detection line parallel to a first boundary of an image can be moved in a direction perpendicular to the first boundary; the coordinates of intersection points on the contour of the graphic that intersect with the detection line can be determined respectively; and the intersection points whose coordinates in a direction along the detection line within a predetermined range on the contour of the graphic have extreme values ​​can be taken as first feature points.

[0041] In some embodiments, determining a first feature point of a graphic contour in an image may include: moving a detection line parallel to a first boundary of the image in a direction perpendicular to the first boundary; respectively determining the coordinates of points on the graphic contour on the graphic that intersect with the detection line; and taking a point with a maximum or minimum coordinate value within a predetermined range on the graphic contour in the direction along the detection line as a first feature point.

[0042] In some embodiments, the highest point position of the continuous contour may be detected from left to right along a detection line extending along the Y direction (the longitudinal direction in the figure) to obtain feature points.

[0043] In some embodiments, the highest point within a given range of a continuous contour is defined as follows: Assuming that the point where the Y-direction detection line intersects the contour of the graphic is P0(x0, y0), then each point on the contour within a given neighborhood δ can be expressed as: P(x, y), where |x-x0|≤δ; when y0-y≥0 is satisfied for all P points, then point P0 is the highest point of the contour within the given range, and at this time, the coordinate in the Y direction can be said to have the maximum coordinate value, in other words, the coordinate of the intersection along the direction of the detection line has an extreme value (that is, the coordinate value in the Y direction has an extreme value). The setting of δ can be given an empirical value according to the number of pixels, for example, it is recommended to be less than 10.

[0044] Combine the following Figure 5 and Figure 6 Give a description. Figure 5 A schematic diagram of detecting feature points in a continuous contour along a first direction in an image in some embodiments of the present disclosure is shown. Figure 5 , the detection line 306 is shown, and the detection line 306 extends along a first direction (longitudinal direction, also referred to as Y direction). During the detection process, the detection line 306 moves to the right parallel to the left edge of the image. Figure 6 A schematic diagram showing feature points in each continuous contour within a given range in an image of some embodiments of the present disclosure is shown. Figure 5 and Figure 6 There are the following five highest points in the figure contour, which are recorded as the characteristic points of the figure in the figure, namely Figure 6 1, 2, 3, 4 and 5 shown in the figure represent the identified feature points, that is, the highest point positions of the continuous contour mentioned above. For example, the feature point represented by the number 1 can be called the first feature point. Figure 6 Five feature points are shown in FIG. 1 . It should be understood that in some embodiments of the present disclosure, it is not necessary to identify all of these feature points, but only one of the feature points may be identified.

[0045] Figure 5 and Figure 6 FIG. 3 shows that the detection line 306 extending in the Y direction moves parallel to the left edge of the image to the right to detect the first feature point. The embodiments of the present disclosure are not limited thereto, and detection lines of different colors along different directions can also be used to perform detection in other directions. Fig.15 , Fig.15 A schematic diagram of detecting feature points in a continuous contour along a second direction in an image according to some embodiments of the present disclosure is shown. Fig.15As shown, the detection line 306 extends in a direction parallel to the bottom edge of the image. During the detection process, the detection line moves perpendicular to the bottom edge. The specific detection process is substantially the same as that described in the previous embodiment. During the detection process, the point with the smallest horizontal coordinate value among the intersections of the detection line and the contour line can be determined. At this time, the coordinate in the X direction has the minimum coordinate value (or the coordinate has an extreme value in the X direction). Or the image is rotated 90 degrees to the right, and the point detected at this time can also be called the highest point.

[0046] At block 204 , a first feature region is determined based on the first feature point, wherein the first feature region surrounds a first graphic contour including the first feature point.

[0047] In some embodiments, the first feature area can be determined as follows: extend from the first feature point along the first direction to a first auxiliary point on a second contour opposite to the first graphic contour; select a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point, so that the second auxiliary point intersects with the graphic contour of the graphic at a third auxiliary point and a fourth auxiliary point in a direction perpendicular to the line segment; and determine the first feature area based on the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point.

[0048] In some embodiments, an area surrounding the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point may be selected as the first feature area.

[0049] Refer to the following Figure 7 Give a description. Figure 7 A schematic diagram showing two points in an image used to determine a first feature area in some embodiments of the present disclosure is shown.

[0050] As mentioned above, 5 feature points have been found. We can take the feature point 1 on the left and move it downward along the Y direction to find the second point where the detection line intersects the contour, which is recorded as P1(x1,y1). Figure 7 As shown, the detection line 306 moves to the right and intersects the contour line 302 at points P0 (ie, the feature point indicated by number 1 above) and P1.

[0051] See also Figure 8 , Figure 8 A schematic diagram of four points used to determine the first feature area in an image of some embodiments of the present disclosure is shown. A second auxiliary point (not shown in the figure) can be selected on the line between P0 and P1 so that the second auxiliary point intersects with the continuous graphic contour at a third auxiliary point P2 and a fourth auxiliary point P3 in a direction perpendicular to the line.

[0052] In some embodiments, the midpoint of a line segment connecting the first feature point and the first auxiliary point may be selected as the second auxiliary point.

[0053] In some embodiments, selecting the second auxiliary point between the line segment connecting the first feature point and the first auxiliary point may include: selecting the second auxiliary point so that the distance between the third auxiliary point and the fourth auxiliary point is maximized.

[0054] For example, after taking the midpoints of P0 and P1, find the intersection points of the left and right contours P2 and P3 along the X direction. TEM images generally cross-cut multiple similar structures, so the shapes are basically similar. In other words, because all the graphics in the image are repetitive, there must be left and right contour boundaries. When the left contour point P2 reaches the left boundary of the image, another point can be selected to repeat the selection of P1, P2, and P3. This aims to obtain as many feature graphics as possible within the range of the points P0, P1, P2, and P3, so as to avoid matching to the wrong position later due to a small selection range.

[0055] In some embodiments, the first feature area can be determined based on the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point. For example, a rectangular area is selected, and the rectangular area includes the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point. The rectangular area is the first feature area. As can be seen from the figure, the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point are respectively located on the edges of the rectangular area.

[0056] In some embodiments, there may be a situation where the graphic outline is unbounded downward, and the second outline intersection point cannot be found. In this case, the maximum span of Y to be searched downward needs to be limited to obtain the position of P1.

[0057] See also Fig.13 and Fig.14 , Fig.13 A schematic diagram showing a contour line that is unbounded downwards according to some embodiments of the present disclosure; Fig.14 Schematic diagrams of unbounded contour lines downward according to other embodiments of the present disclosure are shown. Fig.13 and Fig.14 It can be seen that the contour line has no boundary downward. In this case, the position of P1 can be obtained by limiting the maximum span of Y to be searched downward. For example, determining the first feature area based on the first feature point may include: extending a predetermined distance from the first feature point along the first direction to the first auxiliary point, the predetermined distance may be manually specified, and an appropriate distance may be specified according to actual conditions; selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point, so that the second auxiliary point intersects with the continuous graphic contour at the third auxiliary point and the fourth auxiliary point in the direction perpendicular to the line segment; and determining the first feature area based on the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point, the first feature area includes the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point.

[0058] Select the feature area based on the above four points, and correspond it to the original image to get the original feature graph, recorded as R0, such as Fig. 9 shown.

[0059] See below Fig. 9 , Fig. 9 A schematic diagram showing a contour map and a first feature region in an original image according to some embodiments of the present disclosure is shown. Fig. 9 As can be seen in FIG. 1 , a rectangular first feature area 902 is shown, such as Fig. 9 As shown on the right side of , the first feature area 902 circles a part of the figure on the figure, marked as a “feature figure”, and the outline of the feature figure can be called the second figure outline.

[0060] Back to Figure 2 At box 206, feature matching is performed in the image using the first feature region to determine a second feature region, so that the first graphic contour matches the second graphic contour in the second feature region.

[0061] The image matching method can use the common methods already available in various image libraries, such as Brute-Force matching and Fast-Library for Approximate Nearest Neighbors (FLANN) matching.

[0062] In some embodiments, performing feature matching in the image using the first feature region includes using a brute force matching method, which mainly finds the best feature point matching position in an exhaustive manner.

[0063] In some embodiments, performing feature matching in the image using the first feature region may include using fast approximate neighbor matching, which can be used to efficiently find nearest neighbors in a data set.

[0064] By using the obtained feature graph to perform graph matching in the original image, the positions of more feature graphs can be obtained, which can be recorded as R1, R2, ... respectively.

[0065] See also Fig.10 , Fig.10 Schematic diagrams of various characteristic regions according to some embodiments of the present disclosure are shown. Fig.10 As shown, each of the characteristic regions includes a characteristic graphic, Fig.10 A total of five characteristic patterns are shown, which are labeled as R0, R1, R2, R3 and R4. If R0 is called the first characteristic pattern, one or more of R1 to R4 can be called the second characteristic pattern, and its outline is the second pattern outline.

[0066] The above embodiments are combined with the attached Figure 3 To Attachment Fig.10 As mentioned above, the embodiments of the present disclosure are not limited thereto. Fig.15 As shown, the detection line 306 extends in a direction parallel to the bottom edge of the image. During the detection process, the detection line moves perpendicular to the bottom edge. Fig.16 and Fig.17 The first feature area is determined in the manner shown in .

[0067] See also Fig.16 , Fig.16 A schematic diagram showing two points in an image used to determine a feature area in some embodiments of the present disclosure is shown. Fig.16 As shown, the first feature point P0 and the first auxiliary point P1 are shown.

[0068] Fig.17 A schematic diagram of four points used to determine a feature area in an image of some embodiments of the present disclosure is shown. The four points include a first feature point P0, a first auxiliary point P1, a third auxiliary point P2, and a fourth auxiliary point P3. Thus, an area surrounding the first feature point P0, the first auxiliary point P1, the third auxiliary point P2, and the fourth auxiliary point P3 can be determined as the first feature area. The size of the first feature area can be selected according to actual needs, and the present disclosure does not limit this.

[0069] Figures 18 to 24 The following is a schematic diagram of a process for processing a TEM image according to an example of the present disclosure. The specific process may adopt the method of the above embodiment.

[0070] Fig.18 A schematic diagram showing a TEM image to be processed according to another embodiment of the present disclosure is shown. Fig.19 Shows Fig.18 A schematic diagram of the outline of the image to be processed is shown.

[0071] Fig. 20 The first direction is shown Fig.19 Schematic diagram of detecting feature points in continuous contours. Fig. 20 As shown, P0 and P1 are determined.

[0072] Fig.21 A schematic diagram of four points for determining a first feature domain in some embodiments of the present disclosure is shown, wherein a first feature point P1, a second auxiliary point P1, a third auxiliary point P2 and a fourth auxiliary point P3 are shown.

[0073] At block 208 , an angle by which the image needs to be rotated is determined based on the first graphic outline and the second graphic outline.

[0074] In some embodiments, determining the angle at which the image needs to be rotated based on the first graphic contour and the second graphic contour may include: determining a fitting straight line of the midpoints of the first feature area and the second feature area; determining the angle between the fitting straight line and a horizontal line in the plane where the image is located or a vertical line perpendicular to the horizontal line; and determining the angle as the angle at which the image needs to be rotated.

[0075] See also Fig.11 , Fig.11 A schematic diagram showing the slope fitting of characteristic graphs of some embodiments of the present disclosure is shown.

[0076] In some embodiments, the slopes between the points can be calculated based on the center coordinates of R0, R1, R2, ..., R5, and the angle of inclination of the figure in the image can be converted back. The original image can be leveled based on the converted angle.

[0077] Fig.11 The small dots shown in the figure represent the center coordinates of R0, R1, R2, ..., R5 of the figure contour in each feature area. The fitting curve (specifically a straight line) is obtained by fitting, and its slope can be determined, thereby determining the inclination angle of the figure. Specifically, the slope of the straight line is the inclination angle of the figure. In addition Fig.11 The expression of the fitting curve 1102 is shown in FIG. 1102: y=0.9776x+0.8721, where x is the horizontal coordinate, y is the vertical coordinate, and R 2 =0.9954. The R-squared value is an indicator of the degree of fit of the trend line. Its value can reflect the degree of fit between the estimated value of the trend line and the corresponding actual data. The higher the degree of fit, the higher the reliability of the trend line. The R-squared value is a value between 0 and 1. When the R-squared value of the trend line is equal to 1 or close to 1, its reliability is the highest, otherwise the reliability is low. Ideally, if the points found are all correct, they should all be on a straight line. The better the linearity, that is, the R 2 The larger it is, the more accurate the description.

[0078] In the above embodiment, the slope is determined by solving the expression of the fitting curve according to the parameters in the expression. For example, the expression shown is a one-variable equation, and its slope can be determined, and the slope is the inclination angle.

[0079] By fitting, the influence of abnormal points on the curve can be reduced, so that the inclination angle of the graph can be accurately determined.

[0080] It should be understood that the embodiments of the present disclosure are not limited thereto, but may be variously modified.

[0081] For example, the slope of the line between the midpoints of two characteristic patterns or the midpoints of multiple characteristic patterns can be used as the tilt angle as the basis for leveling. For example, the slope of the line between the midpoint of the first characteristic pattern and the midpoint of the last characteristic pattern can be used as the tilt angle.

[0082] At block 210, the image is rotated within the plane of the image based on an angle so that the section line extends in a direction that is parallel or perpendicular to a horizontal line within the plane of the image.

[0083] See below Fig.12 , Fig.12 FIG. 4 is a schematic diagram showing a leveled image according to some embodiments of the present disclosure. Fig.12 As shown in FIG. 1 , in the leveled graph, the parallel section line 304 is parallel to the upper and lower edges of the image, that is, the section line extends in a direction parallel to the horizontal line in the plane where the image is located. According to actual conditions, it is possible that after leveling, the section line extends in a direction perpendicular to the horizontal line in the plane where the image is located.

[0084] See below Fig. 22 , Fig. 22 Shows Fig.18 Schematic diagram of a feature graph corresponding to the first feature region of the original image shown in . The graph feature R0 is shown.

[0085] Fig.23 Shows Fig.18 Schematic diagram of graphic contours in various feature areas of the original image shown in . Five graphic contours R1, R2, R3, R4 and R5 are shown.

[0086] Fig.24 Shows Fig.23 Schematic diagram of the slope fit of the profile shown in . Figure 3 As shown, the expression of the obtained fitting curve (essentially a straight line here) is: y=0.89+3.9, where R 2 =0.9936.

[0087] Some embodiments of the present disclosure provide methods for image processing. It should be noted that the examples given in the above embodiments are only for illustrating the solutions of the embodiments of the present disclosure, and are not intended to limit the solutions of the present disclosure.

[0088] The method of some embodiments of the present disclosure finds the characteristic graphics by automatically identifying the characteristic points of the graphics in the picture, without manual intervention, avoiding the error caused by human adjustment. The method is universal for different repeated graphics in TEM pictures and can be extended to the leveling processing of TEM pictures of different products. In addition, the method can identify all the characteristic graphics in the picture, and the credibility of the calculation results can be verified by the slope between the characteristic graphics, that is, the slope between the characteristic graphics at different positions obtained by matching should be within the error range. If it exceeds the error range, the result may be abnormal, so the method has the ability to self-judge the leveling result.

[0089] In short, the technical solution disclosed in the present invention can effectively solve the inconvenience caused by manual leveling, and improve the leveling efficiency and leveling accuracy. Specifically, through automatic leveling, the consistency and stability of the leveling results can be guaranteed, and human errors can be reduced or avoided.

[0090] It should be understood that the embodiments shown in the drawings are only for schematically illustrating the solutions of some embodiments of the present disclosure and are not intended to limit the present disclosure. The embodiments of the present disclosure may also have various other forms.

[0091] An electronic device is also disclosed in an embodiment of the present disclosure. The electronic device includes: a processor; and a memory coupled to the processor, the memory having instructions stored therein, and when the instructions are executed by the processor, the device performs an action, the action including: determining a first feature point of a graphic contour in an image, wherein the graphic contour includes a profile line of a parallel layer, and the first feature point is a point with an extreme value within a predetermined range; determining a first feature area based on the first feature point, wherein the first feature area surrounds the first graphic contour including the first feature point; performing feature matching in the image using the first feature area to determine a second feature area, so that the first graphic contour matches the second graphic contour in the second feature area; determining an angle to be rotated for the image based on the first graphic contour and the second graphic contour; and rotating the image in the plane of the image based on the angle so that the profile line extends in a direction parallel to or perpendicular to a horizontal line in the plane where the image is located.

[0092] An embodiment of the present disclosure further discloses a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the image processing method according to the embodiment of the present disclosure is implemented.

[0093] Fig.25A schematic block diagram of an electronic device according to some exemplary embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0094] like Fig.25 As shown, the device 2500 includes a CPU 2501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 2502 or a computer program loaded from a storage unit 2508 into a random access memory (RAM) 2503. In the RAM 2503, various programs and data required for the operation of the device 2500 can also be stored. The CPU 2501, the ROM 2502, and the RAM 2503 are connected to each other via a bus 2504. An input / output (I / O) interface 2505 is also connected to the bus 2504.

[0095] Multiple components in the device 2500 are connected to the I / O interface 2505, including: an input unit 2506, such as a keyboard, a mouse, etc.; an output unit 2507, such as various types of displays, speakers, etc.; a storage unit 2508, such as a disk, an optical disk, etc.; and a communication unit 2509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 2509 allows the device 2500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0096] The various processes and processing described above, such as method 200, can be executed by CPU 2501. For example, in some embodiments, method 200 can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 2508. In some embodiments, part or all of the computer program can be loaded and / or installed on device 2500 via ROM 2502 and / or communication unit 2509. When the computer program is loaded into RAM 2503 and executed by CPU 2501, one or more steps in method 200 described above can be performed.

[0097] The scheme according to the embodiment of the present disclosure may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium on which computer-readable program instructions for executing various aspects of the present disclosure are loaded. The computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. The computer-readable program instructions may be downloaded from the computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network and / or a wireless network.

[0098] Various embodiments of the present disclosure have been described above. The above description is exemplary and is only an optional embodiment of the present disclosure. It is not exhaustive and is not intended to limit the present disclosure. Although the claims in this application have been formulated for specific combinations of features, it should be understood that the scope of the present disclosure also includes any novel feature or any novel combination of features disclosed herein, whether or not it relates to the same scheme in any claim currently claimed for protection. It should be noted that new claims may be formulated into these features and / or combinations of these features during the examination of this application or in any further application derived therefrom.

[0099] The terms used in this article are selected to best explain the principles of each embodiment, practical application or technical improvement in the market, or to enable other ordinary technicians in the field to understand the embodiments disclosed herein. For those skilled in the art, the present disclosure may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. An image processing method, comprising: Determine a first feature point of a graphic contour in an image, wherein the first feature point is a point having an extreme value within a predetermined range in the graphic contour, and the graphic contour in the image includes a profile line contour of parallel layers; Determine a first feature area based on the first feature point, wherein the first feature area surrounds a first graphic outline including the first feature point; Performing feature matching in the image using the first feature area to determine a second feature area, so that the first graphic outline matches the second graphic outline within the second feature area; Determining an angle by which the image needs to be rotated based on the first graphic outline and the second graphic outline; as well as The image is rotated in the plane of the image based on the angle so that the section line extends in a direction parallel to or perpendicular to a horizontal line in the plane where the image is located.

2. The method according to claim 1, wherein determining the first feature point of the contour of the figure in the image comprises: Moving a detection line parallel to a first boundary of the image in a direction perpendicular to the first boundary; respectively determining the coordinates of the intersection points on the graphic contour intersecting with the detection line; as well as An intersection point whose coordinates have extreme values ​​in a predetermined range on the graphic contour in a direction along the detection line is taken as the first feature point.

3. The method according to claim 1, wherein determining a first feature area based on the first feature point comprises: Extending from the first feature point along a first direction to a first auxiliary point on a second contour opposite to the first graphic contour; Selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point so that the second auxiliary point intersects with the contour of the figure at a third auxiliary point and a fourth auxiliary point in a direction perpendicular to the line segment; as well as The first feature area is determined based on the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point.

4. The method according to claim 1, wherein determining a first feature area based on the first feature point comprises: Extending a predetermined distance from the first feature point along a first direction to a first auxiliary point; Selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point so that the second auxiliary point intersects with the contour of the figure at a third auxiliary point and a fourth auxiliary point in a direction perpendicular to the line segment; as well as The first feature area is determined based on the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point.

5. The method according to claim 3 or 4, wherein determining the first feature area based on the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point comprises: An area surrounding the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point is selected as the first feature area.

6. The method according to claim 3 or 4, wherein selecting a second auxiliary point between a line segment connecting the first feature point and the first auxiliary point comprises: Selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point so that the distance between the third auxiliary point and the fourth auxiliary point is maximized; or The midpoint of the line segment connecting the first feature point and the first auxiliary point is selected as the second auxiliary point.

7. The method of claim 1, wherein feature matching is performed in the image using one of the following methods: Brute force matching; and Fast approximate proximity matching.

8. The method according to claim 1, wherein determining the angle by which the image needs to be rotated based on the first graphic outline and the second graphic outline comprises: Determine a fitting straight line of the midpoint of each of the first characteristic region and the second characteristic region; determining an angle between the fitting straight line and the horizontal line in the plane where the image is located or a vertical line perpendicular to the horizontal line; and The included angle is determined as the angle at which the image needs to be rotated.

9. An electronic device, comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, the instructions causing the device to perform actions when executed by the processor, the actions comprising: Determine a first feature point of a graphic contour in an image, wherein the first feature point is a point having an extreme value within a predetermined range in the graphic contour, and the graphic contour in the image includes a profile line contour of parallel layers; Determine a first feature area based on the first feature point, wherein the first feature area surrounds a first graphic outline including the first feature point; Performing feature matching in the image using the first feature area to determine a second feature area, so that the first graphic outline matches the second graphic outline within the second feature area; Determining an angle by which the image needs to be rotated based on the first graphic outline and the second graphic outline; and The image is rotated in the plane of the image based on the angle so that the section line extends in a direction parallel to or perpendicular to a horizontal line in the plane where the image is located.

10. A computer-readable storage medium having machine-executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 8.

Images