Image Processing Method, Electronic Device, and Computer-Readable Storage Medium

Abstract

Embodiments of the present disclosure relate to an image processing method, an electronic device, and a computer-readable storage medium. The method includes: determining a first feature point of a graphic contour in an image, where the first feature point is a point with an extreme value within a predetermined range in the graphic contour, and the graphic contour in the image includes a cross-hatch contour of parallel layers; determining a first feature region based on the first feature point, where the first feature region surrounds a first graphic contour including the first feature point; performing feature matching in the image using the first feature region to determine a second feature region such that the first graphic contour matches a second graphic contour within the second feature region; determining an angle by which the image needs to be rotated based on the first graphic contour and the second graphic contour; and rotating the image within the plane of the image based on the angle so that the cross-hatch extends in a direction parallel to or perpendicular to a horizontal line within the plane of the image. The technical solution of the present disclosure can improve the leveling efficiency and accuracy.

Description

Technical Field

[0001] Embodiments of the present disclosure mainly relate to integrated circuits, and more specifically, to an image processing method, an electronic device, and a computer-readable storage medium. Background Art

[0002] A transmission electron microscope (TEM) is an electron optical instrument with high resolution and high magnification characteristics. The TEM can be used as a failure analysis tool in semiconductor manufacturing.

[0003] During the manufacturing process of semiconductor chips, it is sometimes necessary to perform TEM measurement on the chips during the process. During the measurement process, due to reasons such as sample preparation conditions and the angular rotation of the TEM shooting platform, the sample may not reach the optimal horizontal angle. As a result, the patterns or films to be measured in the captured TEM images will be in an inclined state, which is not conducive to subsequent pattern measurement. Therefore, in most cases, it is necessary to manually adjust the image to be horizontal according to the state of the patterns in the figure before measurement. During the manual leveling process, the results will vary due to factors such as the operator's habits and state.

[0004] The disadvantage of the traditional manual leveling scheme is that it is time-consuming, and at the same time, the accuracy of leveling is greatly affected by subjective factors. Summary of the Invention

[0005] According to an exemplary embodiment of the present disclosure, an image processing solution is provided to at least partially overcome the above or other potential defects.

[0006] According to one aspect of the present disclosure, an image processing method is provided. The method includes: determining a first feature point of a pattern contour in an image, where the first feature point is a point with an extreme value within a predetermined range in the pattern contour, and the pattern contour in the image includes a profile line contour of parallel layers; determining a first feature region based on the first feature point, where the first feature region surrounds a first pattern contour including the first feature point; performing feature matching in the image using the first feature region to determine a second feature region such that the first pattern contour matches a second pattern contour within the second feature region; determining an angle by which the image needs to be rotated based on the first pattern contour and the second pattern contour; and rotating the image within the plane of the image based on the angle so that the profile lines extend in a direction parallel to or perpendicular to the horizontal line within the plane of the image. The technical solution of the present disclosure can effectively solve the inconvenience brought by the manual leveling method, and improve the leveling efficiency and leveling accuracy.

[0007] In a second aspect of the present disclosure, an electronic device is provided. The electronic device includes a processor; and a memory coupled to the processor, the memory having instructions stored therein, the instructions, when executed by the processor, causing the device to perform operations, the operations including: determining a first feature point of a graphic contour in an image, where the first feature point is a point with an extreme value within a predetermined range in the graphic contour, and the graphic contour in the image includes a cross-hatch contour of parallel layers; determining a first feature region based on the first feature point, where the first feature region surrounds a first graphic contour including the first feature point; performing feature matching in the image using the first feature region to determine a second feature region such that the first graphic contour matches a second graphic contour within the second feature region; determining an angle by which the image needs to be rotated based on the first graphic contour and the second graphic contour; and rotating the image within the plane of the image based on the angle so that the cross-hatch extends in a direction parallel to or perpendicular to a horizontal line within the plane of the image.

[0008] In some embodiments, determining the first feature point of the graphic contour in the image includes: moving a detection line parallel to a first boundary of the image in a direction perpendicular to the first boundary; respectively determining the coordinates of the intersection points of the graphic contour with the detection line; and taking, as the first feature point, the intersection point on the graphic contour whose coordinate has an extreme value within a predetermined range along the direction of the detection line.

[0009] In some embodiments, determining the first feature region based on the first feature point includes: extending from the first feature point in a first direction to a first auxiliary point on a second contour opposite to the first graphic contour; selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point such that the direction perpendicular to the line segment through the second auxiliary point intersects the graphic contour of the figure at a third auxiliary point and a fourth auxiliary point respectively; and determining the first feature region based on the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point.

[0010] In some embodiments, determining the first feature region based on the first feature point includes: extending a predetermined distance from the first feature point in a first direction to a first auxiliary point; selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point such that the direction perpendicular to the line segment through the second auxiliary point intersects the graphic contour of the figure at a third auxiliary point and a fourth auxiliary point respectively; and determining the first feature region based on the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point.

[0011] In some embodiments, determining the first feature region based on the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point includes: selecting the region surrounding the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point as the first feature region.

[0012] In some embodiments, selecting a second auxiliary point between the line segments connecting the first feature point and the first auxiliary point includes: selecting a second auxiliary point between the line segments connecting the first feature point and the first auxiliary point such that the distance between the third auxiliary point and the fourth auxiliary point is maximized; or selecting the midpoint of the line segment connecting the first feature point and the first auxiliary point as the second auxiliary point.

[0013] In some embodiments, one of the following methods is used to perform feature matching in an image: brute-force matching; and fast approximate nearest neighbor matching.

[0014] In some embodiments, determining the angle by which an image needs to be rotated based on the first graphic contour and the second graphic contour includes: determining the fitting line of the midpoints of the first feature region and the second feature region respectively; determining the angle between the fitting line and the horizontal line or the vertical line perpendicular to the horizontal line in the plane where the image is located; and determining the angle as the angle by which the image needs to be rotated.

[0015] In a third aspect of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, which when executed by a processor implements the method according to the first aspect of the present disclosure.

[0016] It will be understood from the following description that the technical solution of the present disclosure can effectively solve the inconvenience brought by the manual leveling method, and improve the leveling efficiency and leveling accuracy.

[0017] The summary is provided to introduce a selection of concepts in a simplified form, which will be further described in the detailed description below. The summary is not intended to identify the key features or main features of the present disclosure, nor is it intended to limit the scope of the present disclosure. Brief Description of the Drawings

[0018] Figure 1 A schematic diagram showing an exemplary environment in which embodiments of the present disclosure can be implemented;

[0019] Figure 2 A flowchart showing an image processing method according to some embodiments of the present disclosure;

[0020] Figure 3 A schematic diagram showing a TEM image to be processed according to an embodiment of the present disclosure;

[0021] Figure 4 Shows Figure 3 A schematic contour diagram of the image to be processed shown;

[0022] Figure 5 A schematic diagram showing the detection of feature points in continuous contours in an image in a first direction according to some embodiments of the present disclosure;

[0023] Figure 6 A schematic diagram showing feature points in each consecutive contour within a given range in an image of some embodiments of the present disclosure;

[0024] Figure 7 A schematic diagram showing two points for determining a first feature region in an image of some embodiments of the present disclosure;

[0025] Figure 8 A schematic diagram showing four points for determining a first feature region in an image of some embodiments of the present disclosure;

[0026] Figure 9 A schematic diagram showing a contour map and a first feature region in an original image according to some embodiments of the present disclosure;

[0027] Figure 10 A schematic diagram showing respective feature regions according to some embodiments of the present disclosure;

[0028] Figure 11 A schematic diagram showing the slope fitting of a feature graph in some embodiments of the present disclosure;

[0029] Figure 12 A schematic diagram showing a leveled image according to some embodiments of the present disclosure;

[0030] Figure 13 A schematic diagram showing a downward unbounded contour line according to some embodiments of the present disclosure;

[0031] Figure 14 A schematic diagram showing a downward unbounded contour line according to some other embodiments of the present disclosure;

[0032] Figure 15 A schematic diagram showing the detection of feature points in consecutive contours along a second direction in an image according to some embodiments of the present disclosure;

[0033] Figure 16 A schematic diagram showing two points for determining a feature region in an image of some embodiments of the present disclosure;

[0034] Figure 17 A schematic diagram showing four points for determining a feature region in an image of some embodiments of the present disclosure;

[0035] Figure 18 A schematic diagram showing a TEM image to be processed according to another embodiment of the present disclosure;

[0036] Figure 19 Shows Figure 18 A schematic contour diagram of the image to be processed shown;

[0037] Figure 20Shows the detection of feature points in the continuous contour shown in Figure 19 along the first direction;

[0038] Figure 21 Shows a schematic diagram of four points for determining a first feature domain in some embodiments of the present disclosure;

[0039] Figure 22 Shows Figure 18 a schematic diagram of a graphic contour within a first feature region of the original image shown in

[0040] Figure 23 Shows Figure 18 schematic diagrams of graphic contours within respective feature regions of the original image shown in

[0041] Figure 24 Shows Figure 23 a schematic diagram of slope fitting of the graphic contour shown in ; and

[0042] Figure 25 Shows a block diagram of a computing device capable of implementing multiple embodiments of the present disclosure.

[0043] In each of the drawings, the same or corresponding reference numerals denote the same or corresponding parts. Detailed Description of the Embodiments

[0044] The principles of the present disclosure will be described below with reference to various exemplary embodiments shown in the drawings. It should be understood that the description of these embodiments is only for enabling those skilled in the art to better understand and further implement the present disclosure, and is not intended to limit the scope of the present disclosure in any way. It should be noted that, where feasible, similar or identical reference numerals may be used in the figures, and similar or identical reference numerals may represent similar or identical functions. Those skilled in the art will readily recognize that alternative embodiments of the structures and methods described herein may be employed without departing from the principles of the invention described herein.

[0045] As used herein, the term "comprising" and its variations mean open inclusion, i.e., "including but not limited to". Unless specifically stated otherwise, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "an exemplary embodiment" and "an embodiment" mean "at least one exemplary embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects.

[0046] As mentioned before, before TEM imaging, the sample may tilt for various reasons. The TEM picture (or image) needs to capture the depth and width of the pattern. If the picture is not flat, it will be difficult to measure and may result in large deviations. Specifically, if the picture is tilted, the corresponding pattern is tilted. When measuring the depth and width, the results when the pattern is horizontal need to be obtained. That is to say, the measured depth and width of the sample when it is horizontal will be used as the final result. If the picture is tilted, when measuring this result, the same tilt needs to be maintained to obtain accurate results. This is difficult to operate and the accuracy cannot be guaranteed. After adjusting to be horizontal, the difficulty of measuring the depth and width will be reduced. Usually, the tilt angle is not too large, and the picture can be adjusted so that it is flat relative to the observer. In fact, in other words, leveling is to straighten the picture. After leveling, the width can be measured along the horizontal direction of the pattern, and the depth can be measured in the vertical direction. Measuring along the correct structural direction will improve the accuracy and also conform to the operation habits of engineers.

[0047] The traditional solution is to level the picture by the operator during the process. The result of leveling is highly related to the operator. Because there are often differences in human judgment, and different people have different understandings of the picture. For example, everyone has different sensations of what angle is horizontal, and the adjustment will vary according to different operators, and the operation habits are different, which will lead to different results obtained by different operators for the same picture. Therefore, it is necessary to understand the structure of the sample in the picture to determine the direction of adjustment.

[0048] Currently, in the wafer fab (Fab), manual leveling by humans is mainly adopted, which is greatly affected by human factors. There may be some semi-automatic methods that require manual setting of templates and then matching to achieve semi-automatic leveling. This method still requires manual participation in setting templates, is not fully automatic leveling, and the template making is still affected by human factors and will be different.

[0049] In view of this, the present disclosure provides an improved solution.

[0050] Embodiments of the present disclosure provide an improved image processing method. The method includes: determining a first feature point of a graphic contour in an image, where the first feature point is a point with an extreme value within a predetermined range in the graphic contour, and the graphic contour in the image includes a cross-sectional line contour of parallel layers; determining a first feature region based on the first feature point, where the first feature region surrounds a first graphic contour including the first feature point; performing feature matching in the image using the first feature region to determine a second feature region such that the first graphic contour matches a second graphic contour within the second feature region; determining an angle by which the image needs to be rotated based on the first graphic contour and the second graphic contour; and rotating the image within the plane of the image based on the angle so that the cross-sectional lines extend in a direction parallel to or perpendicular to a horizontal line within the plane of the image. The technical solution of the present disclosure can effectively solve the inconvenience brought by the manual leveling method, and improve the leveling efficiency and leveling accuracy.

[0051] Embodiments of the present disclosure will be specifically described below with reference to the accompanying drawings.

[0052] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. As Figure 1 shown, the example environment 100 includes a computing device 110 and a client 120.

[0053] In some embodiments, the computing device 110 can interact with the client 120. For example, the computing device 110 can receive an input message from the client 120 and output a feedback message to the client 120. In some embodiments, the input message from the client 120 can be image data, such as a TEM picture. The computing device 110 can perform corresponding mathematical operations on the image data and output the corresponding operation results to the client 120.

[0054] In some embodiments, the computing device 110 can include, but is not limited to, a personal computer, a server computer, a handheld or laptop device, a mobile device (such as a mobile phone, a personal digital assistant PDA, a media player, etc.), a consumer electronic product, a minicomputer, a mainframe computer, cloud computing resources, etc.

[0055] It should be understood that describing the structure and function of the example environment 100 only for exemplary purposes is not intended to limit the scope of the subject matter described herein. The subject matter described herein can be implemented in different structures and / or functions. This environment is merely illustrative and not used to limit the application environment of the embodiments of the present disclosure.

[0056] To more clearly explain the principle of the solution of the present disclosure, the following will be described in more detail with reference to Figure 4 .

[0057] Figure 2The flowchart of an image processing method 200 according to some embodiments of the present disclosure is shown.

[0058] At block 202, a first feature point of the graphic contour in the image is determined, where the first feature point is a point with an extreme value within a predetermined range in the graphic contour, and the graphic contour in the image includes a cross - hatch contour of parallel layers.

[0059] A TEM image (or picture) is usually a photograph of a slice. The photograph includes cross - sections of multiple layers. Each layer is usually parallel. The dimensions of relevant characteristic parameters, such as depth, width, etc., can be determined through a TEM picture. After taking a picture of the slice, general software in the industry can be used to measure the taken image to determine the parameters of the graphics therein. The specific measurement process is not described in detail in the present disclosure.

[0060] In some embodiments, the solution of the embodiments of the present disclosure is particularly suitable for processing Figure 3 the shown TEM image.

[0061] See Figure 3 and Figure 4 . Figure 3 The schematic diagram of the TEM image to be processed according to the embodiments of the present disclosure is shown; Figure 4 shows Figure 3 the schematic diagram of the contour of the image to be processed shown. As Figure 3 and Figure 4 shown, the graphic contour 302 is shown therein.

[0062] There are many ways to obtain the contour. For example, common algorithms such as binary edge detection can be used for contour extraction. Specifically, common algorithms include the Sobel operator, the Prewitt operator, the Canny edge detection algorithm, etc. The Sobel operator is an important processing method in the field of computer vision and is commonly used for edge detection. The Prewitt operator is an edge detection of a first - order differential operator. The Canny edge detection algorithm is a multi - level edge detection algorithm. For example, in some embodiments, through any of the above algorithms, the following method can be used to determine the contour line in the picture: First, the edge points in the image can be determined; then, the continuous edge points can be determined from the edge points; finally, the continuous edge points can be determined as the graphic contour.

[0063] Figure 3 and Figure 4The 304 shown in [the figure] may represent the parallel cross - hatched profile in the above - mentioned layer. Among them, 300 represents the substrate, such as a silicon substrate. In fact, the TEM picture features a silicon - based substrate at the bottom, and various structures grown during the chip manufacturing process on it. Therefore, the upper part of the picture is the pattern direction, and the upper - edge contour of the pattern is the uppermost boundary of the pattern. Usually, the patterns in the TEM picture mainly refer to the patterns of the same layer (layer) during the chip manufacturing process, and this layer can also be regarded as a thin film (such as various oxide thin films, metal thin films, etc.). For example, there will be different AA patterns in the Active Area Layer (AA for short). However, since the TEM picture is a cross - section of the chip, it will contain multiple layers or films.

[0064] In some embodiments, a detection line parallel to the first boundary of the image can be moved in a direction perpendicular to the first boundary; the coordinates of the intersection points on the pattern contour that intersect the detection line can be determined respectively; and the intersection points with extreme values of the coordinates in a predetermined range along the direction of the detection line on the pattern contour are taken as the first feature points.

[0065] In some embodiments, determining the first feature points of the pattern contour in the image may include: moving a detection line parallel to the first boundary of the image in a direction perpendicular to the first boundary; determining the coordinates of the points on the pattern contour of the pattern that intersect the detection line respectively; and taking the points with the maximum or minimum coordinate values in a predetermined range along the direction of the detection line on the pattern contour as the first feature points.

[0066] In some embodiments, a detection line extending in the Y - direction (the longitudinal direction in the figure) can be used to detect the position of the highest point of the continuous contour from left to right to obtain the feature points.

[0067] In some embodiments, the definition of the highest point within a given range of the continuous contour is as follows: Assume that the point where the detection line in the Y - direction intersects the pattern contour is P0(x0, y0), then the points on the contour within the given neighborhood δ can be expressed as: P(x, y), where |x - x0| ≤ δ; when for all P points, y0 - y ≥ 0 is satisfied, then the P0 point is the highest point of the contour within the given range. At this time, it can be said that the coordinate in the Y - direction has the maximum coordinate value. In other words, the coordinates of the intersection points along the direction of the detection line have extreme values (i.e., the coordinate values in the Y - direction have extreme values). The setting of δ can be given an empirical value according to the number of pixels. For example, it is recommended to be less than 10.

[0068] The following is combined with Figure 5 and Figure 6 for description. Figure 5 The schematic diagram shows the detection of feature points in the continuous contour along the first direction in the image of some embodiments of the present disclosure. Figure 5The detection line 306 is shown, and the detection line 306 extends along a first direction (longitudinal direction, also referred to as the Y direction). During the detection process, the detection line 306 moves to the right parallel to the left edge of the image. Figure 6 A schematic diagram showing the feature points in each continuous contour within a given range in an image of some embodiments of the present disclosure. For Figure 5 and Figure 6 the following 5 highest points exist in the graphic contour in, and these 5 points are denoted as the feature points of the graphic in this figure, that is Figure 6 1, 2, 3, 4, and 5 shown in, representing the respective identified feature points, that is, the positions of the highest points of the aforementioned continuous contours. For example, the feature point represented by the number 1 can be referred to as the first feature point. Figure 6 5 feature points are shown in, and it should be understood that in some embodiments of the present disclosure, it is not necessary to identify all these feature points, but only one of them can be identified.

[0069] Figure 5 and Figure 6 it is shown that the detection line 306 extending along the Y direction moves to the right parallel to the left edge of the image to detect the first feature point. The embodiments of the present disclosure are not limited thereto, and detection lines of different colors along other directions can also be used for detection. Refer to Figure 15 , Figure 15 a schematic diagram showing the detection of feature points in continuous contours along a second direction in an image according to some embodiments of the present disclosure. As Figure 15 shown, wherein the detection line 306 extends along a direction parallel to the bottom edge of the image, and during the detection process, the detection line moves in a direction perpendicular to the bottom edge. The specific detection process is substantially the same as that introduced in the previous embodiments. During this detection process, the point with the minimum value of the abscissa among the intersection points of the detection line and the contour line can be determined, and at this time, the coordinate in the X direction has the minimum coordinate value (or the coordinate has an extreme value in the X direction). Or rotate the picture 90 degrees to the right, and it can also be said that the detected point at this time is the highest point.

[0070] At block 204, a first feature region is determined based on the first feature point, wherein the first feature region surrounds a first graphic contour including the first feature point.

[0071] In some embodiments, the first feature region can be determined as follows: extending from the first feature point along the first direction to a first auxiliary point on a second contour opposite to the first graphic contour; selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point such that the directions perpendicular to the line segment through the second auxiliary point intersect the graphic contour of the graphic at a third auxiliary point and a fourth auxiliary point respectively; and determining the first feature region based on the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point.

[0072] In some embodiments, a region surrounding the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point may be selected as the first feature region.

[0073] The following is a description with reference to Figure 7 for illustration. Figure 7 FIG. shows a schematic diagram of two points for determining a first feature region in an image according to some embodiments of the present disclosure.

[0074] As previously mentioned, 5 feature points have been found. The leftmost feature point 1 can be taken and the second point where the detection line intersects the contour can be found by moving down along the Y direction, denoted as P1(x1, y1). As Figure 7 shown, where the detection line 306 moves to the right and intersects the contour line 302 at point P0 (i.e., the feature point represented by the number 1 before) and P1.

[0075] Refer to Figure 8 , Figure 8 FIG. shows a schematic diagram of four points for determining a first feature region in an image according to some embodiments of the present disclosure. A second auxiliary point (not shown in the figure) can be selected on the line connecting P0 and P1, such that the third auxiliary point P2 and the fourth auxiliary point P3 are respectively intersected with the continuous graphic contour in the direction perpendicular to the line passing through the second auxiliary point.

[0076] In some embodiments, the midpoint of the line segment connecting the first feature point and the first auxiliary point may be selected as the second auxiliary point.

[0077] In some embodiments, selecting the second auxiliary point between the line segments connecting the first feature point and the first auxiliary point may include: selecting the second auxiliary point such that the distance between the third auxiliary point and the fourth auxiliary point is the largest.

[0078] For example, after taking the midpoint of P0 and P1, the left and right intersection points P2 and P3 of the contour are found along the X direction. In a TEM image, multiple similar structures are generally cut horizontally, so the shapes are basically similar. That is to say, due to the repeatability of all the graphics in the figure, there must be left and right contour boundaries. When the left contour point P2 reaches the left boundary of the image, another point can be selected to repeat the selection of P1, P2, and P3. This is aimed at trying to enclose relatively more feature graphics within the range of the points P0, P1, P2, and P3, and avoiding mis-matching to the wrong position due to too small a selection range.

[0079] In some embodiments, a first feature region may be determined based on a first feature point, a first auxiliary point, a third auxiliary point, and a fourth auxiliary point. For example, a rectangular region is selected, and this rectangular region contains the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point. This rectangular region is the first feature region mentioned above. As can be seen from the figure, the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point are respectively located on the sides of the rectangular region.

[0080] In some embodiments, there may be a situation where the graphic contour is unbounded downward, and then the second contour intersection point cannot be found. In this case, it is necessary to limit the maximum span of Y for downward search to obtain the position of P1.

[0081] See Figure 13 and Figure 14 , Figure 13 which shows a schematic diagram of a downward-unbounded contour line according to some embodiments of the present disclosure; Figure 14 which shows a schematic diagram of a downward-unbounded contour line according to some other embodiments of the present disclosure. As can be seen from Figure 13 and Figure 14 , the contour line is unbounded downward therein. This situation can obtain the position of P1 by limiting the maximum span of Y for downward search. For example, determining the first feature region based on the first feature point may include: extending a predetermined distance from the first feature point in a first direction to the first auxiliary point, and the predetermined distance can be specified manually, and an appropriate distance can be specified according to the actual situation specifically; selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point, such that the direction perpendicular to the line segment passing through the second auxiliary point intersects the continuous graphic contour at the third auxiliary point and the fourth auxiliary point respectively; and determining the first feature region based on the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point, and the first feature region contains the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point.

[0082] Select the feature region bounded by the above four points, and the original feature graphic, denoted as R0, can be obtained corresponding to the original figure, as shown in Figure 9 .

[0083] Next, see Figure 9 , Figure 9 which shows a schematic diagram of the contour map and the first feature region in the original image according to some embodiments of the present disclosure. As can be seen from Figure 9 , the rectangular first feature region 902 is shown therein. As shown on the right side of Figure 9 , the first feature region 902 encloses a partial graphic on the figure, marked as "feature graphic", and the contour of this feature graphic can be called the second graphic contour.

[0084] Return to Figure 2, at block 206, feature matching is performed in the image using the first feature region to determine the second feature region, such that the first graphic contour matches the second graphic contour within the second feature region.

[0085] For the method of image matching, common methods already available in each image library can be used, such as brute-force matching and Fast Library for Approximate Nearest Neighbors (FLANN) matching, etc.

[0086] In some embodiments, performing feature matching in the image using the first feature region includes using the brute-force matching method, which mainly finds the optimal feature point matching position by exhaustive search.

[0087] In some embodiments, performing feature matching in the image using the first feature region may include using fast approximate nearest neighbor matching, which can be used to efficiently find the nearest neighbors in a dataset.

[0088] Performing graphic matching in the original image using the obtained feature graphics can obtain the positions of more feature graphics, which can be respectively denoted as R1, R2, …….

[0089] See Figure 10 , Figure 10 shows a schematic diagram of each feature region according to some embodiments of the present disclosure. As Figure 10 shown, each of the feature regions includes a feature graphic, Figure 10 a total of five feature graphics are shown in , which are respectively labeled as R0, R1, R2, R3, and R4. If R0 is referred to as the first feature graphic, one or more of R1 to R4 can be referred to as the second feature graphic, and its contour is the second graphic contour.

[0090] The above embodiments are described in conjunction with Figure 3 to Figure 10 and the embodiments of the present disclosure are not limited thereto. As mentioned before, as Figure 15 shown, the detection line 306 extends along a direction parallel to the bottom edge of the image, and during the detection process, the detection line moves perpendicular to the bottom edge. Subsequently, the first feature region can be determined in the manner shown in Figure 16 and Figure 17 .

[0091] See Figure 16 , Figure 16 shows a schematic diagram of two points in the image for determining the feature region according to some embodiments of the present disclosure. As Figure 16 shown, the first feature point P0 and the first auxiliary point P1 are shown therein.

[0092] Figure 17 A schematic diagram showing four points for determining a feature region in an image of some embodiments of the present disclosure. Among the four points, there are a first feature point P0, a first auxiliary point P1, a third auxiliary point P2, and a fourth auxiliary point P3. Thus, a region enclosing the first feature point P0, the first auxiliary point P1, the third auxiliary point P2, and the fourth auxiliary point P3 can be determined as the first feature region. The size of the first feature region can be selected according to actual needs, and the present disclosure does not limit this.

[0093] Figures 18 to 24 A schematic diagram showing the process of processing a TEM image according to an example of the present disclosure. The specific process can adopt the method of the above embodiments.

[0094] Figure 18 A schematic diagram of a TEM image to be processed according to another embodiment of the present disclosure. Figure 19 Shows Figure 18 A schematic diagram of the contour of the image to be processed shown in

[0095] Figure 20 Shows the detection of feature points in the continuous contours shown in Figure 19 along the first direction. As Figure 20 shown, P0 and P1 are determined therein.

[0096] Figure 21 A schematic diagram showing four points for determining a first feature domain in some embodiments of the present disclosure. Among them, the first feature point P1, the second auxiliary point P1, the third auxiliary point P2, and the fourth auxiliary point P3 are shown.

[0097] At block 208, the angle by which the image needs to be rotated is determined based on the first graphic contour and the second graphic contour.

[0098] In some embodiments, determining the angle by which the image needs to be rotated based on the first graphic contour and the second graphic contour may include: determining the fitting line of the midpoints of the first feature region and the second feature region respectively; determining the angle between the fitting line and the horizontal line or the vertical line perpendicular to the horizontal line in the plane where the image is located; and determining the angle as the angle by which the image needs to be rotated.

[0099] See Figure 11 , Figure 11 A schematic diagram showing the slope fitting of a feature graphic in some embodiments of the present disclosure.

[0100] In some embodiments, the slope between points can be calculated according to the central coordinates of R0, R1, R2, …, R5, and thus the inclination angle of the graphic in the original image can be converted back. According to the converted angle, the original image can be leveled.

[0101] Figure 11 Each small dot shown in the figure represents the central coordinates of R0, R1, R2, …, R5 of the graphic outline within each characteristic region. By fitting to obtain a fitted curve (specifically a straight line), its slope can be determined, and thus the inclination angle of the figure can be determined. Specifically, the slope of this straight line is the inclination angle of the figure. In addition Figure 11 The expression of the fitted curve 1102 is shown in the figure: y = 0.9776x + 0.8721, where x is the abscissa and y is the ordinate, R 2 = 0.9954. The R-squared value is an index of the fitting degree of the trend line. Its numerical value can reflect the fitting degree between the estimated value of the trend line and the corresponding actual data. The higher the fitting degree, the higher the reliability of the trend line. The R-squared value is a numerical value within the range of 0 to 1. When the R-squared value of the trend line is equal to 1 or close to 1, its reliability is the highest, and vice versa, the reliability is lower. Ideally, if these points found are all correct, then they should be on a straight line. The better the linearity, that is, the larger the R 2 the larger it is, the more accurate it indicates.

[0102] In the above embodiment, by solving the expression of the fitted curve, the slope is determined according to the parameters in its expression. For example, if the shown expression is a unary equation, its slope can be determined, and the slope is the inclination angle.

[0103] By means of fitting, the influence of abnormal points on the curve can be reduced, so that the inclination angle of the figure can be accurately determined.

[0104] It should be understood that the embodiments of the present disclosure are not limited thereto, but various changes can be made.

[0105] For example, the slope of the midpoint of two of the characteristic figures or the connection line between the midpoints of multiple characteristic figures can be used as the inclination angle as the basis for leveling. For example, the slope of the connection line between the midpoint of the first characteristic figure and the midpoint of the last characteristic figure can be used as the inclination angle.

[0106] At block 210, the image is rotated within the plane of the image based on the angle so that the hatching extends in a direction parallel to or perpendicular to the horizontal line within the plane of the image.

[0107] Next, refer to Figure 12 , Figure 12 which shows a schematic diagram of the leveled image according to some embodiments of the present disclosure. As Figure 12As shown, in the leveled figure, the parallel cross-hatching 304 is parallel to the edges on the upper and lower sides of the image, that is, the cross-hatching extends in a direction parallel to the horizontal line in the plane where the image is located. According to the actual situation, it is possible that after leveling, the cross-hatching extends in a direction perpendicular to the horizontal line in the plane where the image is located.

[0108] Refer to the following Figure 22 , Figure 22 which shows Figure 18 a schematic diagram of the characteristic figure corresponding to the first characteristic region of the original image shown in

[0109] Figure 23 which shows Figure 18 schematic diagrams of the graphic contours within each characteristic region of the original image shown in

[0110] Figure 24 which shows Figure 23 a schematic diagram of the slope fitting of the graphic contour shown in Figure 3 As shown in

[0111] the expression of the obtained fitting curve (substantially a straight line here) is: 2 = 0.9936.

[0112] Some embodiments of the present disclosure provide methods for image processing. It should be noted that the examples given in the above embodiments are only for illustrating the solutions of the embodiments of the present disclosure and do not limit the solutions of the present disclosure.

[0113] The methods of some embodiments of the present disclosure find the characteristic figure by automatically identifying the graphic feature points in the picture, without manual intervention, avoiding the errors caused by manual adjustment. This method is universal for different repeated graphics in TEM pictures and can be extended to the leveling processing of TEM pictures of different products. In addition, this method can identify all the characteristic figures in the picture, and the credibility of the calculation results can be verified by the slopes between the characteristic figures, that is, the slopes between the characteristic figures at different positions obtained by matching should be within the error range. If it exceeds the error range, the result may be abnormal. Therefore, this method has the self-judgment ability for the leveling result.

[0114] In short, the technical solution of the present disclosure can effectively solve the inconvenience brought by the manual leveling method, improving the leveling efficiency and leveling accuracy. Specifically, through automatic leveling, the consistency and stability of the leveling result can be ensured, reducing or avoiding human errors.

[0115] It should be understood that the embodiments shown in the drawings are only for schematically showing the solutions of some embodiments of the present disclosure, and are not used to limit the present disclosure. The embodiments of the present disclosure may also have various other forms.

[0116] An electronic device is also disclosed in an embodiment of the present disclosure. The electronic device includes: a processor; and a memory coupled to the processor, the memory having instructions stored therein, the instructions causing the device to perform actions when executed by the processor, the actions including: determining a first feature point of a graphic contour in an image, where the graphic contour includes hatching of parallel layers, and the first feature point is a point having an extreme value within a predetermined range; determining a first feature region based on the first feature point, where the first feature region surrounds a first graphic contour including the first feature point; performing feature matching in the image using the first feature region to determine a second feature region such that the first graphic contour matches a second graphic contour within the second feature region; determining an angle by which the image needs to be rotated based on the first graphic contour and the second graphic contour; and rotating the image within the plane of the image based on the angle so that the hatching extends in a direction parallel to or perpendicular to a horizontal line within the plane of the image.

[0117] A computer-readable storage medium is also disclosed in an embodiment of the present disclosure, on which a computer program is stored, and the program implements an image processing method according to an embodiment of the present disclosure when executed by a processor.

[0118] Figure 25 A schematic block diagram of an electronic device according to some exemplary embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0119] As Figure 25 shown, the device 2500 includes a CPU 2501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 2502 or a computer program loaded from a storage unit 2508 into a random access memory (RAM) 2503. In the RAM 2503, various programs and data required for the operation of the device 2500 can also be stored. The CPU 2501, the ROM 2502, and the RAM 2503 are connected to each other through a bus 2504. An input / output (I / O) interface 2505 is also connected to the bus 2504.

[0120] Multiple components in device 2500 are connected to I / O interface 2505. The multiple components include: an input unit 2506, such as a keyboard, a mouse, etc.; an output unit 2507, such as various types of displays, speakers, etc.; a storage unit 2508, such as a disk, an optical disc, etc.; and a communication unit 2509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 2509 allows device 2500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0121] Each of the processes and treatments described above, such as method 200, can be executed by CPU 2501. For example, in some embodiments, method 200 can be implemented as a computer software program which is tangibly contained in a machine-readable medium, such as storage unit 2508. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 2500 via ROM 2502 and / or communication unit 2509. When the computer program is loaded into RAM 2503 and executed by CPU 2501, one or more steps of method 200 described above can be executed.

[0122] The solutions according to the embodiments of the present disclosure can be a method, an apparatus, a system, and / or a computer program product. The computer program product can include a computer-readable storage medium having thereon computer-readable program instructions for performing various aspects of the present disclosure. The computer-readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. The computer-readable program instructions can be downloaded from the computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or an external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network.

[0123] The embodiments of the present disclosure have been described above. The above description is exemplary, and is only an optional embodiment of the present disclosure, not exhaustive, and is not used to limit the present disclosure. Although the claims in this application have been formulated for specific combinations of features, it should be understood that the scope of the present disclosure also includes any novel feature or any novel combination of features that are explicitly or implicitly disclosed herein or any generalization thereof, regardless of whether it relates to the same solution as any of the currently claimed claims. It should be noted that new claims can be formulated for these features and / or these combinations of features during the examination process of this application or in any further application derived therefrom.

[0124] The selection of the terms used in this document is intended to best explain the principles of the various embodiments, their practical applications, or the improvement of technologies in the market, or to enable other ordinary technicians in the technical field to understand the various embodiments disclosed in this document. For those skilled in the art, various changes and modifications can be made to the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. An image processing method, comprising: Determine a first feature point of a graphic contour in an image, wherein the first feature point is a point having an extreme value within a predetermined range in the graphic contour, and the graphic contour in the image includes a profile line contour of parallel layers; Determine a first feature area based on the first feature point, wherein the first feature area surrounds a first graphic outline including the first feature point; Performing feature matching in the image using the first feature area to determine a second feature area, so that the first graphic outline matches the second graphic outline within the second feature area; Determining an angle by which the image needs to be rotated based on the first graphic outline and the second graphic outline; as well as The image is rotated within the plane of the image based on the angle so that the section line extends in a direction parallel to or perpendicular to a horizontal line within the plane where the image is located.

2. The method according to claim 1, wherein determining the first feature point of the contour of the figure in the image comprises: Moving a detection line parallel to a first boundary of the image in a direction perpendicular to the first boundary; respectively determining the coordinates of the intersection points on the graphic contour intersecting with the detection line; as well as An intersection point whose coordinates have extreme values ​​in a predetermined range on the graphic contour along the direction of the detection line is taken as the first feature point.

3. The method according to claim 1, wherein determining a first feature area based on the first feature point comprises: Extending from the first feature point along a first direction to a first auxiliary point on a second contour opposite to the first graphic contour; Selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point so that the second auxiliary point intersects with the contour of the figure at a third auxiliary point and a fourth auxiliary point in a direction perpendicular to the line segment; as well as The first feature area is determined based on the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point.

4. The method according to claim 1, wherein determining a first feature area based on the first feature point comprises: Extending a predetermined distance from the first feature point along a first direction to a first auxiliary point; Selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point so that the second auxiliary point intersects with the contour of the figure at a third auxiliary point and a fourth auxiliary point in a direction perpendicular to the line segment; as well as The first feature area is determined based on the first feature point, the first auxiliary point, the third auxiliary point, and the fourth auxiliary point.

5. The method according to claim 3 or 4, wherein determining the first feature area based on the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point comprises: An area surrounding the first feature point, the first auxiliary point, the third auxiliary point and the fourth auxiliary point is selected as the first feature area.

6. The method according to claim 3 or 4, wherein selecting a second auxiliary point between a line segment connecting the first feature point and the first auxiliary point comprises: Selecting a second auxiliary point between the line segment connecting the first feature point and the first auxiliary point so that the distance between the third auxiliary point and the fourth auxiliary point is maximized; or The midpoint of the line segment connecting the first feature point and the first auxiliary point is selected as the second auxiliary point.

7. The method of claim 1, wherein feature matching is performed in the image using one of the following methods: Brute force matching; and Fast approximate proximity matching.

8. The method according to claim 1, wherein determining the angle by which the image needs to be rotated based on the first graphic outline and the second graphic outline comprises: Determine a fitting straight line of the midpoint of each of the first characteristic region and the second characteristic region; determining an angle between the fitting straight line and the horizontal line in the plane where the image is located or a vertical line perpendicular to the horizontal line; and The included angle is determined as the angle at which the image needs to be rotated.

9. An electronic device, comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, the instructions causing the device to perform actions when executed by the processor, the actions comprising: Determine a first feature point of a graphic contour in an image, wherein the first feature point is a point having an extreme value within a predetermined range in the graphic contour, and the graphic contour in the image includes a profile line contour of parallel layers; Determine a first feature area based on the first feature point, wherein the first feature area surrounds a first graphic outline including the first feature point; Performing feature matching in the image using the first feature area to determine a second feature area, so that the first graphic outline matches the second graphic outline within the second feature area; Determining an angle by which the image needs to be rotated based on the first graphic outline and the second graphic outline; and The image is rotated within the plane of the image based on the angle so that the section line extends in a direction parallel to or perpendicular to a horizontal line within the plane where the image is located.

10. A computer-readable storage medium having machine-executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 8.

Images