Image processing method, device, equipment and product

By acquiring and matching template images of complete repeating units, the problem of indirect information affecting accuracy in wafer detection is solved, and more efficient and accurate defect detection is achieved.

CN119090809AActive Publication Date: 2024-12-06SHENZHEN PLANCK SEMICON TECHNOLOGY CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411060688.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-12-06
Estimated Expiration
2044-08-02

AI Technical Summary

Technical Problem

In wafer detection, the existence of irrelevant information in the template image affects the defect detection accuracy, resulting in inaccurate detection results.

Method used

By obtaining the initial template image containing only the complete repeating units, the area to be matched with the complete repeating units is determined from the sample image, and matching it with the initial template image is obtained to obtain the matching template image to avoid interference from irrelevant information.

Benefits of technology

Improve the accuracy and efficiency of defect detection, ensure that the matching template image contains only key information, and improve the accuracy of the detection results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090809A_ABST
    Figure CN119090809A_ABST
Patent Text Reader

Abstract

The invention discloses an image processing method, device, equipment and product, and relates to the field of computer vision, and the method comprises the steps: obtaining an initial template image of a sample; acquiring a sample image; according to the initial position information of at least one repetitive unit area in the sample image, obtaining predicted position information of a plurality of repetitive units; obtaining a to-be-matched area of the sample image according to the predicted position information of the repetitive unit; respectively acquiring an area with the highest matching degree with the initial template image from the to-be-matched areas as a matching template image of the to-be-matched area; and detecting a to-be-detected image by using the plurality of matched template images to obtain a detection result. According to the invention, when the matching template images only containing the complete repetitive units are used for comparing the defects existing in the to-be-detected image, the detection efficiency and the detection precision are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to an image processing method, device, equipment and product. Background Art

[0002] During wafer inspection, in order to determine the defects on the wafer, the image to be inspected needs to be compared with the standard template image to obtain the defects in the image to be inspected. However, if the template image contains not only the key position information required for defect detection, but also irrelevant information that affects the accuracy of defect detection, then the defect detection results will inevitably be affected in the subsequent defect detection.

[0003] Therefore, how to retain key location information as much as possible and remove irrelevant information is a technical problem that needs to be solved urgently in the industry. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides an image processing method, device, equipment and product to solve the problem that irrelevant information that affects the defect detection accuracy still exists in the template image in the current wafer defect detection, thereby causing the defect detection results to be inaccurate.

[0005] According to a first aspect, an embodiment of the present invention provides an image processing method, the method comprising:

[0006] Acquire an initial template image of a sample; the surface to be tested of the sample has a plurality of repeating units, and the initial template image only contains an image of a single repeating unit;

[0007] Acquire a sample image, wherein the area where the image of the repeating unit is located in the sample image is a repeating unit area, and the sample image includes initial position information of each repeating unit area;

[0008] Acquire estimated position information of a plurality of repeating unit regions according to initial position information of at least one repeating unit region in the sample image;

[0009] Performing template matching processing on the sample image according to the estimated position information of each repeating unit area respectively; the template matching processing includes: obtaining a to-be-matched area of ​​the sample image according to the estimated position information of the repeating unit area; the sample image includes a plurality of to-be-matched areas, the to-be-matched areas include a complete repeating unit area and the size of the to-be-matched areas exceeds the size of the repeating unit area; obtaining the areas with the highest matching degree with the initial template image from the to-be-matched areas respectively as matching template images of the to-be-matched areas;

[0010] The image to be detected is detected using a plurality of matching template images to obtain a detection result.

[0011] In combination with the first aspect, in a first implementation of the first aspect, acquiring the to-be-matched area of ​​the sample image according to the estimated position information of the repeating unit area includes:

[0012] Based on the estimated position information of each repeating unit, the initial matching area of ​​the corresponding area to be matched is obtained respectively, and each initial matching area in the sample image is expanded according to a preset deviation adjustment value to obtain the area to be matched of the sample image.

[0013] In combination with the first aspect, in a second implementation of the first aspect, obtaining estimated position information of multiple repeating unit areas according to initial position information of at least one repeating unit area in the sample image includes: using the initial position information of at least one repeating unit area in the sample image as estimated position information of the corresponding repeating unit area; or

[0014] Acquiring estimated position information of a plurality of repeating unit regions according to initial position information of at least one repeating unit region in a sample image, and performing template matching processing on the sample image according to the estimated position information of each repeating unit region respectively, including:

[0015] Acquire any one of the repeating unit regions in the sample image as the starting unit region, and use the initial position information of the starting unit region as the estimated position information of the starting unit region;

[0016] Perform the template matching process on the sample image according to the estimated position information of the starting unit area, obtain the matching template image of the starting unit area, and obtain the matching position information of the starting unit area, wherein the matching position information is the position information of the matching template image of the starting unit area;

[0017] Acquire estimated position information of one or more other repeating unit regions based on the matching position information of the starting unit region and the distribution rule of the repeating units of the surface to be tested;

[0018] The template matching process is performed on the sample image according to the estimated position information of the repeating unit area outside the starting unit area to obtain at least one matching template image.

[0019] In combination with the first aspect, in a third implementation of the first aspect, obtaining, from the to-be-matched area, an area with the highest matching degree with the initial template image as the matching template image of the to-be-matched area specifically includes:

[0020] Each to-be-matched region is moved relative to the initial template image respectively, and during the movement, overlapping regions at multiple relative positions between each to-be-matched region and the initial template image are obtained as first overlapping regions;

[0021] Obtaining the matching degree between each first overlapping area of ​​the area to be matched and the initial template image;

[0022] The first overlapping area with the highest matching degree is used as the matching template image.

[0023] In combination with the first aspect, in a fourth implementation of the first aspect, obtaining, from the to-be-matched area, areas with the highest matching degree with the initial template image as matching template images of the to-be-matched area specifically includes:

[0024] Obtaining the region of interest in the initial template image;

[0025] Each to-be-matched region is moved relative to the region of interest, and during the movement, overlapping regions at multiple relative positions between each to-be-matched region and the region of interest are obtained as second overlapping regions;

[0026] Obtaining the matching degree between each second overlapping area of ​​the area to be matched and the area of ​​interest;

[0027] The relative position of the second overlapping area with the highest matching degree, the area to be matched and the first overlapping area of ​​the initial template are used as the matching template image.

[0028] In combination with the first aspect, in a fifth implementation of the first aspect, the region of interest includes a feature of interest in the initial template image, and the feature of interest includes at least one of a geometric region, an edge region, and a center region.

[0029] In combination with the first aspect, in a sixth implementation of the first aspect, detecting the image to be detected using the plurality of matching template images to obtain the detection result specifically includes:

[0030] Obtaining a target template image according to the plurality of matching template images;

[0031] The target template image is used to compare and process the image to be detected to obtain the target to be detected in the image to be detected.

[0032] In combination with the sixth implementation of the first aspect, in the seventh implementation of the first aspect, obtaining the target template image according to the multiple matching template images specifically includes:

[0033] Obtaining the average grayscale value of each pixel of each matching template image and the average grayscale value of each pixel of the image to be detected;

[0034] The matching template image having the smallest average grayscale value difference with the image to be detected is used as the target template image.

[0035] In combination with the sixth implementation of the first aspect, in the eighth implementation of the first aspect, obtaining the target template image according to the multiple matching template images specifically includes:

[0036] Obtaining an average grayscale value of a same matching point of a plurality of matching template images; the same matching point is a pixel point at the same physical position in the matching template image and the initial template image;

[0037] The image obtained according to the average grayscale value of all the same matching points of the matching template image is used as the target template image.

[0038] In combination with the sixth implementation manner of the first aspect, in a ninth implementation manner of the first aspect, the target to be detected is a defect; and using the target template image to perform comparative processing on the image to be detected to obtain the target to be detected in the image to be detected specifically includes:

[0039] Extracting the area with the highest matching degree with the target template image from the image to be detected as the target detection area of ​​the image to be detected;

[0040] Obtain the grayscale difference between the target template image and each pixel point in the target detection area; compare the grayscale difference with a preset threshold to obtain the target to be detected in the image to be detected.

[0041] According to a third aspect, an embodiment of the present invention further provides an image processing device, the device comprising:

[0042] A first acquisition module is used to acquire an initial template image of a sample; the surface to be tested of the sample has a plurality of repeating units, and the initial template image only contains an image of a single repeating unit;

[0043] A second acquisition module is used to acquire a sample image, wherein the area where the image of the repeating unit is located in the sample image is a repeating unit area, and the sample image includes initial position information of each repeating unit area;

[0044] A third acquisition module, used for acquiring estimated position information of a plurality of repeating unit regions according to initial position information of at least one repeating unit region in the sample image;

[0045] The image matching module is used to perform template matching processing on the sample image according to the estimated position information of each repeating unit area; the template matching processing includes: obtaining a to-be-matched area of ​​the sample image according to the estimated position information of the repeating unit area; the sample image includes a plurality of to-be-matched areas, the to-be-matched areas include a complete repeating unit area and the size of the to-be-matched areas exceeds the size of the repeating unit area; and obtaining the areas with the highest matching degree with the initial template image from the to-be-matched areas as matching template images of the to-be-matched areas respectively;

[0046] The detection module is used to detect the image to be detected using multiple matching template images to obtain a detection result.

[0047] According to a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-described image processing methods are implemented.

[0048] According to a fourth aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above-described image processing methods when executed by a processor.

[0049] The image processing method, device, equipment and product provided by the present invention obtain an initial template image that only contains complete repeating units and has the same size as the repeating units, and then determine at least one to-be-matched area that contains complete repeating units and has a size exceeding the size of the repeating units from the sample image, and use the area in each to-be-matched area that has the highest matching degree with the initial template image as a matching template image. In this way, a plurality of matching template images that also only contain complete repeating units are obtained by expanding one initial template image. The matching template image does not contain irrelevant information for defect detection, which solves the problem that irrelevant information that affects the defect detection accuracy still exists in the template image in the current wafer defect detection, thereby causing the defect detection result to be inaccurate. When these matching template images that only contain complete repeating units are used to compare the targets to be detected in the images to be detected, both the detection efficiency and the detection accuracy are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:

[0051] Figure 1 is a flowchart of an image processing method according to an embodiment of the present application;

[0052] Figure 2 is a schematic diagram of an initial template image obtained according to the image processing method of an embodiment of the present application;

[0053] Figure 3 is a schematic diagram of a sample image obtained by the image processing method according to an embodiment of the present application;

[0054] Figure 4 is a matching result graph obtained according to the image processing method of an embodiment of the present application;

[0055] Figure 5 is a flowchart of steps S103 and S104 in another embodiment of the image processing method according to an embodiment of the present application;

[0056] Figure 6is a flowchart of step S105 in another embodiment of the image processing method according to the embodiment of the present application;

[0057] Figure 7 is a flowchart of step S105 in another embodiment of the image processing method according to the embodiment of the present application;

[0058] Figure 8 is a flowchart of step S106 in another embodiment of the image processing method according to the embodiment of the present application;

[0059] Fig. 9 is a flowchart of step S1061 in another embodiment of the image processing method according to the embodiment of the present application;

[0060] Fig.10 is a flowchart of step S1062 in another embodiment of the image processing method according to the embodiment of the present application;

[0061] Fig.11 is a flowchart of step S1061 in another embodiment of the image processing method according to the embodiment of the present application;

[0062] Fig.12 is a schematic structural diagram of an image processing device according to an embodiment of the present application;

[0063] Fig.13 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0065] During wafer inspection, in order to determine the defects on the wafer, it is necessary to compare the image to be inspected with the standard template image to obtain the defects in the image to be inspected. The template image is usually an image that is consistent with the standard wafer image, and is obtained by photographing the object to be inspected that is consistent with the standard wafer. Therefore, the template image and the various parameters of the standard wafer contained in the template image are known parameters.

[0066] In some application scenarios, the defect detection process requires one-to-one correspondence between the pixels in the image to be detected and the template image in order to match the two images, so as to obtain the defects in the image to be detected. If the template image contains not only the key position information required for defect detection, but also irrelevant information that affects the accuracy of defect detection, then the subsequent defect detection will inevitably affect the defect detection results. For example, if the template image contains a cutting path, a large number of cutting path pixels will be compared when matching the two images mentioned above. The cutting path pixels are non-pixels of interest, that is, irrelevant information, and are not pixels of interest, that is, key position information. This causes the overall defect detection process to consume more resources and time, and will affect the accuracy of defect detection.

[0067] Therefore, how to retain key location information as much as possible and remove irrelevant information is a technical problem that needs to be solved urgently in the industry.

[0068] In order to solve the above problems, an image processing method is provided in this specification. The image processing method provided in this specification can be applied to electronic devices with image processing functions. The electronic devices may include notebooks, desktop computers, smart phones, smart wearable devices (virtual reality glasses, smart watches, etc.), tablet computers, etc. Of course, the image processing method provided in this specification can also be applied to applications running in the above-mentioned electronic devices. For example, the message display method can be applied to a browser with image processing functions, and can also be applied to image processing software.

[0069] In this embodiment, an image containing only complete repeating units, such as grain features, and having the same size as the repeating units, i.e., grains, is used as an example to describe the image processing method of the embodiment of the present invention, and at least one template image for defect detection is obtained from other images based on the image. In this way, an initial template image containing only repeating units and having the same size as the repeating units is used, and then at least one area to be matched containing repeating units and having a size exceeding the size of the repeating units is determined from the sample image. Then, the area to be matched is matched with the initial template image, and each area to be matched with the initial template image having the highest degree of match with the initial template image is used as a matching template image. In this way, a plurality of matching template images containing only complete repeating units are obtained using one initial template image. When these matching template images are then compared with the target to be detected in the image to be detected, both the detection efficiency and the detection accuracy are improved. Figure 1 As shown, the method may include:

[0070] S101, obtaining an initial template image of a sample. In this embodiment, the surface to be tested of the sample has multiple repeating units, and the initial template image only contains an image of a single repeating unit. For ease of understanding, an initial template image obtained at a certain time is used as an example for explanation. Figure 2 As shown, the initial template image only contains an image of a single repeating unit, and the size of the initial template image is consistent with that of the image of the repeating unit.

[0071] The repeating unit is a grain, so that the size of the initial template image is the same as the grain size, and the initial template image does not contain irrelevant information such as cutting paths except the grain features.

[0072] The initial template image may be stored in advance in the electronic device, or may be obtained by the electronic device from the outside, or may be manually obtained from the outside and stored in the electronic device. For example, the electronic device may obtain a standard design drawing or a standard wafer image from the outside, or may be manually selected from a standard design drawing or a standard wafer image, etc.

[0073] There is no restriction on the specific acquisition method of the initial template image. It only needs to ensure that the electronic device can acquire the initial template image.

[0074] S102, acquiring a sample image, wherein the region where the image of the repeating unit is located in the sample image is a repeating unit region, and the sample image includes initial position information of each repeating unit region.

[0075] The sample image may be stored in advance in the electronic device, or may be obtained by the electronic device from the outside, or may be manually obtained from the outside and stored in the electronic device. For example, the electronic device may obtain a wafer image from an external charge coupled device (CCD) camera, or the electronic device may extract an image frame containing the wafer from an external live video, or the wafer image may be manually selected from a live image, etc.

[0076] Taking the repeating unit as a crystal grain as an example, it can be understood that the sample image includes a complete wafer or a partial wafer. There is no restriction on the specific acquisition form of the sample image. It only needs to ensure that the electronic device can acquire the sample image.

[0077] In this embodiment, the sample image is a matching image to be matched with the initial template image, and at least one to-be-matched region can be obtained from each matching image.

[0078] Since the sample image contains a complete wafer or a partial wafer, at least one area to be matched can be obtained from a sample image. In this embodiment, considering the number and accuracy of template images that will be used as templates for defect detection later, at least one wafer image is selected as a sample image, so that several template images can be obtained based on the sample image.

[0079] Step S103: acquiring estimated position information of a plurality of repetitive unit regions according to initial position information of at least one repetitive unit region in the sample image.

[0080] In this embodiment, the expected position information is the position information of the area where each repeating unit should be located under an ideal physical state, for example, when a CCD camera shoots a wafer corresponding to a sample image with standard shooting parameters (including but not limited to: standard shooting angle, standard shooting distance and other parameters), the position information or coordinate information of each repeating unit in the image. Due to the influence of factors such as processing equipment, shooting equipment, and on-site environment, the actual position information of the corresponding pixel points of each repeating unit in the sample image actually obtained has a certain deviation from the corresponding expected position information, and the deviation of the repeating unit in each sample image is different.

[0081] Step S104: performing template matching processing on the sample image according to the estimated position information of each repeating unit area; the template matching processing includes:

[0082] Step S1041, obtaining a to-be-matched region of a sample image according to the estimated position information of the repeating unit; the sample image includes a plurality of to-be-matched regions, the to-be-matched regions include a complete repeating unit region and the size of the to-be-matched regions exceeds the size of the repeating unit region.

[0083] Step S1042, respectively obtain the area with the highest degree of matching with the initial template image from the area to be matched as the matching template image of the area to be matched. In step S1042, each area to be matched obtained in step S1041 is matched with the initial template image obtained in step S101, with the purpose of obtaining the area with the highest degree of matching with the initial template image, i.e., the highest similarity, from the area to be matched as the matching template image. It can be understood that each area to be matched can obtain a corresponding matching template image after matching.

[0084] For ease of understanding, the sample surface to be tested has four repeating units as an example. The initial template image obtained in step S101 is as follows: Figure 2 As shown, the sample image obtained in step S102 is as follows Figure 3As shown, there are four areas to be matched, and each area to be matched includes a complete repeating unit area and the size of the area to be matched exceeds the size of the repeating unit area, wherein the repeating unit area is the area where the image of the repeating unit is located in the sample image. The areas to be matched of the sample image obtained in step S1041 include: Figure 3 Each of the to-be-matched regions obtained in step S1041 is compared with the to-be-matched regions obtained in step S101. Figure 2 The initial template image shown in the figure is matched to obtain Figure 4 The matching result graph shown in the figure is Figure 4 The area within the white frame is the area where the to-be-matched area matches the initial template image.

[0085] Since the initial template image only contains complete repeating units and has the same size as the repeating units, when the matching area is subsequently matched with the initial template image, the area in each area to be matched that has the highest matching degree with the initial template image is used as a matching template image. In this way, a plurality of matching template images that also contain only complete repeating units are obtained using one initial template image. When these matching template images are then compared with the targets to be detected in the image to be detected, both the detection efficiency and the detection accuracy are improved. At the same time, the method of obtaining a plurality of template images using an initial template image avoids the situation in which the accuracy of these template images used for defect detection cannot be guaranteed due to the difficulty in screening out useless information when directly obtaining a plurality of template images. The method can also improve the efficiency of obtaining template images by expanding the initial template image to a plurality of matching template images.

[0086] S105 , using a plurality of matching template images to detect the image to be detected, and obtaining a detection result.

[0087] In this embodiment, specifically, the image to be detected is compared with the extracted matching template image to obtain the target to be detected in the image to be detected. Since the template image does not contain irrelevant information such as cutting lines, the detection result obtained in step S105, i.e., the defect detection result, is more accurate and requires less computational cost.

[0088] The image processing method provided by the present invention obtains an initial template image that only contains complete repeating units such as single-chip wafer body features and has the same size as the repeating units, and then determines at least one to-be-matched area that contains complete repeating units and has a size exceeding the size of the repeating units from a sample image, and uses the area in each to-be-matched area that has the highest matching degree with the initial template image as a matching template image. In this way, a plurality of matching template images that also only contain complete repeating units are obtained by expanding one initial template image. The matching template image does not contain irrelevant information for defect detection, which solves the problem that irrelevant information that affects the defect detection accuracy still exists in the template image in the current wafer defect detection, thereby causing the defect detection result to be inaccurate. When these matching template images that only contain complete repeating units are used to compare the targets to be detected in the images to be detected, both the detection efficiency and the detection accuracy are improved.

[0089] In some embodiments, step S1041 includes: obtaining the initial matching areas of the corresponding areas to be matched based on the estimated position information of each repeating unit, and expanding each initial matching area in the sample image according to a preset deviation adjustment value to obtain the areas to be matched of the sample image.

[0090] In this embodiment, the preset deviation adjustment value is used to determine the area to be matched according to the estimated position information of the area where each repeating unit is located, so that even if there is a certain deviation between the actual position information of the repeating unit in the sample image and the corresponding estimated position information, each determined area to be matched includes the complete repeating unit, that is, the grain feature, through the preset deviation adjustment value. It should be noted that a single area to be matched may include irrelevant information such as cutting path information, partial features of at least one adjacent repeating unit of the complete repeating unit, etc.

[0091] It can be understood that the preset deviation adjustment value is a range interval, which can cope with sample images obtained under different circumstances. The preset deviation adjustment value can be configured by the user according to actual production conditions.

[0092] In this embodiment, each to-be-matched region of the sample image may be determined based on the estimated position information of at least one key point such as a center point, an edge point, and a vertex of each repeating unit.

[0093] In this embodiment, if Figure 5 As shown, steps S103 and S104 include:

[0094] S1031. Acquire any one repeated unit region in the sample image as a starting unit region, and use initial position information of the starting unit region as estimated position information of the starting unit region.

[0095] S1032. Perform the template matching process on the sample image according to the estimated position information of the starting unit area, obtain the matching template image of the starting unit area, and obtain the matching position information of the starting unit area, wherein the matching position information is the position information of the matching template image of the starting unit area.

[0096] S1033: Obtain estimated position information of one or more other repeating unit regions based on the matching position information of the starting unit region and the distribution rule of the repeating units of the surface to be tested.

[0097] S1034: Perform the template matching process on the sample image according to the estimated position information of the repeated unit area outside the starting unit area to obtain at least one matching template image.

[0098] In this embodiment, the initial position information of the starting unit area is first used as the estimated position information of the starting unit, and then a template matching process is performed on the sample image according to the estimated position information of the starting unit to obtain the matching template image of the starting unit, and the matching position information of the starting unit area can be obtained from the matching template image; and then the estimated position information of one or more other repeating unit areas is obtained according to the matching position information and the distribution rule of the repeating unit. Obtaining the estimated position information of one or more other repeating unit areas according to the matching position information can improve the accuracy of the estimated position information of the repeating unit area, thereby improving the accuracy of the template matching process. Moreover, this method only changes the order of the template matching process of the starting unit area, and does not increase the processing time.

[0099] Since the repeating units in this embodiment are periodically arranged on the sample surface, there is a periodic distribution rule between the repeating units. Therefore, based on the position information of the matching template image of the starting unit area, the estimated position information of one or more repeating unit areas can be obtained based on the distribution rule of the repeating units on the surface to be tested. In other embodiments, the repeating units are not distributed periodically, but no matter how the repeating units are distributed, the estimated position information of other repeating units can be obtained according to their distribution rules.

[0100] In other embodiments of the present invention, step S103 includes: using the initial position information of at least one repeating unit area in the sample image as the estimated position information of the corresponding repeating unit area. Specifically, using the initial position information of multiple repeating unit areas in the sample image as the estimated position information of the corresponding repeating unit areas. Then, step S104 is performed on each repeating unit area.

[0101] In some embodiments, Figure 6 As shown, step S1042 includes:

[0102] S10421, respectively move each to-be-matched region relative to the initial template image, and obtain overlapping regions at multiple relative positions between each to-be-matched region and the initial template image during the movement process as first overlapping regions. In this step, multiple first overlapping regions are obtained during the relative movement process, and it is understood that the size of the first overlapping region is the same as the size of the initial template image.

[0103] In this step, each to-be-matched area is moved relative to the initial template image according to a preset matching scheme. In this embodiment, the preset matching scheme includes but is not limited to the following parameters: a moving starting point, a moving distance, a moving track, and a moving end point.

[0104] S10422, obtaining the matching degree between each first overlapping region of the area to be matched and the initial template image, that is, performing matching processing between each first overlapping region and the initial template image, and determining the matching degree between each first overlapping region and the initial template image.

[0105] S10423. Use the first overlapping region with the highest matching degree as the matching template image. That is, use the first overlapping region with the highest matching degree as the matching template image.

[0106] When the relative movement between the area to be matched and the initial template image is performed according to the preset matching scheme, a number of first overlapping areas will be obtained. Step S1052 matches each first overlapping area with the initial template image. By calculating the grayscale value between the pixel points of the two, the similarity between the two, that is, the matching degree, can be obtained. The higher the similarity, the higher the matching degree. Conversely, the lower the similarity, the lower the matching degree.

[0107] The similarity can be determined by the sum, average or mean square error of the grayscale differences of the pixels.

[0108] Only when the highest matching degree, i.e. the similarity, exceeds the preset threshold, the corresponding first overlapping area will be used as the template image. If there is no first overlapping area with a matching degree exceeding the preset threshold in the area to be matched, no area in the area to be matched will be selected as the template image.

[0109] Of course, other matching degree parameters may also be used to determine the above matching degree.

[0110] Combine the following Figure 7 The image processing method of the present invention is described. Considering that the amount of calculation required for comparing all pixel points to determine the similarity between the first overlapping area and the initial template image is large, the comparison of pixel points corresponding to the region of interest is used as an example for description. In this way, only pixel comparison is performed on the region of interest, which reduces the amount of calculation in the matching process and further improves the defect detection efficiency. Step S1042 may include:

[0111] S10421. Obtain a region of interest in the initial template image.

[0112] The region of interest includes the feature of interest in the initial template image, and the feature of interest includes at least one of a geometric region, an edge region, and a center region. For a grain, the similarity can be determined by comparing the high-frequency region of the grain, which often contains the key features of the grain, i.e., the feature of interest. In the embodiment of the present invention, high-frequency regions such as geometric regions and edge regions are used as regions of interest, so that the region of interest will contain the feature of interest in the initial template image.

[0113] The edge of the graphic can be obtained by calculating the grayscale gradient value of each pixel of the initial template image, and then the edge area can be obtained.

[0114] S10422, respectively move each to-be-matched region relative to the region of interest, and obtain overlapping regions at multiple relative positions between each to-be-matched region and the region of interest during the movement process as second overlapping regions. This step will obtain multiple second overlapping regions during the relative movement process. It is understandable that the size of the second overlapping region is the same as the size of the region of interest.

[0115] The relative movement and the method of determining the overlapping area refer to the following Figure 6 The description of step S10421 in the illustrated embodiment will not be elaborated upon here.

[0116] S10423, obtaining the matching degree between each second overlapping region of the to-be-matched region and the region of interest, that is, performing matching processing between each second overlapping region and the region of interest, and determining the matching degree between each second overlapping region and the region of interest.

[0117] S10424: Use the relative position of the second overlapping region with the highest matching degree, the region to be matched and the first overlapping region of the initial template as a matching template image.

[0118] The matching degree is determined by referring to Figure 6 In the embodiment shown, step S10422 is described, which will not be elaborated here, except for Figure 6 In the embodiment shown, this embodiment only needs to perform pixel comparison on the region of interest, which reduces the amount of calculation in the matching process and further improves the defect detection efficiency.

[0119] In this embodiment, if Figure 8 As shown, step S105 may include:

[0120] S1051. Obtain a target template image according to the multiple matching template images.

[0121] S1052: Using the target template image to perform comparison processing on the image to be detected, to obtain the target to be detected in the image to be detected.

[0122] The target to be measured is a defect or a characteristic structure on the surface of a sample, and the characteristic structure includes a specific pattern, such as a circuit or a grating or a solder ball or a through silicon via.

[0123] In this embodiment, the comparison process is used to obtain the position of the target to be measured. In other embodiments, the method further includes: obtaining the size of the target to be measured according to the position of the target to be measured.

[0124] In this embodiment, if Fig. 9 As shown, step S1051 includes:

[0125] S10511. Obtain the average grayscale value of each pixel of each matching template image and the average grayscale value of each pixel of the image to be detected.

[0126] S10512. Use the matching template image with the smallest difference in average grayscale value with the image to be detected as the target template image, that is, use the matching template image with the closest average grayscale value to the image to be detected as the target template image.

[0127] In this embodiment, the target to be detected is a defect, such as Fig.10 As shown, step S1052 includes:

[0128] S10521, extracting the area with the highest matching degree with the target template image from the image to be detected as the target detection area, for details, refer to Figure 1 , 6 In the embodiment shown in 7, the area with the highest matching degree is determined as the target detection area by comparing all pixels or by comparing pixels in the area of ​​interest, etc., which is not limited here.

[0129] S10522, obtaining the grayscale difference between the target template image and each pixel point in the target detection area, and comparing the grayscale difference with a preset threshold to obtain the target to be detected in the image to be detected.

[0130] Specifically, in this embodiment, when the grayscale difference is greater than a preset value, the pixel is determined to be a defect.

[0131] Defect detection can be performed through various feature comparisons, threshold segmentation detection, segmentation extraction algorithms and models, etc., and there is no limitation here.

[0132] In other embodiments of the present invention, Fig.11 As shown, step S1051 includes:

[0133] S10511, obtaining an average grayscale value of a same matching point of a plurality of matching template images, wherein the same matching point is a pixel point at the same physical position in the matching template image and the initial template image;

[0134] S10512, the image obtained according to the average grayscale value of all the same matching points of the matching template image is used as the target template image. By calculating the average grayscale value of each corresponding pixel point of multiple matching template images, for example, calculating the grayscale value of the pixel point corresponding to the coordinate position A of multiple matching template images, the average grayscale value of all matching template images at the corresponding pixel point can be obtained, and then the target template image can be obtained according to the average grayscale value of all pixels.

[0135] The image processing device provided by the embodiment of the present invention is described below. The image processing device described below and the image processing method described above can be referred to each other.

[0136] In order to solve the above problems, an image processing device is provided in this specification, and the image processing device provided in this specification can be applied to electronic devices with image processing functions. The electronic devices may include notebooks, desktop computers, smart phones, smart wearable devices (virtual reality glasses, smart watches, etc.), tablet computers, etc.

[0137] In this embodiment, an image containing only complete grain features and the same size as the grain is used, and at least one template image for defect detection is obtained from other images based on the image as an example to describe the image processing method of the embodiment of the present invention. In this way, an initial template image containing only grain features and the same size as the grain is used, and then at least one area to be matched containing grain features and a size exceeding the grain size is determined from the sample image. Then, the area to be matched is matched with the initial template image, and each area to be matched with the initial template image with the highest degree of matching is used as a matching template image. In this way, a plurality of matching template images containing only complete grain features are obtained using one initial template image. When these matching template images are then used to compare the defects in the image to be detected, both the defect detection efficiency and the detection accuracy are improved. Fig.12 As shown, the device may include:

[0138] The first acquisition module 101 is used to acquire an initial template image of a sample. In this embodiment, the surface to be tested of the sample has a plurality of repeating units, and the initial template image only includes an image of a single repeating unit.

[0139] The repeating unit is a grain, so the size of the initial template image is the same as the grain size, and the initial template image does not contain irrelevant information such as cutting lines except the grain features.

[0140] The initial template image may be stored in advance in the electronic device, or may be obtained by the electronic device from the outside, or may be manually obtained from the outside and stored in the electronic device. For example, the electronic device may obtain a standard design drawing or a standard wafer image from the outside, or may be manually selected from a standard design drawing or a standard wafer image, etc.

[0141] There is no restriction on the specific acquisition method of the initial template image. It only needs to ensure that the electronic device can acquire the initial template image.

[0142] The second acquisition module 102 is used to acquire a sample image, wherein the area where the image of the repeating unit is located in the sample image is a repeating unit area, and the sample image includes initial position information of each repeating unit area.

[0143] The third acquisition module 103 is used to acquire estimated position information of multiple repetitive unit areas according to initial position information of at least one repetitive unit area in the sample image.

[0144] The image matching module 104 is used to perform template matching processing on the sample image according to the estimated position information of each repeating unit area. The template matching processing includes: obtaining the to-be-matched area of ​​the sample image according to the estimated position information of the repeating unit, the sample image includes multiple to-be-matched areas, the to-be-matched areas include the complete repeating unit area and the size of the to-be-matched areas exceeds the size of the repeating unit area; and obtaining the areas with the highest matching degree with the initial template image from the to-be-matched areas as the matching template images of the to-be-matched areas. The sample image includes at least one to-be-matched area, the to-be-matched area includes the complete grain features and the size of the to-be-matched area exceeds the size of the single wafer body.

[0145] The sample image may be stored in advance in the electronic device, or may be obtained by the electronic device from the outside, or may be manually obtained from the outside and stored in the electronic device. For example, the electronic device may obtain a wafer image from an external CCD camera, or the electronic device may extract an image frame containing a wafer from an external live video, or the wafer image may be manually selected from a live image, etc.

[0146] Taking the repeating unit as a crystal grain as an example, it can be understood that the sample image includes a complete wafer or a partial wafer. There is no restriction on the specific acquisition form of the sample image. It only needs to ensure that the electronic device can acquire the sample image.

[0147] In this embodiment, the sample image is a matching image to be matched with the initial template image, and at least one to-be-matched region can be obtained from each matching image.

[0148] Since the sample image contains a complete wafer or a partial wafer, at least one area to be matched can be obtained from a sample image. In this embodiment, considering the number and accuracy of template images that will be used as templates for defect detection later, at least one wafer image is selected as a sample image, so that several template images can be obtained based on the sample image.

[0149] The image matching module 104 will perform matching processing on each acquired area to be matched with the initial template image, with the purpose of extracting the area with the highest matching degree, i.e., the highest similarity, with the initial template image from the area to be matched as the matching template image. It can be understood that a corresponding matching template image can be extracted from each area to be matched after the matching processing.

[0150] Since the initial template image only contains complete repeating units and has the same size as the repeating units, when the area to be matched is subsequently matched with the initial template image, the area in each area to be matched that has the highest match with the initial template image is used as a matching template image. In this way, a number of matching template images that also only contain complete repeating units are obtained using one initial template image. When these template images are compared with defects in the image to be inspected, both the defect detection efficiency and the detection accuracy are improved, and the situation in which the accuracy of several template images cannot be guaranteed when used for defect detection when several template images are directly obtained is avoided. The efficiency of obtaining template images can also be improved by expanding the initial template image to several matching template images.

[0151] The detection module 105 is used to detect the image to be detected using the multiple matching template images to obtain a detection result.

[0152] In this embodiment, specifically, the image to be detected is compared with the extracted matching template image to obtain the target to be detected in the image to be detected. Since the matching template image does not contain irrelevant information such as cutting paths, the detection result obtained by the detection module 105 is more accurate and requires less computational cost.

[0153] The image processing device provided by the present invention obtains an initial template image that only contains complete single-chip wafer body features and has the same size as a repeating unit, and then determines at least one to-be-matched area that contains a complete repeating unit and has a size exceeding the size of the repeating unit from a sample image, and uses the area in each to-be-matched area that has the highest matching degree with the initial template image as a matching template image. In this way, a plurality of matching template images that also only contain complete repeating units are obtained by expanding one initial template image. The matching template image does not contain irrelevant information for defect detection, which solves the problem that irrelevant information that affects the defect detection accuracy still exists in the template image in the current wafer defect detection, thereby causing the defect detection result to be inaccurate. When these matching template images that only contain complete repeating units are used to compare the defects in the image to be detected, both the defect detection efficiency and the detection accuracy are improved.

[0154] Fig.13 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.13 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic command in the memory 830 to execute the image processing method.

[0155] In addition, the logic commands in the above-mentioned memory 830 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software medium, and the computer software medium is stored in a storage medium, including several commands to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0156] On the other hand, the present invention also provides a computer program medium, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image processing method provided above.

[0157] On the other hand, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented to perform the above-mentioned image processing method when executed by a processor.

[0158] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0159] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software medium, and the computer software medium can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of commands to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an initial template image of a sample; the surface to be tested of the sample has a plurality of repeating units, and the initial template image only contains an image of a single repeating unit; Acquire a sample image, wherein the area where the image of the repeating unit is located in the sample image is a repeating unit area, and the sample image includes initial position information of each repeating unit area; Acquire estimated position information of a plurality of repeating unit regions according to initial position information of at least one repeating unit region in the sample image; Performing template matching processing on the sample image according to the estimated position information of each repeating unit area respectively; the template matching processing includes: obtaining a to-be-matched area of ​​the sample image according to the estimated position information of the repeating unit area; the sample image includes a plurality of to-be-matched areas, the to-be-matched areas include a complete repeating unit area and the size of the to-be-matched areas exceeds the size of the repeating unit area; obtaining the areas with the highest matching degree with the initial template image from the to-be-matched areas respectively as matching template images of the to-be-matched areas; The image to be detected is detected using a plurality of matching template images to obtain a detection result.

2. The image processing method according to claim 1, characterized in that: The step of obtaining the to-be-matched area of ​​the sample image according to the estimated position information of the repeating unit area comprises: Based on the estimated position information of each repeating unit, the initial matching area of ​​the corresponding area to be matched is obtained respectively, and each initial matching area in the sample image is expanded according to a preset deviation adjustment value to obtain the area to be matched of the sample image.

3. The image processing method according to claim 1, characterized in that: Acquiring estimated position information of a plurality of repeating unit regions according to initial position information of at least one repeating unit region in the sample image includes: using the initial position information of at least one repeating unit region in the sample image as estimated position information of the corresponding repeating unit region; or Acquiring estimated position information of a plurality of repeating unit regions according to initial position information of at least one repeating unit region in a sample image, and performing template matching processing on the sample image according to the estimated position information of each repeating unit region respectively, including: Acquire any one of the repeating unit regions in the sample image as the starting unit region, and use the initial position information of the starting unit region as the estimated position information of the starting unit region; Perform the template matching process on the sample image according to the estimated position information of the starting unit area, obtain the matching template image of the starting unit area, and obtain the matching position information of the starting unit area, wherein the matching position information is the position information of the matching template image of the starting unit area; Acquire estimated position information of one or more other repeating unit regions based on the matching position information of the starting unit region and the distribution rule of the repeating units of the surface to be tested; The template matching process is performed on the sample image according to the estimated position information of the repeating unit area outside the starting unit area to obtain at least one matching template image.

4. The image processing method according to claim 1, characterized in that: The obtaining, from the to-be-matched area, the area with the highest matching degree with the initial template image as the matching template image of the to-be-matched area specifically includes: Each to-be-matched region is moved relative to the initial template image respectively, and during the movement, overlapping regions at multiple relative positions between each to-be-matched region and the initial template image are obtained as first overlapping regions; Obtaining the matching degree between each first overlapping area of ​​the area to be matched and the initial template image; The first overlapping area with the highest matching degree is used as the matching template image.

5. The image processing method according to claim 4, characterized in that: The obtaining, from the to-be-matched area, the area with the highest matching degree with the initial template image as the matching template image of the to-be-matched area specifically includes: Obtaining the region of interest in the initial template image; Each to-be-matched region is moved relative to the region of interest, and during the movement, overlapping regions at multiple relative positions between each to-be-matched region and the region of interest are obtained as second overlapping regions; Obtaining the matching degree between each second overlapping area of ​​the area to be matched and the area of ​​interest; The relative position of the second overlapping area with the highest matching degree, the area to be matched and the first overlapping area of ​​the initial template are used as the matching template image.

6. The image processing method according to claim 5, characterized in that: The region of interest includes a feature of interest in the initial template image, and the feature of interest includes at least one of a geometric region, an edge region, and a center region.

7. The image processing method according to claim 1, characterized in that: The detecting of the image to be detected by using the plurality of matching template images to obtain the detection result specifically includes: Obtaining a target template image according to the plurality of matching template images; The target template image is used to compare and process the image to be detected, so as to obtain the target to be detected in the image to be detected.

8. The image processing method according to claim 7, characterized in that: The step of obtaining a target template image according to the plurality of matching template images specifically includes: Obtaining the average grayscale value of each pixel of each matching template image and the average grayscale value of each pixel of the image to be detected; The matching template image having the smallest average grayscale value difference with the image to be detected is used as the target template image.

9. The image processing method according to claim 7, characterized in that: The step of obtaining a target template image according to the plurality of matching template images specifically includes: Obtaining an average grayscale value of a same matching point of a plurality of matching template images; the same matching point is a pixel point at the same physical position in the matching template image and the initial template image; The image obtained according to the average grayscale value of all the same matching points of the matching template image is used as the target template image.

10. The image processing method according to claim 7, characterized in that: The target to be detected is a defect; and using the target template image to compare and process the image to be detected to obtain the target to be detected in the image to be detected specifically includes: Extracting the area with the highest matching degree with the target template image from the image to be detected as the target detection area of ​​the image to be detected; Obtain the grayscale difference of each pixel point of the target template image relative to the target detection area; compare the grayscale difference with the preset threshold to obtain the target to be detected in the image to be detected.

11. An image processing device based on the image processing method according to claims 1 to 10, characterized in that: The device comprises: A first acquisition module is used to acquire an initial template image of a sample; the surface to be tested of the sample has a plurality of repeating units, and the initial template image only contains an image of a single repeating unit; A second acquisition module is used to acquire a sample image, wherein the area where the image of the repeating unit is located in the sample image is a repeating unit area, and the sample image includes initial position information of each repeating unit area; A third acquisition module, used for acquiring estimated position information of a plurality of repeating unit regions according to initial position information of at least one repeating unit region in the sample image; The image matching module is used to perform template matching processing on the sample image according to the estimated position information of each repeating unit area; the template matching processing includes: obtaining a to-be-matched area of ​​the sample image according to the estimated position information of the repeating unit area; the sample image includes a plurality of to-be-matched areas, the to-be-matched areas include a complete repeating unit area and the size of the to-be-matched areas exceeds the size of the repeating unit area; and obtaining the areas with the highest matching degree with the initial template image from the to-be-matched areas as matching template images of the to-be-matched areas respectively. The detection module is used to detect the image to be detected using multiple matching template images to obtain a detection result.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the image processing method according to any one of claims 1 to 10 are implemented.

13. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 10 are implemented.

Citation Information

Patent Citations

  • A method and an apparatus for generating information

    CN109165572A

  • Visible light positioning method and device, equipment and medium

    CN112183526A

  • Visual detection method and device for transparent object, computer equipment and medium

    CN115205223A

  • Defect detection method and device, electronic equipment and storage medium

    CN116797550A

  • Defect detection method and device based on machine vision and storage medium

    CN116958019A