Defect type detection method and device, electronic equipment and storage medium

Abstract

The invention provides a defect type detection method and device, electronic equipment and a storage medium. The detection method comprises the following steps: determining a first target area and a first defect area in a first image of a to-be-detected object; based on first position information of the first target area in the first image, a second image of the to-be-detected object is obtained, and the definition of the second image is higher than that of the first image; in the second image, determining a second target area corresponding to the first target area; and determining the defect type of the to-be-detected object by using the second image based on the first defect area and the second target area. According to the scheme, the first image with relatively low definition can be firstly processed, and then the second image with relatively high definition is acquired, so that the total processing amount of the second image can be reduced, the data processing difficulty is further reduced, and the determination efficiency of the defect type is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically to a defect type detection method, a defect type detection device, an electronic device, a storage medium and a computer program product. Background Art

[0002] In the industrial field, with the continuous development of production technology, people have put forward higher requirements for the production quality and production efficiency of industrial products. With the development of image processing, more and more product inspections are based on images. If the inspection accuracy is low, the practicality of the image-based inspection method will also be reduced.

[0003] For example, in the process of wafer inspection, the related technology usually uses automated optical inspection (AOI) equipment to perform appearance inspection on the wafer to determine the type of defects on the wafer. However, the different defects on the wafer are highly random, complex and changeable. Therefore, how to accurately detect the defect type is a technical problem that technicians in this field urgently need to solve. Summary of the invention

[0004] The present invention is proposed in view of the above problems. The present invention provides a defect type detection method, a defect type detection device, an electronic device, a storage medium and a computer program product.

[0005] According to one aspect of the present invention, a method for detecting defect types is provided, the method comprising:

[0006] Determine a first target region and a first defect region in a first image of the object to be tested, wherein the first target region is a region corresponding to a target of interest in the object to be tested during quality inspection in the first image, and at least a portion of the first defect region is in the first target region;

[0007] Based on first position information of the first target area in the first image, obtaining a second image of the object to be measured, wherein the clarity of the second image is higher than that of the first image;

[0008] In the second image, determining a second target area corresponding to the first target area;

[0009] Based on the first defect area and the second target area, the defect type of the object to be measured is determined using the second image.

[0010] Exemplarily, the above detection method further includes:

[0011] Determining a second defect area in the second image corresponding to the first defect area based on a position mapping relationship between the second image and the first image;

[0012] Based on the first defect area and the second target area, determining the defect type of the object to be measured by using the second image includes:

[0013] Inputting the second image into the instance segmentation model to obtain instance regions and instance categories corresponding to the instance regions;

[0014] Positionally matching the instance region with the second defect region to determine a successfully matched instance region;

[0015] The defect type of the object to be tested is determined based on the second target area, the successfully matched instance area, and the instance category corresponding to the successfully matched instance area.

[0016] Exemplarily, determining the defect type of the object to be tested based on the second target area, the successfully matched instance area, and the instance category corresponding to the successfully matched instance area includes:

[0017] Determine whether the successfully matched instance area exceeds the boundary of the second target area to obtain a determination result;

[0018] Based on the judgment result and the instance category corresponding to the successfully matched instance area, the defect type of the object to be tested is determined.

[0019] Exemplarily, the successfully matched instance regions include one or more, and determining the defect type of the object to be tested based on the judgment result and the instance category corresponding to the successfully matched instance regions includes:

[0020] Based on the judgment result and the defect type corresponding to each successfully matched instance region, determining the defect level corresponding to each successfully matched instance region, wherein at least two defect types have different defect levels;

[0021] The defect type of the object to be tested is determined based on the defect level corresponding to each successfully matched instance region.

[0022] Exemplarily, the first image is acquired by a first image acquisition device, the second image is acquired by a second image acquisition device, and based on first position information of the first target area in the first image, the second image of the object to be measured is acquired, including:

[0023] Based on a mapping relationship between an image coordinate system of the first image acquisition device and a world coordinate system, converting the first position information into second position information in the world coordinate system;

[0024] adjusting a shooting position of the second image acquisition device based on a mapping relationship between an image coordinate system of the second image acquisition device and a world coordinate system and the second position information;

[0025] The second image is captured by the adjusted second image capturing device.

[0026] Exemplarily, determining a first target area and a first defect area in a first image of the object to be measured includes:

[0027] Matching the first image with the image template of the object to be tested to obtain an area in the first image that successfully matches the image template as a first target area;

[0028] Based on the image template, determining a foreground target area in the first target area, and taking the foreground target area that meets the preset conditions as a first defect area in the first target area;

[0029] The first image is compared with the reference image, and based on the comparison result, a first defect area at least partly located outside the first target area is determined.

[0030] Exemplarily, before determining the first target area and the first defect area in the first image of the object to be detected, the detection method further includes:

[0031] Acquire an initial image, wherein the initial image includes a plurality of initial wafers;

[0032] determining whether each initial wafer in the initial image meets a preset rule;

[0033] An initial wafer that meets the preset rules is used as an object to be tested, and a first image of the object to be tested is generated based on the initial image.

[0034] Exemplarily, the object to be tested is a wafer, and the targets of interest in the quality inspection of the object to be tested include at least pads on the wafer.

[0035] According to another aspect of the present invention, a defect type detection device is provided. The detection device includes: a first determination module, a second image acquisition module, a second determination module and a defect type determination module.

[0036] The first determination module is used to determine a first target area and a first defect area in a first image of the object to be tested, wherein the first target area is an area corresponding to a target of interest in the object to be tested during quality inspection in the first image, and at least a portion of the first defect area is in the first target area;

[0037] The second image acquisition module is used to acquire a second image of the object to be measured based on the first position information of the first target area in the first image, wherein the clarity of the second image is higher than that of the first image;

[0038] The second determination module is used to determine a second target area corresponding to the first target area in the second image;

[0039] The defect type determination module is used to determine the defect type of the object to be measured using the second image based on the first defect area and the second target area.

[0040] According to another aspect of an embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein: the memory is used to store a computer program; and the processor is used to execute the computer program to implement the above-mentioned defect type detection method.

[0041] According to another aspect of the embodiments of the present invention, a storage medium is provided, storing a computer program / instruction, wherein the computer program / instruction is used to execute the above-mentioned defect type detection method when running.

[0042] According to another aspect of an embodiment of the present invention, a computer program product is provided, including computer program instructions, and the computer program instructions are used to execute the above-mentioned defect type detection method when running.

[0043] The above scheme provided by the embodiment of the present invention can determine the first target area and the first defect area in the first image. Then, based on the first position information of the first target area in the first image, a second image of the object to be measured is obtained. Then, in the second image, the second target area corresponding to the first target area is determined. Finally, based on the first defect area and the second target area, the defect type of the object to be measured is determined using the second image. The above scheme can first process the first image with lower clarity, and then obtain the second image with higher clarity, which can reduce the total amount of processing of the second image, thereby reducing the difficulty of data processing, reducing the time cost of data processing, and helping to improve the efficiency of determining the defect type. In addition, judging the defect type of the object to be measured in combination with the second image with higher clarity is conducive to improving the accuracy of the defect type. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other purposes, features and advantages of the present invention will become more apparent by describing the embodiments of the present invention in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0045] Figure 1 A schematic flow chart of a defect type detection method according to an embodiment of the present invention is shown;

[0046] Figure 2 A schematic diagram of a defective wafer die according to an embodiment of the present invention is shown;

[0047] Figure 3A schematic diagram showing a defect type detection method according to an embodiment of the present invention;

[0048] Figure 4 shows a second image captured by a color camera according to an embodiment of the present invention;

[0049] Figure 5 A schematic diagram showing a defect type detection device according to an embodiment of the present invention; and

[0050] Figure 6 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical scheme and advantages of the present invention more obvious, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.

[0052] In the field of component manufacturing, if defective industrial components (such as wafers) are packaged and tested, it will waste manpower and material resources, so it is necessary to inspect industrial components so that unqualified industrial components can be detected and eliminated. However, there may be a variety of defects on industrial components. For example, taking the wafer as an example, there may be defects such as foreign matter and cracks on the wafer. For another example, taking the wafer grains on the wafer as an example, there may be defective needle marks such as pin marks out of bounds and too many pin marks on the wafer grains. The above-mentioned pin marks are marks left when the pin test probe contacts the wafer grains during the pin test of the wafer grains. Under normal circumstances, the pin mark will not affect the test results and performance of the wafer grains, but when the pin mark is offset or the pin mark is too deep, it may cause inaccurate pin test results and affect the performance of the wafer grains (for example: the pin mark is too deep to pierce the solder joint), thereby reducing the overall yield of the wafer.

[0053] With the continuous advancement of semiconductor technology and the development of industrial component manufacturing technology, the detection of defects on industrial components has become an indispensable part of the industrial component manufacturing process. With the continuous advancement of semiconductor technology, the device density and integration of industrial components are increasing, and the requirements for the detection of defects on industrial components are also getting higher and higher.

[0054] In the related art, the detection of defects on industrial components mainly relies on traditional image processing technology. However, the above detection method is not only inefficient, but also has a high error rate and cannot identify the type of defects on industrial components. Therefore, how to improve the efficiency and accuracy of detecting defects on industrial components and identify the types of defects on industrial components has become a problem that the semiconductor industry needs to solve urgently.

[0055] To at least partially address the above issues, see Figure 1 , Figure 1 A schematic flow chart of a defect type detection method according to an embodiment of the present invention is shown, Figure 1 , the above method may include steps S110 to S140.

[0056] In step S110 , a first target area and a first defect area in a first image of the object to be tested are determined.

[0057] The object to be tested may be an industrial component that needs to be quality inspected, such as a circuit board, a wafer, a die in a wafer, etc.

[0058] The first image may be a static image or any video frame in a dynamic video. The first image may be an original image acquired by an image acquisition device (for example, an original image acquired by an image sensor in a camera), or an image obtained after preprocessing the original image (such as digitization, normalization, smoothing, etc.).

[0059] The first target area is the area in the first image corresponding to the target of interest in the quality inspection of the object to be tested. Here, taking the object to be tested as a wafer as an example, the first target area can be the area of ​​interest in the wafer. Specifically, for example, the area of ​​interest in the wafer is a surface area, a trimming area, etc.

[0060] The first defect area is the area in the first image corresponding to the part of the object to be tested that may have defects. For example, the first defect area may be the position in the first image corresponding to the defective part of the wafer grain such as foreign matter, cracks, pin marks, etc. At least a part of the first defect area is in the first target area. For example, the first defect area may be located in the first target area. For a specific example, taking the first target area as a surface area, the first defect area may be the area in the first image corresponding to foreign matter or cracks in the surface area. For another example, the first defect area may be located at the boundary of the first target area.

[0061] The first target region and the first defect region can be obtained through the trained region division model. The embodiment of the present invention provides a training method of a region division model for reference.

[0062] In some embodiments, the first image can be input into a trained detection model (e.g., a trained convolutional neural network, a generative adversarial network, etc.) to output a first target area and a first defect area in the first image. The training process of the detection model may include: obtaining a first training data set. Iteratively training the detection model through the first training data set, and adjusting the model parameters of the detection model through the loss function until the training is completed. The above-mentioned first training data set includes a plurality of images, each image is marked with a first target area and a first defect area (the corresponding labels of the two may be different). The conditions for the completion of the above-mentioned training may be that the loss value calculated by the loss function tends to be stable, the number of iterations reaches a preset number, etc. Based on the above-mentioned training process, a detection model can be obtained to determine the first target area and the first defect area in the first image.

[0063] In other embodiments, the first target area and the first defect area in the first image may be determined using conventional image processing methods. For example, the first target area and the first defect area in the first image may be determined based on pixel distribution characteristics in the first image. For example, the first image may be segmented using threshold segmentation, seed growth, edge detection, and other methods based on the difference between the pixel values ​​of pixels in the first target area and the pixel values ​​of pixels in other areas of the first image, the difference between the pixel values ​​of pixels in the first defect area and the pixel values ​​of pixels in other areas, etc., to determine the first target area and the first defect area in the first image.

[0064] In step S120, a second image of the object to be measured is acquired based on first position information of the first target area in the first image.

[0065] The first position information is used to indicate the position of the first target area in the first image. In one example, the first position information may include at least one set of position information. For example, the first position information may include the coordinates corresponding to the pixel located at the center of the first target area in the first image, and may also include the coordinates corresponding to each vertex of the first target area in the first image. The pixel second image is an image reacquired according to the first position information, in which there is an area corresponding to the first target area in the first image. The second image may be a static image or any video frame in a dynamic video. The second image may be an original image acquired by an image acquisition device (for example, an original image acquired by an image sensor in a camera), or may be an image obtained after preprocessing the original image (such as digitization, normalization, smoothing, etc.).

[0066] The second image has a higher definition than the first image. For example, the first image is an image captured by a camera with fewer pixels in the image sensor, while the second image may be an image captured by a camera with more pixels in the image sensor. For another example, the first image is a grayscale image, and the second image is a color image. For example, a black-and-white low-magnification camera is used to take a picture of the object to be measured to obtain a first image. When determining the first position information of the first target area in the first image, a high-magnification color camera determines the position of the first target area according to the first position information, and then a color image is captured by the high-magnification color camera according to the position of the first target area as the second image.

[0067] The image acquisition device of the first image and the image acquisition device of the second image may be calibrated in the same world coordinate system to establish a correspondence between pixels in the first image and pixels in the second image.

[0068] In one embodiment, a color image including a plurality of objects to be measured may be captured by a color camera, and then based on the position of the first target area in the first image, the area at the corresponding position in the color image may be cropped to obtain a second image.

[0069] In step S130 , a second target area corresponding to the first target area is determined in the second image.

[0070] The second target area may be an area in the second image corresponding to the first target area in the first image. For example, when the first target area corresponds to the surface area A on the wafer die A in the wafer, the second target area also corresponds to the surface area A on the wafer die A in the wafer. For another example, when the first target area corresponds to the trimming area A on the wafer die A in the wafer, the second target area also corresponds to the trimming area A on the wafer die A in the wafer, that is, the spatial position corresponding to the first target area in the object to be measured is the same as the spatial position corresponding to the second target area in the object to be measured.

[0071] The second target area may be determined based on the first target area. For example, the second target area corresponding to the first target area in the second image may be determined based on the correspondence between the pixels in the first image and the pixels in the second image as described above.

[0072] In step S140, based on the first defect area and the second target area, the defect type of the object to be tested is determined using the second image.

[0073] The defect type of the object to be tested can be determined based on the industry's qualified standards for the object to be tested. For example, when the object to be tested is a wafer, if it is specified that a wafer with a needle mark or scratch in the pad area is a substandard product, then the defect type of the object to be tested may include the presence of a needle mark or scratch in the pad. For another example, if it is specified that a wafer with an opaque foreign body in the surface area is a substandard product, then the defect type of the object to be tested may include the presence of an opaque foreign body in the surface area. For another example, if it is specified that a wafer with a transparent foreign body in the trimming area is a substandard product, then the defect type of the object to be tested may include the presence of a transparent foreign body in the trimming area. See also Figure 2 , Figure 2 A schematic diagram of a defective wafer die according to an embodiment of the present invention is shown, combined with Figure 2 As shown, the defect types may include: pin mark out of bounds, pin mark internal scratches, pin mark internal scratches, internal scratches, external scratches, internal cracks, external cracks, light-colored interior (there is a light-colored area in the pad area), transparent foreign matter exterior (there is a transparent foreign matter at the boundary or outside the pad area), solid foreign matter exterior (there is a solid foreign matter at the boundary or outside the pad area), transparent foreign matter interior (transparent foreign matter in the pad area), solid foreign matter interior, etc. (there is a solid foreign matter in the pad area).

[0074] The defect type of the object to be tested corresponding to the second image can be determined based on the relationship between the first defect area and the second target area. For example, the surrounding situation of the second target area in the second image can be determined based on the positional relationship between the first defect area and the first target area in the first image, and whether there is really a defect in the first defect area can be determined based on the second image with higher definition. For example, if the first defect area in the first image is located on the boundary line of the first target area, it can be determined that there is also a possible defect area at the same position of the second target area. At this time, it can be determined whether there is a defect at this position based on the second image with high definition. If there is a defect, the defect type of the object to be tested can be determined based on the specific situation of the corresponding position of the first defect area in the second image. For example, if the first target area is a surface area on a wafer, and the first defect area is located on the boundary line A of the first target area, it can be checked whether there is a defect at the corresponding position on the boundary line A' corresponding to the boundary line A in the second target area in the second image. If there is a defect, the defect type of the object to be tested is determined based on the defect in the second image. If the defect in the second image is determined to be an opaque foreign body, the defect type of the object to be tested may be the presence of an opaque foreign body in the surface area.

[0075] The defect type detection method provided by the embodiment of the present invention can determine the first target area and the first defect area in the first image. Then, based on the first position information of the first target area in the first image, a second image of the object to be measured is obtained. Then, in the second image, the second target area corresponding to the first target area is determined. Finally, based on the first defect area and the second target area, the defect type of the object to be measured is determined using the second image. The above scheme can first process the first image with lower clarity, and then obtain the second image with higher clarity, which can reduce the total amount of processing of the second image, thereby reducing the difficulty of data processing, reducing the time cost of data processing, and helping to improve the efficiency of determining the defect type. In addition, judging the defect type of the object to be measured in combination with the second image with higher clarity is conducive to improving the accuracy of the defect type.

[0076] Exemplarily, the above step S110 also includes steps S210 to S230.

[0077] In step S210, an initial image is acquired.

[0078] See also Figure 3 , Figure 3 A schematic diagram showing a defect type detection method according to an embodiment of the present invention is shown, Figure 3 The initial image may include multiple initial wafer grains. The initial image may be a static image or any video frame in a dynamic video. The initial image may be an original image captured by an image acquisition device (for example, an original image captured by an image sensor in a camera), or an image obtained after preprocessing the original image (such as digitization, normalization, smoothing, etc.). For example, the initial image may be acquired by the first image acquisition device mentioned above.

[0079] In step S220 , it is determined whether each initial wafer in the initial image meets a preset rule.

[0080] The initial wafer that meets the preset rules may be a wafer that may have defects. The initial image may be divided into a plurality of initial sub-images, and the initial sub-images may be screened by calculating the clarity, the number of pixels in the dark area, etc. For example, an initial sub-image with lower clarity and more dark areas (which may be foreign matter or other obstructions) may be used as the first image.

[0081] In step S230, an initial wafer that meets the preset rules is used as an object to be tested, and a first image of the object to be tested is generated based on the initial image. For example, the initial sub-image mentioned above can be cropped by template matching or a machine learning model to obtain a first image of the object to be tested. For another example, the image of the initial wafer grain that meets the preset rules is reacquired. For a specific example, when it is determined that the initial wafer grain A meets the preset rules, the initial wafer grain A is re-photographed by a camera, and the shooting result is used as the first image of the initial wafer grain A.

[0082] According to the above scheme of the embodiment of the present invention, the initial wafer particles in the initial image can be screened and only the first image of the initial wafer particles that meet the preset rules can be obtained, so that a part of the initial wafers can be screened out, the difficulty of executing subsequent defect type detection methods can be reduced, and unnecessary detection time for initial wafers that do not meet the preset rules can be prevented, thereby improving the detection efficiency of defect type detection methods.

[0083] Exemplarily, the object to be tested is a wafer, and the targets of interest in the quality inspection of the object to be tested include at least pads on the wafer.

[0084] When the object to be tested is a wafer, the objects of interest in the quality inspection of the object to be tested include at least the pads on the wafer, and may also include the surface area, trimming area, etc. on the wafer.

[0085] In the case where the object of interest during quality inspection is the pad on the wafer, the above-mentioned first image is an image including the pad on the wafer. The first target area is the area corresponding to the pad in the first image, and the first defect area is the area on the pad in the first image where defects may exist. For example, there may be foreign matter at position A on the pad, and the first defect area includes the area corresponding to position A in the first image. For another example, there may be a pin mark at position B on the pad, and the first defect area includes the area corresponding to position B in the first image. For another example, there may be foreign matter at position A on the pad and there may be a pin mark at position B on the pad, and the first defect area includes the area corresponding to position A in the first image and the area corresponding to position B in the first image.

[0086] When the object of interest during quality inspection is the pad on the wafer, the first position information is the position information of the pad in the first image. Based on the first position information corresponding to the pad (in this example, the first target area) in the first image, a higher definition image with the pad (in this example, the second image) can be obtained. Based on the first position information corresponding to the pad in the first image, a second target area is determined in the second image, and the second target area is the area corresponding to the pad in the second image.

[0087] According to the above scheme of the embodiment of the present invention, the quality inspection of the pads in the wafer grains can be carried out, which can not only efficiently, quickly and accurately determine whether the pads have defects, but also reduce the difficulty of data processing when performing quality inspection on the pads in the wafer grains.

[0088] Exemplarily, step S110 of determining the first target area and the first defect area in the first image of the object to be measured may include: steps S111 to S113.

[0089] In step S111, the first image is matched with the image template of the object to be measured to obtain a region in the first image that successfully matches the image template as a first target region.

[0090] The image template of the object to be tested can be an image of the portion of the object to be tested that corresponds to the target of interest during quality inspection, or it can be an image that includes the entire object to be tested. For example, when the object to be tested is a wafer, and the target of interest during quality inspection is a pad on the wafer, the image template of the object to be tested can be a reference image of a wafer without any defects, or it can be an image that only includes a single pad without any defects.

[0091] Taking the case where the image template of the object to be tested is an image of the part of the object to be tested corresponding to the target of interest during quality inspection as an example, the first image and the image template of the object to be tested can be preprocessed, for example, grayscale conversion, filtering and denoising, etc., to improve the accuracy and efficiency of matching. Then select a suitable matching method according to the application requirements, such as matching based on square difference (SQDIFF), normalized square difference (SQDIFF_NORMED), and correlation coefficient (CCORR). The above matching process includes: placing the image template of the object to be tested at the upper left corner of the first image, then sliding on the first image, and comparing each possible position. Then, for each position in the first image, the similarity or difference between the image template of the object to be tested and the corresponding area of ​​the first image is calculated, which can be a pixel-by-pixel comparison, and finally a result matrix is ​​obtained. Among them, each element in the result matrix represents the matching degree of the image template of the object to be tested at the corresponding position in the first image. When the result matrix is ​​obtained, the cv2.minMaxLoc() function can be used to analyze the result matrix to find the position with the highest matching degree. The position with the highest matching degree can be used as the first target area.

[0092] In step S112, based on the image template, a foreground target area in the first target area is determined, and the foreground target area that meets the preset conditions is used as the first defect area in the first target area.

[0093] The foreground target area in the first target area may be an area in the first target area whose difference from the image template is greater than a threshold. Here, the above-mentioned difference is represented by the absolute value of the difference between the pixel values ​​of the pixels. The pixel values ​​of each pixel in the first target area are respectively subtracted from the pixel values ​​of the pixels at the corresponding position of the pixel in the image template to obtain the pixel difference corresponding to each pixel in the first target area, and the pixels in the first target area whose absolute value of the pixel difference is greater than the threshold are clustered to obtain at least one cluster area. The cluster area can be used as the foreground target area. Then, it is determined whether each foreground target area meets the preset conditions, and the foreground target area that meets the preset conditions is used as the first defect area in the first target area. The threshold and preset conditions in the above text can be determined according to the actual situation. The preset conditions can be that the area of ​​the foreground target area is greater than the preset area value, and the distance between the edge of the foreground target area and the edge of the first target area is less than the preset distance value. Among them, the preset area value and the preset distance value can be determined according to the actual situation. For example, the first target area includes pixels D1, D2, D3...pixel D n , where n is the total number of pixels in the first target area. Subtract the pixel values ​​of each of the above pixels from the pixel values ​​of the pixels at the positions corresponding to the pixels in the image template. If the pixel at the position corresponding to pixel D1 in the image template is D1', then the pixel value of pixel D1 is subtracted from the pixel value of pixel D1' to obtain the pixel difference value X1 corresponding to pixel D1. Similarly, pixel difference values ​​X1, pixel difference values ​​X2, pixel difference values ​​X3, pixel difference values ​​X4... pixel difference values ​​X n When the threshold is A (A is a value greater than zero), the pixel difference X1 to the pixel difference X n , determine the pixel difference X1 to the pixel difference X in the first image n The absolute value of the pixel difference in the pixel is greater than A (i.e., the threshold) corresponding to the pixel difference. 10 To pixel difference X 20 And the pixel difference X 40 To pixel difference X 50 If both are greater than A, pixel D in the first image 10 To pixel D 20 And Pixel D 40 To pixel D 50 Clustering is performed respectively to obtain clustering results Q1 and Q2, and then the area corresponding to the clustering result Q1 in the first target area is used as the foreground target area R1, and the area corresponding to the clustering result Q2 in the first target area is used as the foreground target area R2. If the preset condition is that the area of ​​the foreground target area is greater than the preset area value, then when the area of ​​the foreground target area R1 is greater than the preset area value, the foreground target area R1 is used as the first defect area corresponding to the first image.

[0094] In step S113, the first image is compared with the reference image, and based on the comparison result, a first defect area at least partly located outside the first target area is determined.

[0095] The reference image can be a reference image with the first target area masked and without any defects in the object to be tested. For example, if the object to be tested is a wafer, a wafer image without defects such as pin marks, foreign matter, scratches, etc. can be selected, and the first target area in the image can be masked, so that the area outside the first target area can be targeted for detection.

[0096] When comparing the first image with the reference image, the pixel difference between each pixel in the first image and its corresponding pixel in the reference image can be calculated, and when the absolute value of the calculated result is greater than a threshold, the pixels in the first image whose absolute value of the pixel difference is greater than the threshold are clustered to obtain at least one clustering result. The specific clustering process can be referred to above, and the embodiment of the present invention will not be described in detail here.

[0097] According to the above solution of the embodiment of the present invention, the first target area in the first image can be determined by matching the first image with the image template, and then the first defect area in the first image can be determined according to the image template, so that the first target area and the second target area in the first image can be determined efficiently and accurately.

[0098] According to the above scheme of the embodiment of the present invention, the first image can be matched with the image template of the object to be tested to obtain the area in the first image that successfully matches the image template as the first target area. Then, based on the image template, the foreground target area in the first target area is determined, and the foreground target area that meets the preset conditions is used as the first defect area in the first target area. The first image is compared with the reference image, and based on the comparison result, at least a part of the first defect area outside the first target area is determined. The above scheme can determine the first target area and the first defect area in the first target area through the image template; at least a part of the first defect area outside the first target area can be determined through the reference image, thereby achieving comprehensive positioning of the first defect area. On the one hand, the computing power requirement of the above scheme is relatively small; on the other hand, the accuracy of the first defect area determined by the above scheme is also relatively high.

[0099] Exemplarily, the first image is acquired by a first image acquisition device, and the second image is acquired by a second image acquisition device. Step S120 acquires the second image of the object to be measured based on the first position information of the first target area in the first image, including: steps S121 to S123.

[0100] In step S121, based on the mapping relationship between the image coordinate system of the first image acquisition device and the world coordinate system, the first position information is converted into second position information in the world coordinate system.

[0101] The first image acquisition device may be a digital camera, a scanner, an industrial camera, a depth camera, etc. The image acquisition device of the first image may be calibrated in a world coordinate system to establish a mapping relationship between the image coordinate system of the first image acquisition device and the world coordinate system.

[0102] In step S122, based on the mapping relationship between the image coordinate system of the second image acquisition device and the world coordinate system and the second position information, the shooting position of the second image acquisition device is adjusted.

[0103] The second image acquisition device can also be a digital camera, a scanner, an industrial camera, a depth camera, etc. The image acquisition device of the second image can be calibrated in the world coordinate system mentioned above to establish a mapping relationship between the image coordinate system of the second image acquisition device and the world coordinate system mentioned above. Accordingly, a mapping relationship can be established between the image coordinate system of the first image acquisition device and the image coordinate system of the second image acquisition device. The shooting position of the second image acquisition device can be executed by the position adjustment device of the second image acquisition device, and it can be moved in the world coordinate system.

[0104] In step S123, the second image is captured by the adjusted second image capture device.

[0105] When the second image acquisition device is adjusted to a suitable shooting position, the target of interest in the object to be tested during quality inspection can be photographed to obtain a second image. For example, if the target of interest in the object to be tested during quality inspection is pad A on a wafer, the second image can include an image area corresponding to pad A.

[0106] According to the above scheme of the embodiment of the present invention, the first position information can be converted into the second position information in the world coordinate system based on the mapping relationship between the image coordinate system of the first image acquisition device and the world coordinate system. Then, based on the mapping relationship between the image coordinate system of the second image acquisition device and the world coordinate system and the second position information, the shooting position of the second image acquisition device is adjusted. Finally, the second image is acquired by the adjusted second image acquisition device. The above scheme can use the second image acquisition device to take targeted photos of the object to be tested, which is conducive to improving the accuracy of defect type generation.

[0107] Exemplarily, the detection method of the embodiment of the present invention may further include: determining a second defect area corresponding to the first defect area in the second image based on a position mapping relationship between the second image and the first image.

[0108] The second defect area is obtained based on the mapping of the first defect area, and the first defect area is obtained by detecting the same wafer, so the influence of defects in other wafers or other target areas on the obtained defect type can be avoided. In one example, the position information of the first defect area in the first image can be converted into position information in the world coordinate system, and then converted into position information in the image coordinate system in the second image to obtain the second defect area corresponding to the first defect area.

[0109] If the detection method further includes: determining a second defect area in the second image corresponding to the first defect area based on a position mapping relationship between the second image and the first image, then step S140 determines the defect type of the object to be detected using the second image based on the first defect area and the second target area, and may also include: steps S141 to S143.

[0110] In step S141, the second image is input into an instance segmentation model to obtain instance regions and instance categories corresponding to the instance regions.

[0111] The instance segmentation model can be obtained in the following manner: obtaining a second training data set, iteratively training the detection model through the second training data set, and adjusting the model parameters of the instance segmentation model through the loss function until the training is completed. The above-mentioned second training data set includes multiple images, each image is marked with an instance area and an instance category corresponding to the instance area. The above-mentioned instance category may include the needle marks, cracks, foreign matter, etc. mentioned above. The conditions for the completion of the above-mentioned training may be that the loss function value calculated by the loss function tends to be stable, the number of iterations reaches a preset number, etc. Based on the above-mentioned training process, the instance segmentation model can be obtained to determine the instance area in the second image and the instance category corresponding to the instance area.

[0112] The second image can be obtained by a second image acquisition device according to the first position information of the first image. If the second image is acquired by the second image acquisition device, the second image acquisition device may capture other target areas except the second target area. In some embodiments, in order to reduce the redundant parts of the second image except the second target area, the part with less correlation with the second target area can be cut off. For example, the second target area is expanded according to a preset expansion ratio, and the expansion result is used as a new second image, and the new second image is input into the instance segmentation model. For another example, the union area of ​​the second target area and the second defect area is used as a new second image, and the new second image is input into the instance segmentation model.

[0113] Combination Figure 3, a grayscale image can be captured by a first image acquisition device, in which there may be multiple wafer grains, and then the first image is obtained by cropping each wafer grain area, and the first target area and the first defect area in the first image are determined through the relevant steps of step S111 to step S113 above. According to the position of the first target area in the first image, a color image is captured by a second image acquisition device. Cropping is performed in the manner described above, and the cropped image is input into the instance segmentation model to obtain the instance area. The instance area that successfully matches the second target area and the second defect area is logically judged to obtain the defect type. For example, it is determined whether the number of successfully matched instance areas is too large. The instance area obtained by instance segmentation and the instance category corresponding to the instance area can be related to the defects that may exist in the target that the object to be tested is concerned about during quality inspection. For example, if the object to be tested is a wafer, and the target of quality inspection in the object to be tested is the pad on the wafer, then the instance region output by the instance segmentation model may include the region in the second image indicating the presence of a pin mark on the pad (the instance category in this example is a pin mark), the region in the second image indicating the presence of a crack on the pad (the instance category in this example is a crack), and the region in the second image indicating the presence of a foreign object on the pad (the instance category in this example is a foreign object). In the above examples, the instance categories corresponding to the instance regions output by the instance segmentation model may include cracks, pin marks, and foreign objects.

[0114] See also Figure 4 , Figure 4 FIG. 2 shows a second image captured by a color camera according to an embodiment of the present invention. Figure 4 When the first image acquisition device is a grayscale camera and the second image acquisition device is a color camera, the method of determining the second defect area and the second target area through the first defect area and the first target area can reduce the probability of the second defect area and the second target area being difficult to identify in the second image due to problems such as light scattering and shadow occlusion, thereby improving the accuracy of determining the second defect area in the second image.

[0115] In step S142, the instance region is positionally matched with the second defect region to determine a successfully matched instance region.

[0116] In some embodiments, the instance region and the second defect region may be positionally matched, and if there is an intersection between the matching positions of the two, it may be considered that the two are successfully matched.

[0117] In other embodiments, when the instance segmentation model outputs an instance area in the second image, if the instance area of ​​the second image is the area corresponding to the pin mark located inside the second target area, and the second defect area in the second image is also the area corresponding to the pin mark located inside the second target area, then the area corresponding to the pin mark located inside the second target area (i.e., the instance area) is determined as a successfully matched instance area.

[0118] In step S143, the defect type of the object to be tested is determined based on the second target area, the successfully matched instance area, and the instance category corresponding to the successfully matched instance area.

[0119] The defect type of the object to be tested can be one or more, which can be determined according to the actual situation. Take the case where the successfully matched instance area is a pin mark on the boundary of the second target area as an example. According to the second target area, it can be known that the successfully matched instance area is located on the boundary line of the second target area, and the instance category corresponding to the successfully matched instance area is a pin mark, then it can be determined that the defect type of the object to be tested is a pin mark out of bounds. For another example, the successfully matched instance area is a pin mark on the boundary of the second target area, and there is a foreign object inside the pad. According to the second target area, it can be known that the successfully matched instance area is located on the boundary line and inside of the second target area, and the instance categories corresponding to the two successfully matched instance areas are pin marks and foreign objects, respectively, then it can be determined that the defect type of the object to be tested is a pin mark out of bounds and a foreign object inside the pad.

[0120] According to the above scheme of the embodiment of the present invention, the second defect area in the second image can be obtained directly from the first defect area in the first image according to the position mapping relationship between the second image and the first image, thereby reducing the difficulty of processing the second image. In addition, the method of determining the second defect area and the second target area through the first defect area and the first target area can reduce the probability of the second defect area and the second target area in the second image being difficult to identify due to problems such as light scattering and shadow occlusion, thereby improving the accuracy of determining the second defect area in the second image.

[0121] Exemplarily, the above step S143 also includes: step S1431 to step S1432.

[0122] In step S1431, it is determined whether the successfully matched instance region exceeds the boundary of the second target region to obtain a determination result.

[0123] Whether the boundary is exceeded can be determined by comparing the coordinates of the successfully matched instance area and the second target area.

[0124] In step S1432, based on the judgment result and the instance category corresponding to the successfully matched instance area, the defect type of the object to be tested is determined.

[0125] Combination Figure 3 , Figure 3 The second defect region in is one of the regions that successfully matches the instance region. If the instance category of the second defect region is a pin mark, since it intersects with the boundary of the second target region, the defect type can be determined as a pin mark out of bounds. If the instance category of the second defect region is a foreign body, since it intersects with the boundary of the second target region, the defect type can be determined as a foreign body existing near the boundary of the target region.

[0126] The defect type of the object to be tested can also be determined based on the location, number, size of the successfully matched instance area and the instance category corresponding to the successfully matched instance area. For example, the instance category corresponding to the successfully matched instance area is a pin mark. If the number of successfully matched instance areas is greater than a preset number, and there is a successfully matched instance area that exceeds the boundary of the second target area, then the defect type of the object to be tested is a pin mark number exceeding the limit and a pin mark out of bounds. The preset number can be set according to actual conditions.

[0127] According to the above scheme of the embodiment of the present invention, the defect type of the object to be tested can be more accurately determined based on the positional relationship between the successfully matched instance area and the boundary of the second target area, thereby increasing the defect type of the object to be tested that can be determined by the defect type detection method provided by the embodiment of the present invention.

[0128] Exemplarily, the successfully matched instance regions in the above text include one or more, and step S1432 further includes: step S14321 to step S14322.

[0129] In step S14321, based on the judgment result and the defect type corresponding to each successfully matched instance region, the defect level corresponding to each successfully matched instance region is determined.

[0130] There are at least two defect types with different defect levels. Taking the object to be tested as a wafer, and the target of quality inspection in the object to be tested as the pad on the wafer as an example, the defect types that may exist in the pad include but are not limited to pin print out of bounds, opaque foreign matter inside, opaque foreign matter outside, transparent foreign matter inside, transparent foreign matter outside, and excessive pin print quantity, etc., and the above defect types can be divided into defect levels according to the severity of the impact on the pad function according to actual conditions. For example, the defect level corresponding to the defect type of pin print out of bounds, opaque foreign matter inside, and transparent foreign matter inside can be set to the highest level, the defect level corresponding to the defect type of opaque foreign matter outside and transparent foreign matter outside can be set to the secondary level, and the defect level corresponding to the excessive pin print quantity can be set to the lowest level. The above is only an exemplary description and is not used to limit the defect level corresponding to each successfully matched instance area. The number of successfully matched instance areas corresponding to each defect level can be at least one.

[0131] In step S14322, the defect type of the object to be tested is determined based on the defect level corresponding to each successfully matched instance region.

[0132] In the case that the object to be tested corresponds to only one successfully matched instance region, the successfully matched instance region is the defect type corresponding to the object to be tested.

[0133] In the case where the object to be tested corresponds to multiple successfully matched instance areas, if there are at least two successfully matched instance areas corresponding to different defect types, then according to the defect level corresponding to each successfully matched instance area, the defect level corresponding to the multiple successfully matched instance areas that meets the preset screening conditions can be used as the target defect level, and the defect type corresponding to the target defect level can be used as the defect type of the object to be tested. For example, the object to be tested is a wafer, and the target of quality inspection in the object to be tested is the pad on the wafer. If the defect types corresponding to the multiple successfully matched instance areas corresponding to the object to be tested include pin print out of bounds, opaque foreign matter inside, transparent foreign matter outside, and the number of pin prints exceeds the standard, then the defect level corresponding to each defect type is first determined. Here, pin print out of bounds, opaque foreign matter inside is the highest defect level, transparent foreign matter outside is the secondary defect level, and the number of pin prints exceeds the standard is the lowest defect level. If the preset screening condition is to screen out the defect type with the highest defect level, pin print out of bounds and opaque foreign matter inside are used as the defect type of the object to be tested. If the preset screening condition is to screen out the defect types with the highest two defect levels among the defect types corresponding to at least one successfully matched instance area corresponding to the object to be tested, the pin print out of bounds, the presence of opaque foreign matter inside, and the presence of transparent foreign matter outside are taken as the defect types of the object to be tested. Alternatively, by setting the preset condition, the defect types corresponding to all successfully matched instance areas are taken as the defect types of the object to be tested.

[0134] According to the above scheme of the embodiment of the present invention, the defect types of the object to be tested to be output can be screened by setting the defect level, so that the defect types of the object to be tested that are most suitable for the actual application can be output preferentially. For example, when the defect level is divided according to the degree of influence on the product performance, the defect type detection method provided by the embodiment of the present invention can preferentially output the defect type that most affects the performance of the object to be tested.

[0135] See also Figure 5 A schematic diagram of a defect type detection device according to an embodiment of the present invention is shown, Figure 5 According to another aspect of the present invention, a defect type detection device 300 is also provided, and the detection device includes: a first determination module 310, a second image acquisition module 320, a second determination module 330 and a defect type determination module 340.

[0136] The first determination module 310 is used to determine the first target area and the first defect area in the first image of the object to be tested, wherein the first target area is the area corresponding to the target of interest in the quality inspection of the object to be tested in the first image, and at least a part of the first defect area is in the first target area. The second image acquisition module 320 is used to acquire the second image of the object to be tested based on the first position information of the first target area in the first image, wherein the clarity of the second image is higher than that of the first image. The second determination module 330 is used to determine the second target area corresponding to the first target area in the second image. The defect type determination module 340 is used to determine the defect type of the object to be tested using the second image based on the first defect area and the second target area.

[0137] Exemplarily, the detection device 300 further includes: a second defect area determination module.

[0138] The second defect area determination module is used to determine a second defect area in the second image corresponding to the first defect area based on a position mapping relationship between the second image and the first image.

[0139] The defect type determination module 340 may include: an instance segmentation module, a position matching module, and a first type determination module.

[0140] The instance segmentation module is used to input the second image into the instance segmentation model to obtain the instance region and the instance category corresponding to the instance region. The position matching module is used to positionally match the instance region with the second defect region to determine the successfully matched instance region. The first type determination module is used to determine the defect type of the object to be tested based on the second target region, the successfully matched instance region and the instance category corresponding to the successfully matched instance region.

[0141] Exemplarily, the first type determination module may include: a judgment result generation module and a defect type first determination module.

[0142] The judgment result generation module is used to judge whether the successfully matched instance area exceeds the boundary of the second target area to obtain a judgment result. The defect type first determination module is used to determine the defect type of the object to be tested based on the judgment result and the instance category corresponding to the successfully matched instance area.

[0143] Exemplarily, the successfully matched instance regions include one or more, and the first defect type determination module may include: a defect level determination module and a second defect type determination module.

[0144] The defect level determination module is used to determine the defect level corresponding to each successfully matched instance region based on the judgment result and the defect type corresponding to each successfully matched instance region, wherein at least two defect types have different defect levels. The defect type second determination module is used to determine the defect type of the object to be tested based on the defect level corresponding to each successfully matched instance region.

[0145] Exemplarily, the first image is acquired by a first image acquisition device, and the second image is acquired by a second image acquisition device. The second image acquisition module 320 may include: a first position information conversion module, a position adjustment module, and a second image acquisition module.

[0146] The first position information conversion module is used to convert the first position information into the second position information in the world coordinate system based on the mapping relationship between the image coordinate system of the first image acquisition device and the world coordinate system. The position adjustment module is used to adjust the shooting position of the second image acquisition device based on the mapping relationship between the image coordinate system of the second image acquisition device and the world coordinate system and the second position information. The second image acquisition module is used to acquire the second image through the adjusted second image acquisition device.

[0147] Exemplarily, the first determination module 310 may include: a first matching module, a second matching module, and a third matching module.

[0148] The first matching module is used to match the first image with the image template of the object to be tested, so as to obtain the area in the first image that successfully matches the image template as the first target area. The second matching module is used to determine the foreground target area in the first target area based on the image template, and to use the foreground target area that meets the preset conditions as the first defect area in the first target area. The third matching module is used to compare the first image with the reference image, and based on the comparison result, determine the first defect area that is at least partially located outside the first target area.

[0149] Exemplarily, the above detection device further includes: an initial image acquisition module, a screening module and a to-be-detected object determination module.

[0150] The initial image acquisition module is used to acquire an initial image, wherein the initial image includes a plurality of initial wafers. The screening module is used to determine whether each initial wafer in the initial image meets the preset rules. The object to be tested determination module is used to take the initial wafers that meet the preset rules as the object to be tested, and generate a first image of the object to be tested based on the initial image.

[0151] Exemplarily, the object to be tested is a wafer, and the targets of interest in the quality inspection of the object to be tested include at least pads on the wafer.

[0152] According to another aspect of the present invention, an electronic device is provided. Figure 6 FIG. 2 shows a schematic block diagram of an electronic device according to an embodiment of the present invention. Figure 6 As shown, the electronic device 400 includes a processor 410 and a memory 420. The memory 420 stores a computer program. When the computer program instructions are executed by the processor 410, they are used to execute the above-mentioned defect type detection method.

[0153] In addition, according to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, and when the program instructions are executed by a computer or a processor, the computer or the processor executes the corresponding steps of the above-mentioned defect type detection method according to the embodiment of the present invention, and is used to implement the corresponding modules in the above-mentioned defect type detection device according to the embodiment of the present invention or the corresponding modules in the above-mentioned electronic device. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0154] According to yet another aspect of the present invention, a computer program product is provided, comprising computer program instructions, which, when executed by a computer or a processor, enable the computer or the processor to execute corresponding steps of the above defect type detection method.

[0155] A person skilled in the art may understand the specific implementation scheme of the above-mentioned electronic device and storage medium by reading the above-mentioned description of the detection method of the defect type, and for the sake of brevity, it will not be described here in detail.

[0156] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as required by the appended claims.

[0157] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0158] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0159] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0160] Similarly, it should be understood that in order to streamline the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present invention should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features explicitly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.

[0161] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0162] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0163] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some modules in the defect type detection device according to an embodiment of the present invention. The present invention may also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0164] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0165] The above is only a specific embodiment of the present invention or an explanation of a specific embodiment. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A defect type detection method, characterized in that: The method comprises: Determine a first target region and a first defect region in a first image of the object to be tested, wherein the first target region is a region corresponding to a target of interest in the object to be tested during quality inspection in the first image, and at least a portion of the first defect region is in the first target region; Based on the first position information of the first target area in the first image, acquiring a second image of the object to be measured, wherein the clarity of the second image is higher than that of the first image; In the second image, determining a second target area corresponding to the first target area; Based on the first defect area and the second target area, the defect type of the object to be measured is determined using the second image.

2. The method according to claim 1, characterized in that The method further comprises: Determining a second defect area in the second image corresponding to the first defect area based on a position mapping relationship between the second image and the first image; The method of determining the defect type of the object to be measured by using the second image based on the first defect area and the second target area includes: Inputting the second image into an instance segmentation model to obtain instance regions and instance categories corresponding to the instance regions; Performing position matching between the instance region and the second defect region to determine a successfully matched instance region; The defect type of the object to be detected is determined based on the second target area, the successfully matched instance area, and the instance category corresponding to the successfully matched instance area.

3. The method according to claim 2, characterized in that The determining the defect type of the object to be detected based on the second target area, the successfully matched instance area, and the instance category corresponding to the successfully matched instance area includes: Determine whether the successfully matched instance area exceeds the boundary of the second target area to obtain a determination result; The defect type of the object to be tested is determined based on the judgment result and the instance category corresponding to the successfully matched instance area.

4. The method according to claim 3, characterized in that The successfully matched instance regions include one or more, and determining the defect type of the object to be tested based on the judgment result and the instance category corresponding to the successfully matched instance regions includes: Based on the judgment result and the defect type corresponding to each successfully matched instance region, determining the defect level corresponding to each successfully matched instance region, wherein at least two defect types have different defect levels; The defect type of the object to be tested is determined based on the defect level corresponding to each successfully matched instance region.

5. The method according to any one of claims 1 to 3, characterized in that: The first image is acquired by a first image acquisition device, the second image is acquired by a second image acquisition device, and acquiring the second image of the object to be measured based on first position information of the first target area in the first image includes: Based on a mapping relationship between an image coordinate system of the first image acquisition device and a world coordinate system, converting the first position information into second position information in the world coordinate system; adjusting a shooting position of the second image acquisition device based on a mapping relationship between an image coordinate system of the second image acquisition device and the world coordinate system and the second position information; The second image is acquired by the adjusted second image acquisition device.

6. The method according to any one of claims 1 to 3, characterized in that: The determining of the first target area and the first defect area in the first image of the object to be tested includes: Matching the first image with the image template of the object to be tested to obtain an area in the first image that successfully matches the image template as the first target area; Based on the image template, determining a foreground target area in the first target area, and taking the foreground target area that meets a preset condition as a first defect area in the first target area; The first image is compared with a reference image, and based on the comparison result, a first defect area at least partly located outside the first target area is determined.

7. The method according to any one of claims 1 to 3, characterized in that: Before determining the first target area and the first defect area in the first image of the object to be measured, the method further includes: Acquire an initial image, wherein the initial image includes a plurality of initial wafers; Determining whether each initial wafer in the initial image meets a preset rule; An initial wafer that meets preset rules is used as an object to be tested, and a first image of the object to be tested is generated based on the initial image.

8. The method according to any one of claims 1 to 3, characterized in that The object to be tested is a wafer, and the targets of interest in the quality inspection of the object to be tested at least include pads on the wafer.

9. A defect type detection device, characterized in that: The device comprises: A first determination module is used to determine a first target area and a first defect area in a first image of the object to be tested, wherein the first target area is an area corresponding to a target of interest in the object to be tested during quality inspection in the first image, and at least a portion of the first defect area is in the first target area; A second image acquisition module, configured to acquire a second image of the object to be measured based on first position information of the first target area in the first image, wherein the second image has higher clarity than the first image; A second determining module, configured to determine, in the second image, a second target area corresponding to the first target area; A defect type determination module is used to determine the defect type of the object to be tested using the second image based on the first defect area and the second target area.

10. An electronic device, characterized in that: It comprises a memory and a processor, wherein: the memory is used to store a computer program; the processor is used to execute the computer program to implement the defect type detection method according to any one of claims 1 to 8.

11. A storage medium storing a computer program / instruction, characterized in that: The computer program / instructions are used to execute the defect type detection method according to any one of claims 1 to 8 when running.

12. A computer program product comprising computer program instructions, characterized in that The computer program instructions are used to execute the defect type detection method according to any one of claims 1 to 8 when running.

Images