Defect merging method and defect merging apparatus

Abstract

The application discloses a defect merging method and a defect merging device. The method comprises the following steps: obtaining a defect image to be merged; performing first processing on each target scratch area according to the type of the target scratch area to obtain trace information in each target scratch area; obtaining profile decomposition data of each trace information in each target scratch area; performing second processing on the profile decomposition data of each trace information to determine first candidate trace information which is suspected to belong to a physical scratch; performing third processing on end point profile information of the first candidate trace information to determine second candidate trace information which belongs to the same physical scratch; determining at least two regions belonging to the same physical scratch in the defect image to be merged based on the second candidate trace information, end point profile information of the second candidate trace information and region information of the target scratch area; and merging the at least two regions.

Description

Technical Field

[0001] This application relates to the field of defect processing technology, and in particular to a defect merging method and defect merging device. Background Technology

[0002] In semiconductor wafer defect detection technology, defect information on the wafer surface can be obtained using optical or electronic inspection equipment. This defect information includes scratch defects and contamination defects. Scratch defects typically exhibit the following characteristics: ① elongated strip-like structures; ② continuous or broken distribution; ③ possibly curved or intersecting shapes.

[0003] Taking the detection of defects on the wafer surface using electronic inspection equipment as an example, the image to be inspected is typically input into a defect detection model, which then detects whether defects exist on the wafer surface. Considering that the defect detection model may be an artificial intelligence model or an algorithmic model composed of defect detection rules, due to the limitations of the model itself, the same physical scratch on the wafer surface may be detected as multiple scratch regions by the defect detection model. How to accurately merge multiple scratch regions belonging to the same physical scratch to precisely reconstruct the true scratch structure of the wafer surface has become a pressing technical problem to be solved. Summary of the Invention

[0004] This application provides a defect merging method and defect merging device to at least solve the above-mentioned technical problems existing in the prior art.

[0005] According to a first aspect of this application, a defect merging method is provided, comprising: Obtain a defective image to be merged, the defective image to be merged includes M target scratch regions, where M is a positive integer greater than or equal to 2; Based on the type of each target scratch area, the first processing is performed on each target scratch area to obtain the trace information within each target scratch area; Obtain contour decomposition data of each trace within each target scratch area; The contour decomposition data of each trace information is processed a second time to determine the first candidate trace information that is suspected to be a physical scratch. Obtain the endpoint contour information of the first candidate trace information; The endpoint contour information of the first candidate trace information is processed in a third step to determine the second candidate trace information belonging to the same physical scratch. Based on the second candidate trace information, the endpoint contour information of the second candidate trace information, and the regional information of the target scratch area, at least two regions belonging to the same physical scratch in the defect to be merged image are determined. Merge at least two regions belonging to the same physical scratch in the defective images to be merged.

[0006] In one possible implementation, the first processing of each target scratch region according to its type to obtain trace information within each target scratch region includes: Skeletonization is performed on the target scratch area to obtain the line structure information within the skeletonized area; If the line structure information includes intersecting lines, the type of the target scratch area is determined to be the first type. In response to the target scratch area being the first type, the intersecting area is obtained based on the intersecting lines. Based on the endpoint information of the intersecting lines within the target scratch area, the intersecting area of ​​the intersecting lines, and the line structure information of the intersecting lines outside the intersecting area, the trace information within the target scratch area is obtained. If the line structure information includes non-intersecting lines, the type of the target scratch area is determined to be the second type. Based on the line structure information in the area after skeletonization, the trace information in the target scratch area is obtained.

[0007] In one possible implementation, it further includes: If the line structure information includes intersecting lines and non-intersecting lines, the type of the target scratch region is determined to be the third type. In response to the target scratch region being of the third type, the intersecting region is obtained based on the intersecting lines. Based on the line structure information of the non-intersecting lines within the target scratch region, the endpoint information of the intersecting lines, the intersecting region of the intersecting lines, and the line structure information of the intersecting lines outside the intersecting region, the trace information within the target scratch region is obtained.

[0008] In one possible implementation, obtaining the contour decomposition data of each trace information within each target scratch area includes: performing contour processing on each trace information within each target scratch area to obtain the contour of the trace information; performing a smoothing operation on the contour of the trace information; and performing line structure decomposition on the smoothed contour to obtain the contour decomposition data of the trace information.

[0009] In one possible implementation, the trace information corresponds to at least one contour decomposition data; The second processing of the contour decomposition data of each trace information to determine the first candidate trace information that is suspected to be a physical scratch includes: obtaining at least one of the following: minimum distance information between two or more contour decomposition data, offset information in a preset direction, and directional angle deviation information; based on the at least one of the information between two or more contour decomposition data and a first correlation threshold of the at least one of the information, determining the trace information that is suspected to be a physical scratch for each trace information in the defect to be merged image; and marking the trace information that is suspected to be a physical scratch to obtain the first candidate trace information.

[0010] In one possible implementation, the first candidate trace information corresponds to contour decomposition data with identification information; The endpoint contour information for obtaining the first candidate trace information includes: Obtain the length information of each contour decomposition data in the contour decomposition data that is identified with the same identification information; in response to the length information of the contour decomposition data being greater than or equal to a first length threshold, segment the contour decomposition data, and generate endpoint contour information of the contour decomposition data based on the point sequence of the segmented contour decomposition data; in response to the length information of the contour decomposition data being less than the first length threshold, generate endpoint contour information of the contour decomposition data based on the point sequence of the contour decomposition data; and obtain the endpoint contour information of the first candidate trace information based on the endpoint contour information of the contour decomposition data.

[0011] In one embodiment, the third processing of the endpoint contour information of the first candidate trace information to determine the second candidate trace information belonging to the same physical scratch includes: performing a first sub-process on the endpoint contour information of the first candidate trace information to determine the third candidate trace information belonging to the physical scratch; performing a second sub-process on the endpoint contour information of the third candidate trace information to determine the second candidate trace information belonging to the same physical scratch; wherein the third processing includes the first sub-process and the second sub-process.

[0012] In one possible implementation, the first candidate trace information corresponds to contour decomposition data with identification information; The first sub-processing of the endpoint contour information of the first candidate trace information to determine the third candidate trace information belonging to the physical scratch includes: obtaining the orientation angle information and average orientation angle information of each point in the endpoint contour information of each contour decomposition data that is identified with the same identification information; and determining the third candidate trace information belonging to the physical scratch from the first candidate trace information based on the orientation angle information and average orientation angle information of each point in the endpoint contour information.

[0013] In one possible implementation, the second sub-processing of the endpoint contour information of the third candidate trace information to determine the second candidate trace information belonging to the same physical scratch includes: Obtain at least one of the following information from the minimum distance between two or more endpoint contour information, offset information in a preset direction, and directional angle deviation information of the third candidate trace information; determine trace information belonging to the same physical scratch based on the at least one information between the two or more endpoint contour information and a second correlation threshold of the at least one information; and use the trace information belonging to the same physical scratch as the second candidate trace information.

[0014] In one possible implementation, determining at least two regions belonging to the same physical scratch in the defect-to-be-merged image based on the second candidate trace information, the endpoint contour information of the second candidate trace information, and the region information of the target scratch region includes: The endpoint contour information of the second candidate trace information is fused to obtain fused trace information; a first correspondence is obtained between the endpoint contour information of the second candidate trace information and the region information of the target scratch region; a second correspondence is obtained between the fused trace information and the endpoint contour information; based on the first and second correspondences, a third correspondence is obtained between the fused trace information and the target scratch region; through the third correspondence, at least two regions belonging to the same physical scratch in the defect to be merged image are obtained.

[0015] In one possible implementation, obtaining the defective image to be merged includes: obtaining the defective image to be merged, the defective image to be merged including N original scratch regions, where N is a positive integer greater than or equal to 2; using the N original scratch regions as M target scratch regions; or, preprocessing the N original scratch regions to obtain M target scratch regions, where M is a positive integer greater than or equal to 2.

[0016] In one possible implementation, the preprocessing of the N original scratch regions includes: Calculate the area of ​​each region in the N original scratch regions; based on the area of ​​each original scratch region, determine multiple first scratch regions from the N original scratch regions; perform a filling operation on each region in the multiple first scratch regions to obtain multiple second scratch regions; perform morphological operations on each region in the multiple second scratch regions to obtain multiple third scratch regions; and take the multiple third scratch regions as M target scratch regions.

[0017] In one possible implementation, obtaining the defective image to be merged includes: Obtain the image to be identified; input the image to be identified into the defect detection model to obtain the target scratch area output by the defect detection model; use the image to be identified containing the target scratch area as the defect image to be merged.

[0018] According to a second aspect of this application, a defect merging device is provided, comprising: The first obtaining unit is used to obtain a defective image to be merged, the defective image to be merged includes M target scratch regions, where M is a positive integer greater than or equal to 2; The second obtaining unit is used to perform a first processing on each target scratch area according to the type of each target scratch area, so as to obtain trace information in each target scratch area. The third acquisition unit is used to acquire contour decomposition data of each trace information within each target scratch area; The first determining unit is used to perform a second processing on the contour decomposition data of each trace information to determine the first candidate trace information that is suspected to be a physical scratch. The fourth obtaining unit is used to obtain the endpoint contour information of the first candidate trace information; The second determining unit is used to perform third processing on the endpoint contour information of the first candidate trace information to determine the second candidate trace information belonging to the same physical scratch. The third determining unit is used to determine at least two regions belonging to the same physical scratch in the defect-to-merge image based on the second candidate trace information, the endpoint contour information of the second candidate trace information, and the region information of the target scratch region. The merging unit is used to merge at least two regions belonging to the same physical scratch in the defective image to be merged.

[0019] According to a third aspect of this application, an electronic device is provided, comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described in this application.

[0020] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this application.

[0021] According to a fifth aspect of this application, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method described in this application.

[0022] The technical solution of this application can accurately merge multiple scratch regions belonging to the same physical scratch, so as to accurately restore the real scratch structure on the wafer surface, and provide technical support for the real and effective presentation of the scratch structure.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0024] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, wherein: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0025] Figure 1 This document illustrates the implementation flow of the defect merging method in an embodiment of this application. Figure 1 ; Figure 2 This illustration shows a diagram of how the same physical scratch is divided into multiple segments in an embodiment of this application; Figure 3(a) shows a schematic diagram of the cross lines in an embodiment of this application; Figure 3(b) shows a schematic diagram of a unidirectional line segment in an embodiment of this application; Figure 3(c) shows a schematic diagram of non-crossing lines in an embodiment of this application; Figure 4 A schematic diagram of the outline line segments in an embodiment of this application is shown; Figure 5 This paper illustrates the implementation flow of the defect merging method in an embodiment of this application. Figure 2 ; Figure 6 A schematic diagram of the endpoint contours in an embodiment of this application is shown. Figure 1 ; Figure 7 The third illustration shows the implementation flow diagram of the defect merging method in the embodiments of this application; Figure 8(a) shows a schematic diagram of the endpoint contours in an embodiment of this application. Figure 2 ; Figure 8(b) shows a schematic diagram of the fusion of endpoint contours in an embodiment of this application; Figure 9 This paper illustrates the implementation flow of the defect merging method in an embodiment of this application. Figure 4 ; Figure 10 A schematic diagram of the composition structure of the defect merging device in an embodiment of this application is shown; Figure 11 A schematic diagram of the composition structure of the electronic device in an embodiment of this application is shown. Detailed Implementation

[0026] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The defect merging scheme provided in this application, based on the second candidate trace information belonging to the same physical scratch, the endpoint contour information of the second candidate trace information, and the regional information of the target scratch area, determines at least two regions belonging to the same physical scratch in the defect to be merged image, thereby achieving merging. Therefore, the technical solution of this application can accurately merge multiple scratch regions belonging to the same physical scratch, precisely restoring the true scratch structure of the wafer surface, and providing technical support for the realistic and effective representation of scratch structures.

[0028] Figure 1 This paper illustrates the implementation flow of the defect merging method in an embodiment of this application. Figure 1 .like Figure 1 As shown, the defect merging method includes: S (Step) 101: Obtain the defect image to be merged, which includes M target scratch regions, where M is a positive integer greater than or equal to 2.

[0029] In this step, the images to be merged are those containing scratch defects, where the same physical scratch is divided into several independent parts to be merged. Within the same image to be merged, there may be one or more (real) physical scratches. Each physical scratch may be segmented into two or more independent parts in the image. For example... Figure 2 As shown, the same physical scratch is divided into three segments. The target scratch area refers to the area in the image to be merged containing the scratch defect. The technical solution of this application aims to accurately merge multiple scratch areas belonging to the same physical scratch in the image to be merged to restore the true scratch structure.

[0030] In implementation, the following steps can be taken: First, obtain an image to be identified. Second, input the image to be identified into a defect detection model to obtain the target scratch region output by the defect detection model. Third, use the image to be identified containing the target scratch region as the defect image to be merged. In this application, the image to be identified can be an image showing defects on the wafer surface. The defect detection model detects scratch defects in the image to be identified and outputs the region containing the scratch. The image to be identified with the output scratch region is then used as the defect image to be merged. The defect detection model can be an artificial intelligence model or an algorithm model composed of defect detection rules.

[0031] In layman's terms, the technical solution of this application can effectively merge scratches that belong to the same physical scratch but were originally divided into several segments on the wafer surface, making up for the deficiency of defect detection models that cannot merge the same physical scratches.

[0032] S102: Based on the type of each target scratch area, perform a first processing on each target scratch area to obtain trace information within each target scratch area.

[0033] In this step, the target scratch area is categorized into three types: a first type with only intersecting lines, a second type with only non-intersecting lines, and a third type with both intersecting and non-intersecting lines. The type of the target scratch area is identified, and based on this type, the target scratch area undergoes a first processing step to obtain trace information, such as scratch trace information, within the target scratch area.

[0034] For the target scratch area, the scratch marks within the area can be obtained through operations related to skeletonization. Therefore, the first process involves at least operations related to the skeletonization of the area.

[0035] S103: Obtain contour decomposition data of each trace information within each target scratch area.

[0036] In this step, the contour decomposition data can be the contour structure of a scratch mark that constitutes the target scratch area. For example, multiple or several contour line segments that constitute a scratch mark.

[0037] In this application, the scratch marks within the target scratch area can be processed by skeleton contouring, contour decomposition, and other related processes, thereby decomposing the trace information at the contour level and obtaining the contour line segments of the scratch marks.

[0038] It should be noted that region skeletonization and contouring are two different implementations. For details on the specific implementation, please refer to the relevant documentation.

[0039] S104: Perform a second processing on the contour decomposition data of each trace information to determine the first candidate trace information that is suspected to be a physical scratch.

[0040] In the images to be merged, there may be pseudo-scratches such as curved line segments. These pseudo-scratches may be caused by other reasons, such as dirt, rather than by real scratches. Therefore, in some scenarios, these pseudo-scratches caused by other reasons may be mistaken for real scratches.

[0041] In this step, by utilizing the contour decomposition data of each trace information, trace information suspected of belonging to physical scratches is identified from the trace information of the defective image to be merged. The first candidate trace information can be trace information suspected of belonging to physical scratches in the trace information of the image to be merged. It can be understood that the first candidate trace information may include traces belonging to real physical scratches, or it may include traces of pseudo-scratches.

[0042] The second process can be considered as the process of determining trace information that is suspected to be a physical scratch in this application. In implementation, certain (first) constraints can be used to process the contour decomposition data of the trace information, thereby determining the trace information that is suspected to be a physical scratch.

[0043] S105: Obtain the endpoint contour information of the first candidate trace information.

[0044] In this step, the endpoint contour information of the trace information can be generated by using the contour of the first candidate trace information.

[0045] S106: Perform third processing on the endpoint contour information of the first candidate trace information to determine the second candidate trace information belonging to the same physical scratch.

[0046] In this step, the second candidate trace information belonging to the same physical scratch in the defect-to-be-merged image is determined by the third processing of the endpoint contour of the first candidate trace information.

[0047] The third process includes the process of determining the candidate trace information from (first) suspected physical traces to candidate trace information that belongs to physical traces, and the process of determining the physical trace information to the same physical trace information.

[0048] S107: Based on the second candidate trace information, the endpoint contour information of the second candidate trace information, and the regional information of the target scratch area, determine at least two regions in the defect to be merged that belong to the same physical scratch.

[0049] In this step, the two or more regions belonging to the same physical scratch in the defect-to-be-merged image are determined by using three pieces of information: the second candidate trace information, the endpoint contour information of the second candidate trace information, and the regional information of the target scratch area.

[0050] S108: Merge at least two regions in the defective image that belong to the same physical scratch.

[0051] In this step, at least two regions are merged to obtain a defect merged image. It is understood that in the defect image to be merged, marks belonging to the same (real) physical scratch may appear independently in the image due to segmentation, making it impossible to identify which marks belong to the same (real) physical scratch. However, in the defect merged image, the merging scheme of this application can effectively show which independently segmented marks belong to the same (real) physical scratch.

[0052] The technical solutions shown in S101-S108 can merge multiple scratch regions belonging to the same physical scratch. Specifically, contour decomposition data of the trace information is used to determine the first candidate trace information that is suspected to belong to the physical scratch; the endpoint contour information of the first candidate trace information is used to determine the second candidate trace information belonging to the same physical scratch, ensuring the accuracy of the first and second candidate trace information. Based on the second candidate trace information, the endpoint contour information of the second candidate trace information, and the regional information of the target scratch region, accurate merging is achieved, precisely restoring the true scratch structure of the wafer surface, providing technical support for the realistic and effective representation of the scratch structure.

[0053] In some embodiments, the aforementioned S101 can be implemented using the following methods: Obtain the defective image to be merged, which includes N original scratch regions, where N is a positive integer greater than or equal to 2; use the N original scratch regions as M target scratch regions; or, preprocess the N original scratch regions to obtain M target scratch regions, where M is a positive integer greater than or equal to 2.

[0054] In the foregoing, the scratch regions output by the defect detection model can be used as the original scratch regions, and there are N original scratch regions in the defect image to be merged. In some scenarios, the N scratch regions output by the defect detection model can be used as the M target scratch regions of this application. Alternatively, considering the limitations of the defect detection model, the N original scratch regions it detects may contain noise regions (noise points) that are not scratch structures. In this application, a preprocessing scheme can be used to at least filter out noise points and retain the real scratch structures, so as to provide accurate basic data for the accurate merging of multiple trace regions belonging to the same physical scratch.

[0055] The aforementioned scheme involves preprocessing N original scratch regions, including the following steps: calculating the area of ​​each region within the N original scratch regions; determining multiple first scratch regions from the N original scratch regions based on their area; performing a filling operation on each region within the multiple first scratch regions to obtain multiple second scratch regions; performing morphological operations on each region within the multiple second scratch regions to obtain multiple third scratch regions; and using the multiple third scratch regions as M target scratch regions. Specifically, the preprocessing in this application mainly includes the following three aspects.

[0056] (1) Noise reduction in small areas Implementation: Take N original scratch regions detected by the defect detection model as a scratch region set, perform connected domain analysis on the scratch region set, calculate the area of each scratch region in the scratch region set, denoted as Area_i. Read a preset area threshold A_th (for example, 5 to 50 pixels), and compare the magnitude relationship between the area Area_i of each scratch region and A_th. Keep the scratch regions where Area_i ≥ A_th (such scratch regions can be used as first scratch regions), and delete the scratch regions that satisfy Area_i < A_th. The scratch regions with Area_i < A_th are noise points or pseudo-small regions, and these regions usually do not belong to real scratch structures.

[0057] Purpose of this preprocessing operation: Remove isolated noise points or pseudo-small regions generated by the defect detection model, which usually do not belong to real scratch structures, so as to ensure the accuracy of basic data and the processing stability of subsequent solutions.

[0058] (2) Small hole filling Implementation: Perform a region filling operation on each scratch region where Area_i ≥ A_th to fill the holes inside the region, so that the region becomes a solid structure. For example, in implementation, a region filling function (Fill_up) can be used to fill the region (Region), for example, RegionFilled = FillUp(Region) to obtain the second scratch region RegionFilled.

[0059] Purpose: Due to the incomplete segmentation of the original scratch regions detected by the defect detection model, holes or breaks may exist inside the original scratch regions, which will lead to problems such as discontinuous skeletons and abnormal number of endpoints in subsequent processing. In the technical solution of the present application, by adding the filling operation, the scratch structure is made more complete, thereby ensuring the continuity and correctness of the skeleton extraction result.

[0060] (3) Morphological closing operation to smooth the boundary of the region Implementation: Perform a morphological closing operation on the scratch regions after small hole filling to smooth the region boundaries, eliminate boundary burrs, and connect tiny breaks between regions. The operation of smoothing the boundary can avoid the problem of large merging error of scratch regions caused by zigzag branches appearing on the skeleton, and ensure the merging accuracy.

[0061] In some embodiments, the foregoing solution in S102 of performing first processing on each target scratch region according to the type of each target scratch region to obtain trace information in each target scratch region includes: Performing skeletonization processing on the target scratch region to obtain line structure information in the skeletonized region; If the line structure information includes intersecting lines, the type of the target scratch area is determined to be the first type. In response to the target scratch area being the first type, the intersecting area is obtained based on the intersecting lines. Based on the endpoint information of the intersecting lines within the target scratch area, the intersecting area of ​​the intersecting lines, and the line structure information of the intersecting lines outside the intersecting area, the trace information within the target scratch area is obtained.

[0062] If the line structure information includes non-intersecting lines, the type of the target scratch area is determined to be the second type. Based on the line structure information in the area after skeletonization, the trace information in the target scratch area is obtained.

[0063] If the line structure information includes intersecting lines and non-intersecting lines, the type of the target scratch region is determined to be the third type. In response to the target scratch region being of the third type, the intersecting region is obtained based on the intersecting lines. Based on the line structure information of the non-intersecting lines within the target scratch region, the endpoint information of the intersecting lines, the intersecting region of the intersecting lines, and the line structure information of the intersecting lines outside the intersecting region, the trace information within the target scratch region is obtained.

[0064] In the aforementioned scheme, the preprocessed target scratch region is skeletonized to obtain the line structure information within the skeletonized region. Typically, scratches within an image region are line structures, such as... Figure 2 The line segments shown. Within the same area, there may be intersecting lines as shown in Figure 3(a), or there may be no intersecting lines but non-intersecting lines, such as... Figure 2 The diagram shows individual line segments. Intersecting and non-intersecting lines may also exist simultaneously. Based on this, after skeletonizing an image region, the resulting data consists of intersecting and non-intersecting lines (independent line segments) within the image region. These intersecting and non-intersecting lines are used as line structure information.

[0065] The system identifies whether intersecting and non-intersecting lines exist within the skeletonized target scratch area. If only intersecting lines exist, the area is considered a Type 1 area. If only non-intersecting lines exist, such as only independent line segments, the area is considered a Type 2 area. If both intersecting and non-intersecting lines exist, the area is considered a Type 3 area.

[0066] During implementation, for the target scratch area containing only intersecting lines, the area is skeletonized and then transformed into a single-pixel-wide centerline structure, resulting in two intersecting lines as shown in Figure 3(a). Lines AB and CD intersect through the intersecting line ef. The area is expanded outward from the intersecting line ef to obtain the intersection point area. Morphological dilation is performed on the intersection point area to obtain an elliptical area as shown in Figure 3(a). The elliptical area is then deleted, and the resulting line endpoints a, b, c, and d form a certain line structure with the four endpoints of lines AB and CD. This line structure can be four non-intersecting unidirectional line segments, as shown in Figure 3(b): line segment Aa, line segment Bb, line segment Cc, and line segment Dd. These unidirectional line segments are used as the trace information within this area.

[0067] During implementation, for target scratch areas with only non-intersecting lines, the area is skeletonized and then transformed into a single-pixel-wide centerline structure, resulting in unidirectional line segments a1b1 and c1d1 as shown in Figure 3(c). Line segments a1b1 and c1d1 are used as the trace information for this area.

[0068] During implementation, for target scratch areas that have both intersecting and non-intersecting lines, the processing of intersecting and non-intersecting lines is as described above, thereby obtaining unidirectional line segments within the area that can be used as trace information.

[0069] It is understandable that the skeletonization process performed on the target scratch area in the aforementioned solution is equivalent to extracting the skeleton of that area. Typically, this solution extracts the skeleton of the target scratch area, detects intersections within the skeleton, and removes these intersections after dilation, thus decomposing the originally intersecting scratch structure into multiple independent line segments with consistent directions. This method effectively avoids the problem of scratches from different directions being incorrectly merged at intersections, thereby improving the accuracy and stability of subsequent merging.

[0070] In some embodiments, the scheme for obtaining the contour decomposition data of each trace information in each target scratch area in S103 includes: performing contour processing on each trace information in each target scratch area to obtain the contour of the trace information; performing a smoothing operation on the contour of the trace information; and performing line structure decomposition on the smoothed contour to obtain the contour decomposition data of the trace information.

[0071] During implementation, each unidirectional line segment within the target scratch area is scaled. For example, a scaling factor (Scale) is used to scale the skeleton line segments, resulting in reduced skeleton line segments. The scaling factor Scale ranges from 0.2 to 0.5. Here, the unidirectional line segments are line segments at the skeleton level. Using a scaling factor less than 1 to scale the skeleton line segments can accelerate the calculation and shorten the calculation process. The reduced skeleton line segments are then contoured to obtain the corresponding contour. Using a scale recovery factor of 1 / Scale, the contour is scaled back to obtain the original scale contour (ContourOrig) corresponding to the skeleton line segment. The original scale contour ContourOrig is then smoothed to reduce noise interference. The smoothed original scale contour ContourOrig is then decomposed into line segments to obtain contour segments (ContoursSplit). The contour segments obtained from the line segment decomposition of the original scale contour can be used as contour decomposition data.

[0072] This method of transforming the skeleton into a contour and then decomposing the contour into line segments is easy to implement in engineering, highly feasible, and can also improve the accuracy of identifying subsequent candidate trace information that may belong to physical scratches.

[0073] In some embodiments, the trace information corresponds to at least one contour decomposition data. The aforementioned S104 method for performing a second processing on the contour decomposition data of each trace information to determine a first candidate trace information suspected of belonging to a physical scratch may be: Obtain at least one of the following: minimum distance information between two or more contour decomposition data, offset information in a preset direction, and directional angle deviation information; based on the at least one of the following information between the two or more contour decomposition data and a first correlation threshold of the at least one information, determine the trace information that is suspected to be a physical scratch for each trace information in the defect to be merged image; mark the trace information that is suspected to be a physical scratch to obtain the first candidate trace information.

[0074] by Figure 4 Taking the four outline segments (Ee, Ff, Gg, and Hh) as an example, multiple marks belonging to the same physical scratch have collinear outline segments. To determine which two or more of the four outline segments are collinear, it is necessary to check whether the outline segments satisfy the first constraint condition. The first constraint condition includes the minimum distance between outline segments. Offset information in the preset direction Direction angle deviation information .in, For the preset distance threshold, For offset threshold, These are the orientation angle deviation thresholds. These three thresholds can be used as the first relevant thresholds.

[0075] d (distance between contour segments): Represents the minimum Euclidean distance between two contour segments, usually taken as the distance between the endpoints or the nearest point of the two contour segments. In implementation, the minimum distance d between the two contour segments is obtained by calculating the minimum Euclidean distance between them.

[0076] Dmax (distance threshold): Used to control the maximum spacing allowed for merging between contour line segments; also known as the maximum distance threshold. Physically, it represents the maximum discontinuity length of a scratch under fracture conditions.

[0077] Δx (lateral offset): Represents the vertical offset distance of one contour segment relative to another contour segment in the principal direction, such as the lateral direction. In practice, the lateral offset Δx between the two contour segments is obtained by calculating the vertical offset distance between the two contours in the lateral direction.

[0078] Smax (Offset Threshold): Represents the maximum allowed lateral offset, also known as the maximum offset threshold. It is used to prevent adjacent but non-collinear scratches from being incorrectly identified as collinear scratches.

[0079] θ1 and θ2 (direction angles of the contour line segments): These represent the direction angles of the two contour line segments, usually calculated from the coordinates of the endpoints of the segments, such as θ = arctan(Δy / Δx). (Δx, Δy) are the coordinate differences between the two endpoints of the contour line segments. It is easy to understand that the position of the contour line segment in the image and the positions of its endpoints within the contour line segment are readily known. Therefore, the coordinate difference between the endpoints is easily determined. In practice, the direction angles of the contour line segments can be easily calculated using the aforementioned formula.

[0080] Δθ (direction angle deviation): Δθ = |θ1 θ2| represents the difference between the direction angles of the two contour line segments. In practice, the direction angle deviation can be obtained by subtracting the calculated direction angles of the two contour line segments.

[0081] θmax (direction angle deviation threshold): The maximum allowable direction angle deviation, which can also be regarded as the maximum angle threshold. It is used to constrain contour line segments to be approximately collinear in terms of angle.

[0082] In the aforementioned scheme, d can be considered as spatial distance, Δx as lateral offset, and Δθ as directional consistency. Determining whether the first constraint condition is met can be simply understood as judging whether the contour segments are collinear based on three dimensions: spatial distance, lateral offset, and directional consistency. This judgment based on the first constraint condition ensures the accuracy of identifying traces as suspected physical scratches, providing a certain guarantee for merging multiple traces belonging to the same physical scratch.

[0083] If two contour segments satisfy the first constraint condition, then these two contour segments are collinear, and the marks corresponding to these two contour segments are marks that are suspected physical scratches. In this step, some pseudo-scratches, such as curved lines, may also satisfy the aforementioned first constraint condition, and therefore may also be identified as marks that are suspected physical scratches.

[0084] The first constraint mentioned above applies. Figure 4 In the given condition, if line segments Ee and Hh, and line segments Ff and Gg satisfy the first constraint condition, then the marks corresponding to line segments Ee, Hh, Ff, and Gg are considered to be suspected physical scratches. In implementation, we can use identifier 1 to identify line segments Ee and Hh that satisfy the first constraint condition, and identifier 2 to identify line segments Ff and Gg that satisfy the first constraint condition. Correspondingly, the marks corresponding to line segments Ee and Hh can also be identified using identifier 1, and the marks corresponding to line segments Ff and Gg can also be identified using identifier 2. The mark information identified using the corresponding identifier information is considered as the first candidate mark information. Different identifiers can distinguish which contour line segments or marks are collinear. The identifier information can be any identifier that distinguishes different collinearity situations, such as color, line size, etc.

[0085] In some embodiments, the first candidate trace information corresponds to contour decomposition data with identification information. Combined with Figure 5 As shown, the aforementioned scheme for obtaining the endpoint contour information of the first candidate trace information in S105 includes: S501: Obtain the length information of each contour decomposition data in the contour decomposition data that is identified as the same identification information.

[0086] In this step, with Figure 6 Taking line segments Ee and Hh, labeled as 'Identifier 1', as an example, for each contour line segment in Ee and Hh, the (coordinate) point sequence of each contour line segment is obtained, and the length of the point sequence of each contour line segment is calculated. The length of the point sequence of each contour line segment is used as the length information of each contour decomposition data.

[0087] Determine whether the length of the point sequence of the contour line segment is greater than or equal to a first length threshold set empirically. If yes, proceed to step S502. If no, proceed to step S503.

[0088] S502: In response to the length information of the contour decomposition data being greater than or equal to a first length threshold, the contour decomposition data is segmented, and the endpoint contour information of the contour decomposition data is generated based on the (coordinate) point sequence of the segmented contour decomposition data.

[0089] In this step, if the length of the point sequence of the contour line segment is greater than or equal to the first length threshold, it indicates that the contour line segment is a long contour line segment. The contour line segment is then divided into two or more segments, such as dividing it into two equal segments. For each segment, at one end (the same end as both ends of the contour line segment), a certain number of coordinate points are extracted to generate the endpoint contour of each segment. The set of endpoint contours of two or more segments constitutes the endpoint contour information of the contour line segment.

[0090] by Figure 6 As shown in the example, the contour line segment Ee is divided into two equal segments. Figure 6 The portion covered by the dashed circle can serve as the endpoint contours of the two segments located at one end of each segment; that is, contour segment Ee has two endpoint contours. Similar to the generation scheme of the endpoint contours of contour segment Ee, contour segment Hh, which shares the same identifier as contour segment Ee, also has two endpoint contours. In this application, the endpoint contours are line structures, such as line segments, with a certain length.

[0091] S503: In response to the length information of the contour decomposition data being less than the first length threshold, generate the endpoint contour information of the contour decomposition data based on the point sequence of the contour decomposition data.

[0092] In this step, if the length of the point sequence of the contour line segment is less than the first length threshold, it indicates that the contour line segment is a short contour line segment. The contour line segment is then treated as a whole, and a certain number of coordinate points at both ends are extracted to generate the endpoint contours of the contour line segment. This method is similar to the method for generating the endpoint contours of the two segments in S502, and will not be elaborated further. Alternatively, the entire short contour line segment can be used as its endpoint contour.

[0093] S502 and S503 represent two scenarios that can be handled flexibly according to the actual situation, without limiting the execution order. In S502 and S503, a certain number of values ​​can be related to the value of the first length threshold, such as taking a value that is half of the first length threshold.

[0094] S504: Based on the endpoint contour information of the contour decomposition data, obtain the endpoint contour information of the first candidate trace information.

[0095] In this step, the endpoint contours of the contour decomposition data can be used as the endpoint contours of the first candidate trace information.

[0096] In S501~S504, the endpoint contours of the contour line segments are obtained in different ways based on the relationship between the contour line segments and the first length threshold. This method of obtaining endpoint contours is easy to implement in engineering, highly feasible, and can ensure the accuracy of the endpoint contours, thus providing data assurance for the accurate merging of multiple marks belonging to the same physical scratch.

[0097] In S501~S504, the distinction is made between long and short contour segments. For long contour segments, the local segments at both ends of the long contour are extracted as endpoint contours. For short contour segments, the entire short contour is retained as the endpoint contour, or the endpoints at both ends are extracted to generate the endpoint contour. This facilitates the execution of subsequent solutions and provides a certain technical foundation for the accurate fusion of subsequent endpoint contours.

[0098] In some embodiments, combined with Figure 7 As shown, the aforementioned S106 method for performing a third process on the endpoint contour information of the first candidate trace information to determine the second candidate trace information belonging to the same physical scratch includes: S701: Perform the first sub-processing on the endpoint contour information of the first candidate trace information to determine the third candidate trace information that belongs to the physical scratch.

[0099] In this step, the first candidate trace information corresponds to contour decomposition data with identification information. Thus, during implementation, the orientation angle information and average orientation angle information of each point in the contour information of each endpoint of each contour decomposition data identified with the same identification information can be obtained; based on the orientation angle information and average orientation angle information of each point in the contour information of each endpoint, the third candidate trace information belonging to physical scratches can be determined from the first candidate trace information.

[0100] by Figure 6 Taking a contour segment Ee with two endpoints as an example, according to the curvature calculation formula... This is used to determine whether a line structure formed by points in the endpoint contour is a straight line structure. In the curvature calculation formula, P represents the number of points in the endpoint contour of the contour line segment Ee. θi represents the local orientation angle of the i-th point (obtained from the coordinates of the point sequence of the endpoint contour). (Average direction angle) represents the (primary) direction angle of all points in the entire endpoint contour of the contour line segment Ee (obtained from the coordinates of the point sequence of the entire endpoint contour).

[0101] when is less than C_th, it is determined that the line structure formed by the end point contour is a straight line structure; otherwise, it is not considered as a straight line structure. Herein, C_th is a preset threshold, which can be flexibly set according to actual situations. It can be understood that when C is less than C_th, C can be considered relatively small, the directions of points in all end point contours are approximately consistent, the line structure formed by such points is considered to be close to a straight line, and such end point contours are retained. When it is greater than C_th, C is considered relatively large, the direction of points in the end point contour changes greatly, most of the line structures formed by such points are curved structures, and such end point contours are deleted.

[0102] It can be understood that the traces of physical marks are usually straight line structures, and pseudo-physical scratches such as curved line segments are usually not straight line structures but curved structures. According to the technical solution of the present application, end point contours that can form a straight line structure are screened out by calculating the curvature, which is equivalent to screening out physical scratches and removing pseudo-scratches such as curved structures.

[0103] For the retained end point contours, such end point contours correspond to contour line segments, and such contour line segments correspond to trace information. Among the first candidate trace information, the trace information corresponding to the contour line segments whose end point contours are retained is taken as third candidate trace information. It can be understood that different from the first candidate trace information which includes traces of physical scratches and may include pseudo-scratches, the third candidate trace information excludes pseudo-scratches and includes traces of physical scratches.

[0104] S702: performing a second sub-process on the end point contour information of the third candidate trace information to determine second candidate trace information belonging to the same physical scratch; wherein the third process includes a first sub-process and a second sub-process.

[0105] In implementation, at least one of minimum distance information, offset information in a preset direction, and direction angle deviation information between two or more pieces of end point contour information of the third candidate trace information may be obtained; based on the at least one piece of information between the two or more pieces of end point contour information and a second correlation threshold of the at least one piece of information, trace information belonging to the same physical scratch is determined; and the trace information belonging to the same physical scratch is taken as the second candidate trace information.

[0106] In simple terms, the end point contour information may be in the form of line segments, such as each line segment shown in Figure 8(a). From all end point contours (end point line segments) of the third candidate trace information, it is determined which end point line segments are collinear line segments, and the traces corresponding to the collinear end point line segments may be the same physical scratch. To determine which end point line segments are collinear line segments, it is necessary to determine whether a second constraint condition is satisfied between the end point line segments. The second constraint condition includes the minimum distance between the end point line segments , offset information in a preset direction , .in, For the preset distance threshold, For offset threshold, These are the orientation angle deviation thresholds. These three thresholds can be used as a second correlation threshold.

[0107] To understand the parameters involved in the second constraint, you can adapt the understanding by replacing the contour line segments in the description of the first constraint with endpoint line segments; further explanation is unnecessary. The first constraint aims to filter out marks that are suspected to be physical scratches, while the second constraint aims to filter out marks belonging to the same physical scratch. Considering the rigor of the second constraint, in implementation, the value of the second correlation threshold can be lower than the first correlation threshold.

[0108] It should be noted that for two or more endpoint contours identified by the same identifier, if the curvature calculation formula is retained, then most of these endpoint contours will satisfy the second constraint condition by calculation, and are considered to be collinear line segments, belonging to the same physical trace.

[0109] The aforementioned scheme determines whether endpoint line segments are collinear based on three dimensions: spatial distance, lateral offset, and directional consistency. This judgment, based on the second constraint, ensures the accuracy of identifying traces belonging to the same physical scratch, providing a certain guarantee for merging multiple traces belonging to the same physical scratch.

[0110] If two or more endpoint line segments satisfy the second constraint condition, then these two endpoint line segments are collinear line segments, and the marks corresponding to these two contour line segments belong to the same physical scratch. Merging such endpoint line segments yields merged line segments. For example, merging the right-hand endpoint line segments shown in Figure 8(a) yields merged line segment 1 (shown as a dashed line) as shown in Figure 8(b). Merging the left-hand endpoint line segments yields merged line segment 2 (shown as a dashed line) as shown in Figure 8(b). Collecting these merged line segments yields a set of merged line segments.

[0111] In some embodiments, such as Figure 9 As shown, the aforementioned S107 scheme for determining at least two regions belonging to the same physical scratch in the defect-to-be-merged image based on the second candidate trace information, the endpoint contour information of the second candidate trace information, and the region information of the target scratch region includes: S901: The endpoint contour information of the second candidate trace information is fused to obtain fused trace information.

[0112] In this step, as shown in Figure 8(b), the endpoint line segments that satisfy the second constraint condition are merged to obtain fusion trace information (fusion line segments).

[0113] S902: Obtain the first correspondence between the endpoint contour information of the second candidate trace information and the regional information of the target scratch area.

[0114] In this application, the target scratch area is the area with scratch defects output by the defect detection model, such as having... Figure 2 The three scratches are shown in three regions. Since the endpoint segments are generated from some points of the scratches within the target scratch region, that is, a certain endpoint segment originates from a certain target scratch region, there is the following (first) correspondence between the endpoint segments and the target scratch regions.

[0115] Edge_k → OriginalContour_i Here, Edge_k represents the contour information (endpoint line segment) of the k-th endpoint. OriginalContour_i represents the ith target scratch region. The same target scratch region may correspond to two or more endpoint line segments. That is, the first correspondence is many-to-one.

[0116] S903: Obtain the second correspondence between fusion trace information and endpoint contour information.

[0117] In this step, each fusion segment in the fusion segment set is a line segment, and thus the fusion trace information can be considered as line segments. For each fusion segment, taking fusion segment 1 shown in Figure 8(b) as an example, the endpoint line segments within a certain pixel distance (e.g., 20 pixels) from fusion segment 1 are found. Based on the coordinates of each point on fusion segment 1 and the coordinates of each point on the found endpoint line segments, the overlap between fusion segment 1 and which endpoint line segments are calculated is determined. Among the endpoint line segments that overlap with fusion segment 1, the endpoint line segments with an overlap area greater than or equal to a first area threshold are selected. The selected endpoint line segments and fusion segment 1 can establish the following (second) correspondence.

[0118] RegionUnion_j → Edge_k This correspondence represents the relationship between the merged line segment j (RegionUnion_j) and the selected endpoint line segments Edge_k whose overlapping area is greater than or equal to the first area threshold. The first area threshold is a value flexibly set based on the actual situation, such as 5 pixels. For the same merged line segment, there may be multiple endpoint line segments corresponding to it. That is, the second correspondence is a one-to-many relationship.

[0119] In this application, the fused line segments can also be converted from line segments into regions to obtain fused regions. For each fused region, the endpoint line segments within a certain pixel distance from the fused region are found, and the correspondence between the fused region and the endpoint line segments is established by calculating the overlapping area of ​​the endpoint line segments.

[0120] S904: Based on the first and second correspondences, a third correspondence is obtained between the fused trace information and the target scratch area.

[0121] In this step, a third correspondence is obtained based on the first and second correspondences. Both the first and second correspondences share the same endpoint segment Edge_k; this endpoint segment Edge_k can be used to establish the third correspondence.

[0122] RegionUnion_j → OriginalContour_i It is understandable that, given that the first correspondence is many-to-one and the second is one-to-many, the third correspondence is also one-to-many. The same RegionUnion_j corresponds to multiple target scratch regions. In the third correspondence, OriginalContour_i can be written as a set, and the third correspondence transforms into the following: RegionUnion_j → {OriginalContour_i} S905: Through the third correspondence, at least two regions belonging to the same physical scratch are obtained in the defect-to-be-merged image.

[0123] It can be understood that in the third correspondence, the same RegionUnion_j corresponds to multiple target scratch regions. These multiple target scratch regions are the areas in the defective images to be merged that belong to the same physical scratch.

[0124] For example, if for the same RegionUnion_j, there are three target scratch regions S1, S2 and S3 in the third correspondence, then the scratches in these three regions S1, S2 and S3 are actually the same physical scratch divided into three regions.

[0125] It is understandable that by performing the aforementioned S901~S905 operations on each fusion segment in the fusion segment set, we can obtain the regions belonging to the same real physical scratch in the image to be merged for each real physical scratch and defect.

[0126] At least two regions belonging to the same physical scratch in the defect-to-be-merged image are merged to obtain a merged defect image. In the merged defect image, target scratch regions belonging to the same physical scratch area can be identified with the same labeling information, such as using the same color. Target scratch regions belonging to different physical scratch areas can be identified with different labeling information, such as using two different colors to identify target scratch regions belonging to two different physical scratches. This effectively shows which originally independently segmented marks belong to the same physical scratch. The merged defect image contains several physical scratches.

[0127] In this application, the characteristic that scratches are approximately straight lines within a local area is utilized to effectively eliminate false scratches by using endpoint contours, significantly reducing the probability of false merging. This scheme of using endpoint contours to represent contour lines and thereby eliminates false scratches can effectively improve the situation where the real physical scratches may be complex curved structures, and the technical solution of this application has a wide range of applications.

[0128] In this application, a fused line segment is generated by fusing endpoint line segments, and a multi-level correspondence / mapping relationship is established between the fused line segment, endpoint contour, and target scratch region. When scratches within the target scratch region are considered original scratches within the defective image to be merged, this multi-level correspondence / mapping relationship transforms the determination of whether original scratches belong to the same physical scratch into an association analysis of the fused line segment. This method avoids directly searching or matching multiple regions belonging to the same physical scratch at the original scratch level, thus improving robustness and accuracy.

[0129] In this application, the original scratch corresponding to each fusion line segment is represented as a set. By merging the scratches in this set, the regions in the defective image to be merged that were originally divided into several independent segments but belong to the same physical scratch can be obtained. This scheme can effectively avoid erroneous merging in scenarios with dense scratches (high scratch density) and ensure that the merging result conforms to the real physical structure.

[0130] In this application, the introduction and use of endpoint contours improves the ability to handle curved scratches. By removing and splitting intersecting lines, it addresses the situation where scratches intersect in practical applications. The introduction and use of the first and second constraint conditions effectively reduce the probability of erroneous merging. The technical solution of this application can effectively eliminate the influence of cracks appearing on the wafer surface and enhance anti-interference capabilities.

[0131] In layman's terms, the technical solution of this application is a scratch merging scheme that can accurately merge scratches even when scratches are curved, intersecting, or dense, and effectively avoids erroneous merging. Scaling of unidirectional line segments within the target scratch area is to accelerate calculation and shorten the process. In establishing the second correspondence, using the endpoint segments within a certain pixel distance of the merging line segment to find its corresponding endpoint segments also speeds up the calculation process. In other words, this application also considers computational efficiency.

[0132] In summary, the technical solution of this application enables the accurate merging of multiple scratch regions belonging to the same physical scratch, so as to accurately restore the real scratch structure on the wafer surface, providing technical support for the true and effective presentation of the scratch structure.

[0133] This application provides a defect merging device, such as Figure 10 As shown, it includes: The first obtaining unit 1001 is used to obtain a defective image to be merged, the defective image to be merged includes M target scratch regions, where M is a positive integer greater than or equal to 2; The second obtaining unit 1002 is used to perform a first processing on each target scratch area according to the type of each target scratch area, so as to obtain trace information in each target scratch area. The third acquisition unit 1003 is used to acquire contour decomposition data of each trace information within each target scratch area; The first determining unit 1004 is used to perform a second processing on the contour decomposition data of each trace information to determine the first candidate trace information that is suspected to be a physical scratch. The fourth obtaining unit 1005 is used to obtain the endpoint contour information of the first candidate trace information; The second determining unit 1006 is used to perform third processing on the endpoint contour information of the first candidate trace information to determine the second candidate trace information belonging to the same physical scratch. The third determining unit 1007 is used to determine at least two regions belonging to the same physical scratch in the defect-to-merge image based on the second candidate trace information, the endpoint contour information of the second candidate trace information, and the region information of the target scratch region. The merging unit 1008 is used to merge at least two regions belonging to the same physical scratch in the defective image to be merged.

[0134] In some embodiments, the second obtaining unit 1002 is used to: perform skeletonization processing on the target scratch area to obtain line structure information within the skeletonized area; If the line structure information includes intersecting lines, the type of the target scratch area is determined to be the first type. In response to the target scratch area being the first type, the intersecting area is obtained based on the intersecting lines. Based on the endpoint information of the intersecting lines within the target scratch area, the intersecting area of ​​the intersecting lines, and the line structure information of the intersecting lines outside the intersecting area, the trace information within the target scratch area is obtained. If the line structure information includes non-intersecting lines, the type of the target scratch area is determined to be the second type. Based on the line structure information in the area after skeletonization, the trace information in the target scratch area is obtained.

[0135] In some embodiments, the second obtaining unit 1002 is configured to: If the line structure information includes intersecting lines and non-intersecting lines, the type of the target scratch region is determined to be the third type. In response to the target scratch region being of the third type, the intersecting region is obtained based on the intersecting lines. Based on the line structure information of the non-intersecting lines within the target scratch region, the endpoint information of the intersecting lines, the intersecting region of the intersecting lines, and the line structure information of the intersecting lines outside the intersecting region, the trace information within the target scratch region is obtained.

[0136] In some embodiments, the third obtaining unit 1003 is configured to: perform contouring processing on each trace information within each target scratch area to obtain the contour of the trace information; perform a smoothing operation on the contour of the trace information; and perform line structure decomposition on the smoothed contour to obtain contour decomposition data of the trace information.

[0137] In some embodiments, the trace information corresponds to at least one contour decomposition data; the first determining unit 1004 is configured to: obtain at least one of the following information among the minimum distance information between two or more contour decomposition data, offset information in a preset direction, and directional angle deviation information; based on the at least one of the information between two or more contour decomposition data and a first correlation threshold of the at least one of the information, determine trace information suspected to be physical scratches for each trace information in the defect to be merged image; and identify the trace information suspected to be physical scratches to obtain first candidate trace information.

[0138] In some embodiments, the first candidate trace information corresponds to contour decomposition data with identification information; the fourth obtaining unit 1005 is configured to: obtain the length information of each contour decomposition data in the contour decomposition data identified as having the same identification information; in response to the length information of the contour decomposition data being greater than or equal to a first length threshold, segment the contour decomposition data, and generate endpoint contour information of the contour decomposition data based on the point sequence of the segmented contour decomposition data; in response to the length information of the contour decomposition data being less than the first length threshold, generate endpoint contour information of the contour decomposition data based on the point sequence of the contour decomposition data; and obtain the endpoint contour information of the first candidate trace information based on the endpoint contour information of the contour decomposition data.

[0139] In some embodiments, the second determining unit 1006 is configured to: perform a first sub-processing on the endpoint contour information of the first candidate trace information to determine a third candidate trace information belonging to a physical scratch; perform a second sub-processing on the endpoint contour information of the third candidate trace information to determine a second candidate trace information belonging to the same physical scratch; wherein the third processing includes the first sub-processing and the second sub-processing.

[0140] In some embodiments, the second determining unit 1006 is configured to: obtain the orientation angle information and average orientation angle information of each point in the contour information of each endpoint of each contour decomposition data that is identified with the same identification information; and determine the third candidate trace information belonging to physical scratches from the first candidate trace information based on the orientation angle information and average orientation angle information of each point in the contour information of each endpoint.

[0141] In some embodiments, the second determining unit 1006 is configured to: obtain at least one of the following information among the minimum distance information between two or more endpoint contour information of the third candidate trace information, offset information in a preset direction, and directional angle deviation information; determine trace information belonging to the same physical scratch based on the at least one information between the two or more endpoint contour information and a second correlation threshold of the at least one information; and use the trace information belonging to the same physical scratch as the second candidate trace information.

[0142] In some embodiments, the third determining unit 1007 is configured to: fuse the endpoint contour information of the second candidate trace information to obtain fused trace information; obtain a first correspondence between the endpoint contour information of the second candidate trace information and the region information of the target scratch region; obtain a second correspondence between the fused trace information and the endpoint contour information; obtain a third correspondence between the fused trace information and the target scratch region based on the first and second correspondences; and obtain at least two regions belonging to the same physical scratch in the defective image to be merged through the third correspondence.

[0143] In some embodiments, the first obtaining unit 1001 is configured to: obtain a defective image to be merged, the defective image to be merged including N original scratch regions, where N is a positive integer greater than or equal to 2; use the N original scratch regions as M target scratch regions; or, preprocess the N original scratch regions to obtain M target scratch regions, where M is a positive integer greater than or equal to 2.

[0144] In some embodiments, the first obtaining unit 1001 is configured to: calculate the area of ​​each region in N original scratch regions; determine a plurality of first scratch regions from the N original scratch regions based on the area of ​​each original scratch region; perform a filling operation on each region in the plurality of first scratch regions to obtain a plurality of second scratch regions; perform morphological operations on each region in the plurality of second scratch regions to obtain a plurality of third scratch regions; and use the plurality of third scratch regions as M target scratch regions.

[0145] In some embodiments, the first obtaining unit 1001 is configured to: obtain an image to be identified; input the image to be identified into a defect detection model to obtain a target scratch region output by the defect detection model; and use the image to be identified containing the target scratch region as a defect image to be merged.

[0146] It should be noted that the defect merging device in this application embodiment solves the problem in a similar way to the aforementioned defect merging method. Therefore, the implementation process and implementation principle of the defect merging device can be found in the description of the implementation process and implementation principle of the aforementioned method, and the repeated parts will not be repeated.

[0147] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0148] The electronic device includes at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, enable the at least one processor to perform the defect merging method described in this application. The computer instructions are used to cause the computer to perform the defect merging method described in this application.

[0149] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the defect merging method of this application.

[0150] Figure 11A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0151] like Figure 11 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0152] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0153] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as defect merging methods. For example, in some embodiments, the defect merging method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the defect merging method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform defect merging methods by any other suitable means (e.g., by means of firmware).

[0154] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0155] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0156] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A defect merging method, characterized in that, include: Obtain a defective image to be merged, the defective image to be merged includes M target scratch regions, where M is a positive integer greater than or equal to 2; Based on the type of each target scratch area, the first processing is performed on each target scratch area to obtain the trace information within each target scratch area; Obtain contour decomposition data of each trace within each target scratch area; The contour decomposition data of each trace information is processed a second time to determine the first candidate trace information that is suspected to be a physical scratch. Obtain the endpoint contour information of the first candidate trace information; The endpoint contour information of the first candidate trace information is processed in a third step to determine the second candidate trace information belonging to the same physical scratch. Based on the second candidate trace information, the endpoint contour information of the second candidate trace information, and the regional information of the target scratch area, at least two regions belonging to the same physical scratch in the defect to be merged image are determined. Merge at least two regions belonging to the same physical scratch in the defective images to be merged; Wherein, the first candidate trace information corresponds to contour decomposition data with identification information; The endpoint contour information for obtaining the first candidate trace information includes: Obtain the length information of each contour decomposition data in the contour decomposition data that is identified with the same identification information; In response to the length information of the contour decomposition data being greater than or equal to a first length threshold, the contour decomposition data is segmented, and the endpoint contour information of the contour decomposition data is generated based on the point sequence of the segmented contour decomposition data. In response to the length information of the contour decomposition data being less than a first length threshold, the endpoint contour information of the contour decomposition data is generated based on the point sequence of the contour decomposition data. Based on the endpoint contour information of the contour decomposition data, the endpoint contour information of the first candidate trace information is obtained.

2. The method according to claim 1, characterized in that, The first processing step, based on the type of each target scratch region, to obtain trace information within each target scratch region includes: Skeletonization is performed on the target scratch area to obtain the line structure information within the skeletonized area; If the line structure information includes intersecting lines, the type of the target scratch area is determined to be the first type. In response to the target scratch area being the first type, the intersecting area is obtained based on the intersecting lines. Based on the endpoint information of the intersecting lines within the target scratch area, the intersecting area of ​​the intersecting lines, and the line structure information of the intersecting lines outside the intersecting area, the trace information within the target scratch area is obtained. If the line structure information includes non-intersecting lines, the type of the target scratch area is determined to be the second type. Based on the line structure information in the area after skeletonization, the trace information in the target scratch area is obtained. If the line structure information includes intersecting lines and non-intersecting lines, the type of the target scratch region is determined to be the third type. In response to the target scratch region being of the third type, the intersecting region is obtained based on the intersecting lines. Based on the line structure information of the non-intersecting lines within the target scratch region, the endpoint information of the intersecting lines, the intersecting region of the intersecting lines, and the line structure information of the intersecting lines outside the intersecting region, the trace information within the target scratch region is obtained.

3. The method according to claim 1, characterized in that, The contour decomposition data for obtaining trace information within each target scratch area includes: The trace information within each target scratch area is contoured to obtain the trace information contour. Smooth the outline of the trace information; The smoothed contour is decomposed into line structures to obtain contour decomposition data of trace information.

4. The method according to claim 1, characterized in that, The trace information corresponds to at least one contour decomposition data; The second processing of the contour decomposition data of each trace information to determine the first candidate trace information that is suspected to be a physical scratch includes: Obtain at least one of the following: minimum distance information between two or more contour decomposition data, offset information in a preset direction, and orientation angle deviation information; Based on at least one of the information between two or more contour decomposition data and a first correlation threshold of the at least one information, for each trace information in the defective image to be merged, trace information suspected to be physical scratches is determined; Marking the trace information that is suspected to be a physical scratch, we obtain the first candidate trace information.

5. The method according to claim 1, characterized in that, The third processing of the endpoint contour information of the first candidate trace information to determine the second candidate trace information belonging to the same physical scratch includes: The endpoint contour information of the first candidate trace information is processed in the first sub-process to determine the third candidate trace information that belongs to the physical scratch. The endpoint contour information of the third candidate trace information is subjected to a second sub-processing to determine the second candidate trace information belonging to the same physical scratch. The third process includes the first sub-process and the second sub-process.

6. The method according to claim 5, characterized in that, The first candidate trace information corresponds to contour decomposition data with identification information; The first sub-processing of the endpoint contour information of the first candidate trace information to determine the third candidate trace information belonging to physical scratches includes: Obtain the orientation angle information and average orientation angle information of each point in the endpoint contour information of each contour decomposition data that is identified with the same identification information; Based on the orientation angle information and average orientation angle information of each point in the contour information of each endpoint, the third candidate trace information belonging to physical scratches is determined from the first candidate trace information. The second sub-processing of the endpoint contour information of the third candidate trace information to determine the second candidate trace information belonging to the same physical scratch includes: The minimum distance information between the two or more endpoint contour information of the third candidate trace information, the offset information in the preset direction, and the orientation angle deviation information are obtained; Based on at least one of the endpoint contour information and a second correlation threshold of the at least one information, trace information belonging to the same physical scratch is determined; Trace information belonging to the same physical scratch is used as the second candidate trace information.

7. The method according to claim 1, characterized in that, Based on the second candidate trace information, the endpoint contour information of the second candidate trace information, and the region information of the target scratch region, at least two regions belonging to the same physical scratch in the defective image to be merged are determined, including: The endpoint contour information of the second candidate trace information is fused to obtain the fused trace information; The first correspondence between the endpoint contour information of the second candidate trace information and the region information of the target scratch area is obtained; Obtain a second correspondence between fusion trace information and endpoint contour information; Based on the first and second correspondences, a third correspondence is obtained between the fused trace information and the target scratch area; By using the third correspondence, at least two regions belonging to the same physical scratch in the defective images to be merged are obtained.

8. The method according to claim 1, characterized in that, The process of obtaining the defective image to be merged includes: Obtain a defective image to be merged, the defective image to be merged includes N original scratch regions, where N is a positive integer greater than or equal to 2; Use N original scratch regions as M target scratch regions; or, preprocess the N original scratch regions to obtain M target scratch regions, where M is a positive integer greater than or equal to 2. The preprocessing of the N original scratch areas includes: Calculate the area of ​​each region in the N original scratch regions; Based on the area of ​​each original scratch region, multiple first scratch regions are determined from N original scratch regions; Fill each region within the multiple first scratch regions to obtain multiple second scratch regions; Morphological operations are performed on each region within the multiple second scratch regions to obtain multiple third scratch regions; Multiple third scratch areas are defined as M target scratch areas.

9. A defect merging device, characterized in that, include: The first obtaining unit is used to obtain a defective image to be merged, the defective image to be merged includes M target scratch regions, where M is a positive integer greater than or equal to 2; The second obtaining unit is used to perform a first processing on each target scratch area according to the type of each target scratch area, so as to obtain trace information in each target scratch area. The third acquisition unit is used to acquire contour decomposition data of each trace information within each target scratch area; The first determining unit is used to perform a second processing on the contour decomposition data of each trace information to determine the first candidate trace information that is suspected to be a physical scratch. The fourth obtaining unit is used to obtain the endpoint contour information of the first candidate trace information; The second determining unit is used to perform third processing on the endpoint contour information of the first candidate trace information to determine the second candidate trace information belonging to the same physical scratch. The third determining unit is used to determine at least two regions belonging to the same physical scratch in the defect-to-merge image based on the second candidate trace information, the endpoint contour information of the second candidate trace information, and the region information of the target scratch region. The merging unit is used to merge at least two regions belonging to the same physical scratch in the defective image to be merged; Wherein, the first candidate trace information corresponds to contour decomposition data with identification information; The fourth obtaining unit is further configured to: Obtain the length information of each contour decomposition data in the contour decomposition data that is identified with the same identification information; In response to the length information of the contour decomposition data being greater than or equal to a first length threshold, the contour decomposition data is segmented, and the endpoint contour information of the contour decomposition data is generated based on the point sequence of the segmented contour decomposition data. In response to the length information of the contour decomposition data being less than a first length threshold, the endpoint contour information of the contour decomposition data is generated based on the point sequence of the contour decomposition data. Based on the endpoint contour information of the contour decomposition data, the endpoint contour information of the first candidate trace information is obtained.

Images