A marking area determination method, apparatus, storage medium, and device

Abstract

In a marking area determination method, device, storage medium and equipment provided in the application, a to-be-marked image is first acquired, and a plurality of initial image areas are identified from the to-be-marked image. Then, the similarity between the plurality of initial image areas is calculated, and based on the similarity, a target image area is screened from the plurality of initial image areas. After that, a plurality of candidate marking areas of different sizes are segmented from the target image area, the uniqueness evaluation value of the candidate marking areas in the to-be-marked image is calculated, and based on the uniqueness evaluation values of the plurality of candidate marking areas, a target marking area is determined. This method can find a target marking area with obvious distinguishing features from the to-be-marked image through two stages of matching, thereby not only reducing the cost, but also improving the accuracy of determining the marking area.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, and in particular to a marked region determination method and device, a storage medium and an equipment. BACKGROUND

[0002] In the fields of semiconductor manufacturing, printed circuit board manufacturing, and flat panel display manufacturing, a process of using mark points (i.e., Marks) to align and position images is involved to detect whether there are defects in the panels, PCB boards, wafers, etc. in the manufacturing process, thereby improving the yield rate.

[0003] In practical applications, there are mainly two ways to set mark points. One is to set physical mark points in the articles, and the other is to add virtual mark points in the collected images of the panels, PCB boards, wafers, etc.

[0004] Although both of the two ways can achieve subsequent image alignment, positioning, etc., they both have certain problems.

[0005] For the first way, the addition of physical mark points not only increases the cost, but also may damage the target device. For the second way, the virtual mark points are often added in the images by human experience, which makes the added virtual mark points not have significant distinguishing features compared with other elements in the images, so that the accurate alignment and positioning of the images cannot be achieved in the subsequent process.

[0006] Based on this, the present application provides a marked region determination method, device, storage medium and equipment. SUMMARY

[0007] The present application provides a marked region determination method, device, storage medium and equipment to partially solve the above-mentioned problems existing in the prior art.

[0008] The present application adopts the following technical solution: the present application provides a marked region determination method, which comprises:

[0009] obtaining a to-be-marked image, and identifying a plurality of initial image regions from the to-be-marked image;

[0010] calculating the similarity between the plurality of initial image regions;

[0011] selecting a target image region from the plurality of initial image regions based on the similarity;

[0012] dividing a plurality of candidate marked regions of different sizes from the target image region;

[0013] calculating a uniqueness evaluation value of the candidate marked region in the to-be-marked image, the uniqueness evaluation value representing the uniqueness of the pattern of the candidate marked region in the to-be-marked image;

[0014] determine the target marking area based on the uniqueness evaluation values of the plurality of candidate marking areas.

[0015] Optionally, the similarity between the plurality of initial image areas is calculated, specifically including:

[0016] For each initial image area, the initial image area is divided to obtain a plurality of segmentation areas, and the extraction feature corresponding to the initial image area is determined according to the image features of the plurality of segmentation areas.

[0017] The similarity between the plurality of initial image areas is calculated according to the extraction features corresponding to the initial image areas.

[0018] Optionally, the initial image area is divided to obtain a plurality of segmentation areas, specifically including:

[0019] The initial image area is divided into a plurality of segmentation areas in the horizontal direction, and / or the initial image area is divided into a plurality of segmentation areas in the vertical direction.

[0020] Optionally, the extraction feature corresponding to the initial image area is determined according to the image features of the plurality of segmentation areas, specifically including:

[0021] For each segmentation area, the average gray value corresponding to the segmentation area is determined according to the gray values of the pixel points contained in the segmentation area, and the gray value standard deviation corresponding to the segmentation area is determined according to the average gray value corresponding to the segmentation area and the gray values of the pixel points contained in the segmentation area.

[0022] The average gray value corresponding to the segmentation area and / or the gray value standard deviation corresponding to the segmentation area is determined as the image feature of the segmentation area, so as to determine the extraction feature corresponding to the initial image area according to the image features of the segmentation areas.

[0023] Optionally, the extraction feature corresponding to the initial image area is determined according to the image features of the segmentation areas, specifically including:

[0024] According to the center line of the initial image area, at least one group of symmetrical segmentation areas located on both sides of the center line is determined from the segmentation areas of the initial image area.

[0025] The symmetry feature of the initial image area is determined according to the image features of the at least one group of symmetrical segmentation areas, and the extraction feature corresponding to the initial image area is determined based on the symmetry feature, the average gray value and the gray value standard deviation.

[0026] Optionally, the symmetry feature of the initial image area is determined according to the image features of the at least one group of symmetrical segmentation areas, specifically including:

[0027] obtaining a plurality of feature fusion strategies for calculating symmetric features;

[0028] from a plurality of symmetric partitioned regions, selecting at least one symmetric partitioned region corresponding to each feature fusion strategy;

[0029] based on each feature fusion strategy, fusing image features of the at least one symmetric partitioned region corresponding to the feature fusion strategy to obtain a plurality of symmetric features.

[0030] Optionally, a plurality of initial image regions are identified from the image to be marked, specifically including:

[0031] slidingly sampling the image to be marked by a preset sliding window according to a preset path with a preset step to obtain a plurality of initial image regions, the preset step being smaller than the size of the preset sliding window.

[0032] Optionally, based on the similarity, a target image region is selected from the plurality of initial image regions, specifically including:

[0033] for each initial image region, the similarity between the initial image region and a plurality of other initial image regions is sorted from large to small, and the similarity located before the first set ranking is taken as the target similarity corresponding to the initial image region;

[0034] the target similarity corresponding to each initial image region is sorted from small to large, and the initial image region located before the second set ranking is determined as the target image region.

[0035] Optionally, the similarity between the plurality of initial image regions is calculated according to the extraction features corresponding to the initial image regions, including:

[0036] the distance values between the extraction features corresponding to each initial image region are calculated, and the distance values are determined as the similarity between the plurality of initial image regions;

[0037] based on the similarity, a target image region is selected from the plurality of initial image regions, specifically including:

[0038] for each initial image region, the distance values between the initial image region and a plurality of other initial image regions are sorted from small to large, and the distance value located before the third set ranking is taken as the target distance value corresponding to the initial image region;

[0039] the target distance value corresponding to each initial image region is sorted from large to small, and the initial image region located before the fourth set ranking is determined as the target image region.

[0040] Optionally, based on the uniqueness evaluation value of the plurality of candidate marking regions, a target marking region is determined, specifically including:

[0041] The plurality of candidate marking regions are sorted in descending order of the uniqueness evaluation values, and a candidate marking region ranked before a fifth preset rank is taken as the target marking region.

[0042] Optionally, the uniqueness evaluation value of the candidate marking region in the image to be marked is calculated, and the calculation specifically includes:

[0043] For each candidate marking region, the candidate marking region is slid in the image to be marked to calculate the matching degree between the candidate marking region and each sliding region.

[0044] The plurality of matching degrees are sorted in descending order, and a matching degree located before a sixth preset rank and smaller than a preset value is determined as the target matching degree of the candidate marking region, and the target matching degree is determined as the uniqueness evaluation value of the candidate marking region in the image to be marked.

[0045] The target marking region is determined based on the uniqueness evaluation values of the plurality of candidate marking regions, and the determination specifically includes:

[0046] The plurality of candidate marking regions are sorted in ascending order of the target matching degrees, and a candidate marking region ranked before a seventh preset rank is taken as the target marking region.

[0047] Optionally, the method further includes:

[0048] An other image is acquired, and the other image and the image to be marked are images to be aligned.

[0049] An image region matched with the target marking region in the other image is determined as an alignment region corresponding to the other image.

[0050] The image to be marked and the other image are aligned according to the target marking region and the alignment region.

[0051] Optionally, the image to be marked and the other image are wafer images.

[0052] The method further includes:

[0053] Wafer defect detection is performed according to the aligned image to be marked and the other image.

[0054] The application provides a marking region determination device, including:

[0055] An acquisition module is configured to acquire an image to be marked and identify a plurality of initial image regions from the image to be marked.

[0056] A first calculation module is configured to calculate the similarity between the plurality of initial image regions.

[0057] A screening module is configured to screen a target image region from the plurality of initial image regions based on the similarity.

[0058] a segmentation module configured to segment a plurality of candidate marking regions of different sizes from the target image region;

[0059] a second calculation module configured to calculate a uniqueness evaluation value of the candidate marking regions in the image to be marked, the uniqueness evaluation value representing a uniqueness degree of a pattern of the candidate marking regions in the image to be marked;

[0060] a determination module configured to determine the target marking region based on the uniqueness evaluation values of the plurality of candidate marking regions.

[0061] The application provides a computer readable storage medium, the storage medium has computer programs / instructions, the computer programs / instructions are executed by a processor to realize the above marking region determination method.

[0062] The application provides an electronic device, comprising a memory, a processor and a computer program stored on the memory and running on the processor, the processor executes the program to realize the above marking region determination method.

[0063] The application provides a computer program product, comprising computer programs / instructions, the computer programs / instructions are executed by a processor to realize the steps of the above marking region determination method.

[0064] The above at least one technical scheme adopted by the application can achieve the following beneficial effects: first, the image to be marked is obtained, and a plurality of initial image regions are identified from the image to be marked, then, the similarity between the plurality of initial image regions is calculated, and the target image region is selected from the plurality of initial image regions based on the similarity. Then, a plurality of candidate marking regions of different sizes are segmented from the target image region, and the uniqueness evaluation value of the candidate marking regions in the image to be marked is calculated, and the target marking region is determined based on the uniqueness evaluation values of the plurality of candidate marking regions.

[0065] As can be seen from the above method, the application can first determine a plurality of target image regions with significant distinguishing characteristics from the overall perspective of the image to be marked by calculating and comparing the similarity between the image regions, the target image regions have significant characteristics compared to most image regions, then, the target image regions are segmented to obtain a plurality of candidate marking regions of different sizes, and the uniqueness evaluation value of the candidate marking regions is calculated, so as to further determine the most unique target marking region from the global perspective of the image to be marked from the plurality of candidate marking regions of different sizes and from different target image regions. This not only does not increase additional physical marking points, but also reduces the cost, and can ensure that the determined target marking region can be significantly distinguished from other image elements in the image to be marked, so as to ensure the accurate alignment and positioning of the subsequent image. BRIEF DESCRIPTION OF DRAWINGS

[0066] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0067] Figure 1 A flowchart of a marker region determination method provided in an embodiment of the application;

[0068] Figure 2A And Figure 2B A schematic diagram of two image capturing modes provided in an embodiment of the application;

[0069] Figure 3A And Figure 3B A schematic diagram of dividing an initial image region in horizontal and vertical directions provided in an embodiment of the application;

[0070] Figure 4A And Figure 4B A process schematic diagram of determining a plurality of candidate marker regions of different sizes provided in an embodiment of the application;

[0071] Figure 5 A schematic diagram of a marker region determination device provided in an embodiment of the application;

[0072] Figure 6 A schematic diagram of an electronic device structure for implementing a marker point determination method provided in the application. DETAILED DESCRIPTION

[0073] In order to make the objects, technical solutions and advantages of the application clearer, the following will combine the embodiments of the application and corresponding drawings to clearly and completely describe the technical solutions of the application. Obviously, the described embodiments are only a part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without any creative work fall within the protection scope of the application.

[0074] In order to avoid the problems caused by the traditional marker point determination method, in the embodiments of the application, a target marker region is determined from a to-be-labeled image through a two-stage image matching process. The target marker region can be regarded as a marker point determined from the to-be-labeled image, which can not only significantly reduce the cost problem caused by setting a physical marker point, but also ensure the accuracy of the determined target marker region.

[0075] The marker region determination method provided in the embodiments of the present application is similar to the way of determining the virtual marker point in the prior art. However, unlike the way of determining the virtual marker point in the prior art by using artificial experience, the method provided in the embodiments of the present application mainly determines the target marker region with significant distinction on the image features by analyzing the image features of the plurality of sub-image regions in the image.

[0076] In addition, the marker region determination method provided in the embodiments of the present application can be applied in many fields. For example, the marker region determination method provided in the embodiments of the present application can be applied to a quantity detection device in the semiconductor field, which includes various types of quantity detection function devices involved in the whole manufacturing process of semiconductors, to realize quantity detection functions such as thin film measurement, line width measurement, overlay accuracy measurement, bump detection, defect detection, and the like.

[0077] The quantity detection device includes components such as a control system, a stage, a robot, and the like. The control system includes a terminal (such as an industrial computer), a server, a motion controller, and the like. The devices in the control system are communicatively connected based on actual needs. The marker region and the schemes such as image alignment and wafer positioning based on the marker region can be implemented on a computer or a server, and relevant information can be displayed through a visual interface of the quantity detection device, the motion controller can be used to control the movement of the robot and / or the stage, and relevant information can be reported to a manufacturing execution system of a factory.

[0078] Of course, the marker region determination method provided in the embodiments of the present application can also be implemented on an electronic device (terminal, server, and the like) before the quantity detection device is put into the production line, and the obtained marker region can be saved in the control system of the quantity detection device for subsequent use.

[0079] The technical solutions provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0080] Figure 1 A flowchart of a marker region determination method provided in the embodiments of the present application includes the following steps:

[0081] S100: Obtain a to-be-labeled image, and identify a plurality of initial image regions from the to-be-labeled image.

[0082] In the embodiments of the present application, an image for which a marker region needs to be determined, i.e., a to-be-labeled image, can be obtained. The to-be-labeled image can be obtained by image acquisition on a specified target object. The specific form of the target object is determined according to the actual application scenario. For example, in the semiconductor manufacturing field, the target object can be a wafer, a die, a chip, or the like to be detected. For another example, in the printed circuit board manufacturing field, the target object can be a printed circuit board to be detected. Other specific forms of the target object are not illustrated one by one here.

[0083] The marking region determination method provided in this application can be executed by various entities, including terminal devices such as desktop computers and laptops, clients installed on terminal devices, servers, or metrology equipment specifically used for wafer / chip metrology inspection. For ease of explanation, the following description uses a metrology equipment as an example to illustrate the marking region determination method provided in this application.

[0084] After acquiring the image to be labeled, it is necessary to identify and segment the image to determine multiple initial image regions. The initial image regions may include a portion of the image to be labeled. In this embodiment, multiple initial image regions can be segmented in various ways. For example, the quantity detection device can identify various units contained in the image to be labeled and determine the position of each unit in the image. Then, using the center point of each unit's (such as spatially adjacent superpixels with similar grayscale / color) position in the image to be labeled as the center, initial image regions of a preset size are determined.

[0085] For example, the measurement detection device can preset the step and slide the image to be marked through a preset sliding window according to the preset path to obtain multiple initial image regions.

[0086] Among these methods, the quantity detection device can identify multiple initial image regions by sliding a preset sliding window across the image to be labeled in various ways, such as... Figure 2A and Figure 2B As shown.

[0087] Figure 2A and Figure 2B This is a schematic diagram of two image acquisition methods provided in the embodiments of this application.

[0088] exist Figure 2A In the image, the dashed box on the left represents a sliding window used to divide and identify the initial image region. Therefore, it is possible to... Figure 2A Multiple initial image regions are determined in the image to be labeled on the right using this sliding window. Among them, in Figure 2A The initial image regions identified do not overlap with each other. Therefore, Figure 2A The method shown can be understood as performing image sampling in the image to be labeled without repetition, so that the initial image regions obtained by the previous and subsequent samplings do not overlap with each other.

[0089] And in Figure 2B In the middle, the dashed box on the left is still used to represent the sliding window for dividing and recognizing the initial image region, except that... Figure 2BIn the image to be labeled on the right, the quantity detection device can slide the sliding window in a preset step size smaller than the sliding window to divide the image into initial image regions. The two initial image regions divided by the sliding window after two slides have overlapping parts.

[0090] In both of the above methods, the sliding path of the sliding window can be varied. For example, it can start from the top left corner of the image to be labeled, slide from left to right to divide an initial image region, then move the sliding window down by a set step, and then slide from left to right until all initial image regions are divided; or, for another example, it can start from the top left corner of the image to be labeled, slide from top to bottom to divide a column of initial image regions, then move the sliding window to the right by a set step, and then slide from top to bottom until all initial image regions are divided.

[0091] Of course, except Figure 2A and Figure 2B Besides the method shown, there are other ways to divide the path, which will not be detailed here. Similarly, in practical applications, other sliding paths besides the ones mentioned above can also be included, which will not be illustrated here.

[0092] It is important to emphasize that in order to ensure that the final target marking region is significantly distinguishable in the entire image to be marked, it is necessary to ensure that each initial image region is divided to cover all image regions in the image to be marked as a whole, and the preset step size needs to be smaller than the preset sliding window size.

[0093] S102: Calculate the similarity between multiple initial image regions.

[0094] In the embodiments of the present application, the quantity detection device can calculate the similarity between the plurality of initial image regions in a plurality of ways. The similarity can represent the degree of similarity / correlation between the image regions, for example, the quantity detection device can determine the corresponding gray scale image of each initial image region (if the image to be marked is a gray scale image, it can be used directly), and then for any two initial image regions, the joint distribution of the two initial image regions is counted by using a histogram. This joint distribution can reflect the combination frequency of the gray scale values of the pixels in the same image position of the two initial image regions. If the two initial image regions have a high frequency in some gray scale combinations, it means that the two initial image regions have strong correlation. Then, according to the joint distribution between the initial image regions, the mutual information value between the initial image regions is calculated, and then according to the calculated mutual information value, the similarity between the initial image regions is determined. For two initial image regions, the greater the mutual information value between the two initial image regions, the more shared information the two initial image regions contain, and the higher the similarity between the two initial image regions.

[0095] For another example, after the quantity detection device divides the initial image regions, it can determine the image features corresponding to each initial image region. Then, according to the distance between the image features corresponding to the initial image regions, the similarity between the initial image regions is calculated. The image features can be determined according to the pixel values of the pixels contained in the image region. Then, the quantity detection device can use a plurality of ways to determine the image features, for example, by calculating the average value of the pixel values of the pixels in the initial image region to determine the image features; for another example, by calculating the pixel value gradient in the initial image region to determine the image features. Other ways will not be illustrated one by one here.

[0096] When determining the image features corresponding to the initial image regions, the quantity detection device can further divide the initial image regions, so as to determine the image features of the initial image regions based on the divided segmentation regions.

[0097] Specifically, for each initial image region, the quantity detection device can divide the initial image region to obtain a plurality of segmentation regions, and then determine the image features corresponding to each segmentation region according to the pixel values of the pixels in each segmentation region, and then determine the extraction features corresponding to the initial image region according to the image features corresponding to each segmentation region. The extraction features can include the representative information of the image region in several dimensions.

[0098] The segmented region can include a part of the initial image region, and the quantity detection device can segment the initial image region from a horizontal direction or from a vertical direction, as shown in Figure 3A and Figure 3B .

[0099] Figure 3A and Figure 3B are schematic diagrams provided by embodiments of the present application for dividing the initial image region from the horizontal direction and the vertical direction.

[0100] Figure 3A are eight segmented regions obtained by the quantity detection device segmenting the initial image region from the horizontal direction, as shown in Figure 3B are eight segmented regions obtained by the quantity detection device segmenting the initial image region from the vertical direction.

[0101] In embodiments of the present application, the quantity detection device can segment the initial image region only from the horizontal direction, only from the vertical direction, or both from the horizontal direction and the vertical direction. If the above examples shown in Figure 3A and Figure 3B are followed, when the quantity detection device segments the same initial image region from the horizontal direction and the vertical direction, then Figure 3A and Figure 3B 16 segmented regions will be segmented, as shown in

[0102] After the segmented regions are segmented, the quantity detection device can determine, for each segmented region, an average gray value of the segmented region according to the gray values of the pixels included in the segmented region, and determine a gray value standard deviation corresponding to the segmented region according to the average gray value corresponding to the segmented region and the gray values of the pixels included in the segmented region, and then the determined average gray value and / or gray value standard deviation can be determined as the image feature corresponding to the segmented region.

[0103] The image feature of the segmented region can reflect the information of a plurality of specific local parts of the initial image region, in order to more comprehensively extract the information of the initial image region, the present application analyzes the symmetry of a plurality of segmented regions in the initial image region, and fuses the features obtained by the symmetry analysis into the extracted features of the initial image region.

[0104] Specifically, the quantity detection device can determine, for each of the segmented regions, at least one set of symmetric segmented regions from the segmented regions in the initial image region in which the segmented region is located, according to a center line of the initial image region, and determine the symmetry feature of the initial image region based on image features of the symmetric segmented regions.

[0105] The symmetry feature can represent a degree of symmetry of the image region relative to the center line. For the determination of the symmetry feature described above, since different symmetric regions are located at different distances from the center line, in order to better reflect whether the pattern contained in the initial image region has symmetry, different feature fusion strategies can be used for the combination of different symmetric segmented regions. These feature fusion strategies can be determined based on human experience.

[0106] Specifically, the quantity detection device can obtain a plurality of feature fusion strategies for calculating the symmetry feature, which can be pre-determined and saved, and when the symmetry feature needs to be determined, the feature fusion strategies are read from the storage space.

[0107] And for different feature fusion strategies, they correspond to different sets of symmetric segmented regions, wherein for any one feature fusion strategy, it can correspond to not only one set of symmetric segmented regions, but also multiple sets of symmetric segmented regions.

[0108] Therefore, the quantity detection device can select at least one set of symmetric segmented regions corresponding to each feature fusion strategy from multiple sets of symmetric segmented regions, and then fuse image features of the at least one set of symmetric segmented regions corresponding to each feature fusion strategy based on each feature fusion strategy to obtain multiple symmetry features.

[0109] For the feature fusion strategies corresponding to different sets of symmetric segmented regions, reference can be made to Table 1 and Table 2 as follows.

[0110] Table 1

[0111]

[0112] In the above Table 1, it is derived under the background of dividing the initial image region into 8 segmented regions, wherein HM2-HM7 represent average gray values corresponding to the 2nd segmented region to the 7th segmented region, and the segmented regions can be as shown in Figure 3A and / or Figure 3B The HZ1-HZ5 are symmetry features determined to reflect the symmetry of the image to some extent.

[0113] For the symmetry feature of HZ1, the difference between the average gray value of the fifth partition region and the average gray value of the fourth partition region is calculated. If the difference is smaller, it means that the fourth partition region and the corresponding symmetric region, the fifth partition region, are close in image features, and the initial image region contains image elements with strong symmetry (or the symmetry of the image elements crossed by the center line of the initial image region is strong). On the contrary, it means that the two partition regions have certain differences in image features, and the image elements contained in the initial image region are weak in symmetry (or the symmetry of the image elements crossed by the center line of the initial image region is weak). And (HM5-HM4) x 2 is the feature fusion strategy corresponding to the fourth partition region and the fifth partition region.

[0114] Similarly, for the symmetry feature of HZ2, the difference between the average gray value of the fifth partition region and the average gray value of the fourth partition region, and the difference between the average gray value of the sixth partition region and the average gray value of the third partition region are calculated, and the sum of the two differences is calculated. If the initial image region contains image elements with strong symmetry, the sum value is low in value, reflecting that the sixth partition region and the corresponding symmetric region, the third partition region, are close in image features, and the fourth partition region and the corresponding symmetric region, the fifth partition region, are close in image features, and it can be determined that the initial image region contains image elements with strong symmetry. On the contrary, if the sum value is low in value, it means that the image elements contained in the initial image region are weak in symmetry. And HM5-HM4+HM6-HM3 is the feature fusion strategy corresponding to the fourth partition region and the fifth partition region, and the third partition region and the sixth partition region.

[0115] The symmetry feature HZ3 is also the same, and will not be illustrated in detail. The symmetry features HZ4 and HZ5 can also reflect the symmetry of the image to some extent. Taking the symmetry feature HZ4 as an example, the calculated average value can be compared with the average gray value of the fourth partition region and the average gray value of the fifth partition region after the average gray value of the fourth partition region and the average gray value of the fifth partition region are further averaged. If the calculated average value is not much different from the average gray value of the fourth partition region and the average gray value of the fifth partition region, it indicates that the initial image region contains image elements with strong symmetry, otherwise, it indicates that the image elements contained in the initial image region have weak symmetry. The same is true for HZ5, which will not be illustrated in detail. The feature fusion strategy for determining the symmetry feature HZ5 is the feature fusion strategy of the second partition region and the seventh partition region, the third partition region and the sixth partition region, and the fourth partition region and the fifth partition region.

[0116] Table 2

[0117]

[0118] In the above table 2, it is also derived under the background of dividing the initial image region into 8 partition regions, wherein VM2-VM7 represent the gray value standard deviation of the second partition region to the seventh partition region, and the partition region can be divided as shown in Figure 3A and / or Figure 3B , or can be obtained based on other reasonable division methods. VZ1-VZ5 are symmetry features determined to reflect the symmetry of the image to some extent. For specific explanations, refer to the explanation of the symmetry features in table 1, which will not be described in detail here.

[0119] The present application can determine the extraction features of the initial image region according to the image features, symmetry features and the like of the partition region. The image features of the initial image region directly determined in the foregoing are also one kind of extraction features.

[0120] The quantity detection device can use various ways to obtain the above extraction features in the process of determining the above extraction features.

[0121] For example, the quantity detection device can fuse the determined average gray value, the gray value standard deviation and the symmetry feature of each segmentation region by weighting, obtain the fusion feature of the segmentation region, and then add the fusion features corresponding to each segmentation region in the initial image region to obtain the extraction feature corresponding to the initial image region. For another example, the quantity detection device can re-average the average gray values of all segmentation regions to obtain the average gray value corresponding to the initial image region, average the gray value standard deviations of all segmentation regions to obtain the average standard deviation corresponding to the initial image region, and average the symmetry features of all segmentation regions to obtain the average symmetry feature corresponding to the initial image region, and then fuse the average gray value corresponding to the initial image region, the average standard deviation corresponding to the initial image region and the average symmetry feature corresponding to the initial image region to obtain the extraction feature corresponding to the initial image region. For another example, for each initial image region, the quantity detection device can splice the average gray value, the gray value standard deviation and the symmetry feature of each segmentation region included in the initial image region to obtain a vector with a dimension smaller than the pixel size of the initial image region, and take the vector as the extraction feature corresponding to the initial image region. Other manners are exemplified one by one herein.

[0122] After determining the extraction feature corresponding to each initial image region, the similarity between the initial image regions can be calculated, that is, the quantity detection device can calculate the similarity between each initial image region and other initial image regions, so as to perform the first region screening based on the calculated similarity in the subsequent process.

[0123] In the embodiments of the present application, there can be various ways for the quantity detection device to calculate the similarity between the initial image regions, for example, the similarity between the initial image regions can be determined by calculating the Euclidean distance between the extraction features corresponding to the initial image regions; for another example, the similarity between the initial image regions can be determined by calculating the cosine distance between the extraction features corresponding to the initial image regions.

[0124] It should be noted that the distance value calculated by using the distance calculation method to determine the similarity is negatively correlated with the similarity, that is, the greater the distance value between the extraction features corresponding to two initial image regions, the less similar the two initial image regions, and the lower the similarity between the two initial image regions. Conversely, the smaller the distance value between the extraction features corresponding to two initial image regions, the more similar the two initial image regions, and the higher the similarity between the two initial image regions.

[0125] Therefore, in the embodiments of the present application, the quantity detection device can directly determine the distance value between the extracted features corresponding to the initial image regions as the similarity between the initial image regions, so that the distance value and the similarity are equivalent in value. Of course, the quantity detection device can also determine the similarity between the initial image regions based on the calculated distance value in a manner that the distance value and the similarity are negatively correlated.

[0126] The quantity detection device can also calculate the similarity between the initial image regions in other manners. For example, the quantity detection device can input the extracted features corresponding to the initial image regions into a preset model, determine the correlation degree of the extracted features in the feature space based on the feature expression analysis capability learned by the preset model through pre-training, and output the similarity between the initial image regions. Other manners of calculating the similarity will not be illustrated one by one here.

[0127] It should be noted that, before the plurality of initial image regions are divided from the to-be-labeled image, the to-be-labeled image can be preprocessed. For example, the to-be-labeled image can be scaled to adjust to a unified specified pixel size; for another example, the to-be-labeled image can be subjected to noise filtering processing to eliminate noise pixels contained in the to-be-labeled image. Other preprocessing manners will not be illustrated one by one here.

[0128] S104: screening a target image region from the plurality of initial image regions based on the similarity.

[0129] For any one initial image region, the greater the similarity between the initial image region and other initial image regions, the stronger the commonality of the initial image region and the other initial image regions in the graphical features, and vice versa.

[0130] On this basis, the quantity detection device can screen a target image region having a significant difference from the plurality of initial image regions based on the determined similarity. In the embodiments of the present application, there can be various manners of screening the target image region based on the similarity. For example, the quantity detection device can sort the similarities between each initial image region and a plurality of other initial image regions from large to small, and take the similarity located before the first set ranking as the target similarity corresponding to the initial image region. Then, the quantity detection device can sort the target similarities corresponding to the initial image regions from small to large, and further determine the initial image region located before the second set ranking as the screened target image region.

[0131] For another example, after determining the similarity, the quantity detection device can further determine a score corresponding to each initial image region according to the similarity and the texture feature of each initial image region, and select the target image region from the plurality of initial image regions according to the calculated score. If the similarity corresponding to an initial image region is lower and the texture feature is more obvious, the score corresponding to the initial image region is higher. Therefore, the initial image region with the highest score can be selected as the target image region. Other manners are not exemplified here.

[0132] As can be seen from the above method, for each initial image region, the quantity detection device actually determines the greater (or maximum) similarity between the initial image region and other initial image regions as a representative of whether the pattern of the initial image region is similar to other patterns in the entire image to be marked (i.e., target similarity). Then, the initial image region with the smaller (or smallest) target similarity is selected as the determined target image region.

[0133] Therefore, if the target similarity of an initial image region is smaller, it means that the difference between the pattern on the initial image region and other patterns in the entire image to be marked is greater, and the uniqueness of the initial image region in the entire image to be marked is stronger.

[0134] Therefore, the target image region selected by the above method is a region with strong uniqueness in the entire image to be marked, which is significantly different from other initial image regions in pattern characteristics.

[0135] In addition, the quantity detection device can use the calculated distance value as the similarity, and the distance value and the similarity are in a negative correlation relationship. Therefore, in the process of selecting the target image region based on the similarity, the quantity detection device can sort the distance values between each initial image region and a plurality of other initial image regions from small to large, and use the distance value before the third set ranking as the target distance value corresponding to the initial image region. Then, the quantity detection device can sort the target distance values corresponding to the initial image regions from large to small, and determine the initial image region before the fourth set ranking as the target image region.

[0136] Similar to the above principle, the target image region is screened by the target distance value, which is essentially to screen the initial image region with strong uniqueness from the image to be marked. Only because the distance value here is negatively correlated with the similarity, the distance values between the initial image region and multiple other initial image regions are sorted from small to large, and the smaller (or smallest) distance value between the initial image region and other initial image regions is determined as a representative of whether the pattern of the initial image region is similar to other patterns in the entire image to be marked (i.e., the target distance value), and then the target distance value is sorted from large to small to screen the target image region, and the target image region with weak commonality or not similar to other initial image regions in pattern characteristics is screened from the image to be marked.

[0137] In addition, no matter which way is used to determine the target image region, the determined target image region can be at least one, in other words, in order to more accurately determine the region with the strongest uniqueness in the image to be marked, the quantity detection device can screen multiple target image regions from the initial image region. The specific screening of multiple target image regions can be determined based on the second set ranking and the fourth set ranking determined according to actual needs.

[0138] Of course, the first set ranking and the third set ranking can also be determined according to actual needs, so in actual application, the quantity detection device does not necessarily take the maximum similarity as the target similarity, or the minimum distance value as the target distance value. In this way, more suitable target image regions can be screened in the first screening process, so that the region with the highest uniqueness can be more comprehensively covered in the image to be marked, to ensure the accuracy of subsequent determination of the target marking region.

[0139] S106: Divide multiple candidate marking regions of different sizes from the target image region.

[0140] After the target image region is determined, the quantity detection device can divide it into multiple candidate marking regions of different sizes. The purpose of this is to determine multiple image regions where patterns with obvious uniqueness may appear within the neighborhood range of the target image region, so as to accurately determine the region containing the pattern with obvious uniqueness as the target marking region in the subsequent process. The determination of multiple candidate marking regions of different sizes can refer to the processes shown in Figure 4A and Figure 4B .

[0141] Figure 4A and Figure 4B The process diagram for determining multiple candidate marking regions of different sizes provided by the embodiments of the present application.

[0142] Figure 4AThe dashed-lined area in the image to be labeled is the identified target image region. The measurement detection device can use the target image region as a reference to identify multiple candidate labeling regions of different sizes in the image to be labeled. Figure 4B The multiple candidate label regions of different sizes in the image are the areas covered by multiple dashed boxes in the image to be labeled.

[0143] In the embodiments of this application, the measurement detection device can determine multiple candidate marker regions of different sizes in various ways. For example, the measurement detection device can first determine the center point of the target image region, and then scale the target image region with the center point as the center, thereby determining multiple candidate marker regions that have the same center point as the target image region but different sizes.

[0144] For example, a measurement detection device can first determine the center point of the target image region, and then move that center point within a preset range to obtain multiple center point positions. The measurement detection device can then use these center point positions as centers to determine multiple candidate marker regions of different sizes. The size of these candidate marker regions can be determined by adjusting them within a certain range based on the size of the target image region.

[0145] For example, a measurement detection device can slide the target image region around a certain range, and then scale the size of the image region obtained by the sliding to different degrees, thereby obtaining multiple candidate marker regions of different sizes.

[0146] Of course, in practical applications, measurement and detection equipment can also determine multiple candidate marking regions of different sizes through other methods, which will not be listed here.

[0147] S108: Calculate the uniqueness evaluation value of the candidate labeled region in the image to be labeled.

[0148] The uniqueness evaluation value can characterize the degree to which the graphic of the candidate labeling region is unique in the image to be labeled, and it can be calculated in a variety of ways. For example, Fourier transform can be performed on the candidate labeling region and the image to be labeled to obtain their respective frequency domain representations. Then, relevant information of the frequency spectrum can be extracted from the frequency domain representations, and difference analysis (such as mean square error and KL divergence) can be performed on the relevant information between the candidate labeling region and the image to be labeled to obtain the uniqueness evaluation value of the candidate labeling region.

[0149] For example, the quantity detection device can also determine the uniqueness evaluation value of the candidate marker image in the image to be labeled by calculating the matching degree between the candidate marker region and other regions. In the specific process, the quantity detection device can slide the candidate marker region in the image to be labeled for each candidate marker region to calculate the matching degree between the candidate marker region and each sliding region. Then, the calculated matching degrees can be sorted in descending order, and the matching degree that is lower than the sixth preset value and is located before the sixth preset position is determined as the target matching degree of the candidate marker region.

[0150] The sixth setting here, including the ranking and preset values, can be flexibly set based on the scheme design and practical needs. For example, the preset values ​​can be used to eliminate the matching degree calculated with the candidate marker area itself, or to eliminate the matching degree that may bring redundancy to the application accuracy.

[0151] The above method of calculating the matching degree can be implemented in a similar way to calculating the similarity of image features. The higher the matching degree, the more similar the candidate labeling region is to the sliding region. The lower the uniqueness of the candidate labeling region in the image to be labeled (which can be characterized by the uniqueness evaluation value), that is, the matching degree and the uniqueness evaluation value are negatively correlated.

[0152] Therefore, the quantitative detection device can determine the target matching degree as the uniqueness evaluation value of the candidate labeling region in the image to be labeled.

[0153] Other methods for calculating uniqueness evaluation values ​​will not be illustrated here.

[0154] S110: Determine the target labeled region based on the uniqueness evaluation values ​​of multiple candidate labeled regions.

[0155] There are several ways to select a target marker region from multiple candidate marker regions. For example, one can first initially filter out some unique evaluation values ​​(such as filtering out unique evaluation values ​​corresponding to candidate marker regions located in specific areas of the wafer image (such as edge regions, regions scanned by the camera later), and filtering out unique evaluation values ​​outside the acceptable range (determined by engineers based on theory and practical experience, such as calculating based on design drawings, where the unique evaluation value is less than a certain threshold, and the acceptable range is determined by this threshold; another example is setting an acceptable range reasonably to improve processing speed). Then, the remaining unique evaluation values ​​are sorted from largest to smallest, and the candidate marker regions corresponding to unique evaluation values ​​greater than the preset ranking are determined as target marker regions. Another example is to directly sort the unique evaluation values ​​of each candidate marker region in descending order, and the candidate marker regions ranked before the fifth preset ranking are taken as target marker regions.

[0156] For example, when the target matching degree is used as the uniqueness evaluation value of the candidate labeling region in the image to be labeled, the quantity detection device can sort the candidate labeling regions in ascending order of the target matching degree, and take the candidate labeling regions ranked before the seventh set position as the target labeling regions.

[0157] Since regions with high uniqueness will actually have low (or minimal) matching degrees with other sliding regions, the target marking regions determined in ascending order of target matching degree represent the regions with high (or highest) uniqueness in the image to be marked. The selected target marking regions can be one or multiple, the specific number depending on the value of the seventh setting. The specific values ​​of the sixth and seventh settings can be determined according to actual needs.

[0158] As can be seen from the above method, target marker regions with significant distinguishing characteristics in the image to be labeled can be found through a two-stage matching process. First, several candidate marker regions with significant distinguishing characteristics are identified on the overall image to be labeled. Then, target marker regions that are significantly different from other candidate marker regions are identified from these candidate regions. This not only eliminates the need to add additional physical markers to the target device, reducing costs, but also ensures that the identified target marker regions can be significantly distinguished from other image elements in the image to be labeled, thereby guaranteeing accurate alignment and positioning of subsequent images.

[0159] Furthermore, the target marking area determined by the measurement detection device in the above manner is equivalent to the marking point determined in the image to be marked. Based on this, the measurement detection device can perform image alignment operation based on the target marking area determined in the image to be marked.

[0160] Specifically, the measurement device can acquire other images that need to be aligned with the image to be labeled. Then, through image region matching, it determines the image regions in the other images that match the target label region, which serve as the alignment regions for the other images. Afterward, the measurement device can align the image to be labeled with the other images based on the target label region and the alignment regions.

[0161] The image region matching method used during image alignment can be the same as the method used to determine the matching degree, and will not be elaborated here. Once the measurement detection device completes the image alignment, it can perform subsequent operations based on the alignment result.

[0162] For example, when the image to be labeled and other images are wafer images, the quality inspection equipment can perform wafer defect detection based on the aligned image to be labeled and other images. Specifically, when the image to be labeled is a defect-free wafer image, it can be used as a template image. By aligning the template image with other images, image comparison can be performed to detect whether the wafer corresponding to the other images has defects.

[0163] For example, when the image to be labeled is a wafer image, the metrology equipment can further determine the image coordinates of the target labeling region in the image. Then, the metrology equipment can determine whether it is necessary to adjust the position of the wafer corresponding to the image on the equipment by calculating the difference between the image coordinates of the target labeling region and its theoretical coordinates. The theoretical coordinates of the target labeling region can refer to the image coordinates of the target labeling region in the image to be labeled when the wafer is correctly positioned on the equipment.

[0164] Furthermore, the measurement and inspection equipment can control designated equipment to adjust the position of the wafer on the machine based on the determined coordinate differences. Specifically, this can be achieved by adjusting the angle, adjusting the position of the wafer center, etc.

[0165] The specified equipment can include various types, such as a robotic arm for picking up wafers, a wafer stage, etc. Based on this, when the specified equipment is a robotic arm, the metrology equipment can plan the robotic arm's operating path according to the determined coordinate differences, thereby controlling the robotic arm to pick up the wafer and reposition it in the planned location. When the specified equipment is a wafer stage, the metrology equipment can fine-tune the stage according to the determined coordinate differences, thereby adjusting the wafer's position and / or angle.

[0166] In addition, the measurement and detection equipment can use preset tools to achieve the above process in determining the target marking area. These preset tools can refer to tools such as Halcon.

[0167] In the process of determining the candidate marker region, the measurement device can first determine multiple initial candidate marker regions of different sizes, and then determine multiple candidate marker regions of different sizes based on these initial candidate marker regions. The resulting candidate marker regions can be determined based on the initial candidate marker regions through position fine-tuning or angle rotation.

[0168] Specifically, when using the halcon tool to perform image region matching to determine target marked regions, you can enter the following fields in the halcon tool:

[0169] create_shape_model(Template::NumLevels,AngleStart,AngleExtent,AngleStep,Optimization,

[0170] Metric,Contrast,MinContrast : ModelID).

[0171] The purpose of this field is to identify multiple candidate labeling regions of different sizes in the image to be labeled using the Halcon tool. Therefore, Template indicates that candidate labeling regions that meet the requirements should be identified, and the subsequent fields of Template can be understood as condition fields used to determine the candidate labeling regions.

[0172] NumLevels is used to represent the maximum pyramid level. This is actually used to reflect whether the candidate labeling region needs to be determined in the initial candidate labeling region. For example, when NumLevels=1, it means that the candidate labeling region is determined only on the basis of the initial candidate labeling region, and the initial candidate labeling region is not scaled in size.

[0173] AngleStart represents the minimum rotation radius, and AngleExtent represents the rotation radius range. This is mainly used to reflect the rotation range when determining the candidate marker region based on the initial candidate marker region. For example, when AngleStart=-0.1 (-5.7°) and AngleExtent=0.2 (11.5°), it means that the Halcon tool will rotate within the range of -5.7° to 11.5° around the center point of the initial candidate marker region to determine the candidate marker region corresponding to the initial candidate marker region.

[0174] AngleStep is used to represent the step size of the rotation in radians, that is, the step size of each rotation when rotating the initial candidate marker region through its center point. For example, when AngleStep='auto', the halcon tool automatically selects the rotation step size to balance computational efficiency and accuracy.

[0175] Optimization indicates the optimization method used. Optimization primarily reflects how much image detail needs to be retained or preserved in the final candidate region. For example, when Optimization='auto', the Halcon tool will automatically select the optimization method. Different optimization methods will result in variations in the number of image feature points contained in the final candidate region, and whether low-contrast edges are filtered out, among other image details.

[0176] Metrics are used to represent the grayscale characteristics considered when determining candidate label regions. They reflect whether the final determined candidate label regions are consistent with the initial candidate label regions in terms of grayscale characteristics. Generally, it is necessary to ensure that the final searched candidate label regions are consistent with the initial candidate label regions in terms of grayscale characteristics.

[0177] Contrast is used to represent the contrast threshold set for the pyramid level. This is mainly used to constrain the correlation between the finally searched candidate marker regions and the initial candidate marker regions. For example, when Contrast=[14,29,5], 14 means that the finally explored candidate marker regions need to contain edge points or contour points with a contrast greater than or equal to 14 in the initial candidate marker regions; 29 means that the finally explored candidate marker regions need to contain edge points or contour points with a contrast less than or equal to 29 in the initial candidate marker regions; and 5 means that the finally explored candidate marker regions need to contain at least 5 unit pixel regions from the initial candidate marker regions.

[0178] MinContrast is used to represent minimum contrast, limiting the minimum contrast of edge or contour points of interest when searching for candidate marker regions. For details, please refer to the example used to explain Contrast above.

[0179] ModelID is the identifier information used to represent the initial candidate label region that has been determined.

[0180] The measurement device can use the above fields and the Halcon tool to identify multiple candidate marker regions of different sizes. Then, the measurement device can further use the Halcon tool to determine the target marker region from these candidate marker regions.

[0181] For example, you can enter the following fields in the Halcon tool:

[0182] find_shape_model(Image::ModelID,AngleStart,AngleExtent,MinScore,NumMatches,MaxOverlap,

[0183] SubPixel, NumLevels, Greediness: Row, Column, Angle, Score).

[0184] ModelID is the identifier information corresponding to the candidate marker region determined by the above method.

[0185] AngleStart and AngleExtent indicate that when matching candidate marker regions with sliding regions, rotation can be performed within a certain angle range, and the matching process is completed through the rotated region.

[0186] The `MinScore` parameter essentially specifies which sliding regions' calculated matching scores for the candidate marker region should be retained. For example, when `MinScore=0`, it means that the matching scores calculated for the candidate marker region with each sliding region should be retained.

[0187] NumMatches actually specifies the number of regions that need to be identified from each sliding region that match the candidate marker region. For example, when NumMatches=2, it means that the two sliding regions that best match the candidate marker region need to be identified from each sliding region.

[0188] MaxOverlap specifies how to filter regions from each sliding region to match the candidate marker region based on the degree of region overlap. For example, when MaxOverlap=0.1, it means that if two sliding regions have a high degree of matching with the candidate marker region, but the overlap between the two sliding regions exceeds 10%, then one of the sliding regions can be selected and returned.

[0189] SubPixel represents the precision of the most matching region returned when filtering from various sliding regions to find the region that best matches the candidate marker region (including the candidate marker region itself). For example, when SubPixel='none', it means that only the integer coordinates of the most matching region will be returned in this case, without sub-pixel precision.

[0190] NumLevels is used to represent the strategy employed during the matching process. In other words, the number of image pyramid levels (i.e., NumLevel) actually specifies how many pyramid images need to be controlled for the search during the matching process.

[0191] The matching process based on NumLevel can be roughly viewed as first finding several sliding regions in the image to be labeled that may match the candidate label region. These sliding regions are often larger in size than the candidate label region. Then, the region that best matches the candidate label region can be found within these sliding regions. The finally determined region is also comparable in size to the candidate label region.

[0192] Therefore, the value of NumLevel reflects the number of pyramid image layers used in the matching process. For example, when NumLevel=3, it indicates that several sliding regions that may match the candidate label region are first matched, then several regions that are even better matches the candidate label region are searched among these sliding regions, and finally, the region that best matches the candidate label region is searched again from these even better matches. The entire process involves three searches, hence NumLevel=3.

[0193] When NumLevel=0, the halcon tool will automatically select the appropriate number of pyramid images for the matching process.

[0194] Greediness indicates the greedy strategy employed during the matching process. The specific greedy strategy used depends on the value of Greediness. For example, when Greediness is close to 1 (e.g., 0.9), it means that a certain degree of matching error is acceptable in exchange for faster matching speed. Conversely, when Greediness is close to 0, it indicates that the algorithm will be more conservative, trying more possibilities to ensure that a sliding region matching the candidate marker region is found.

[0195] Row and Column represent the image coordinates of the final matched sliding region, while Angle refers to the matching angle used to obtain the best matching sliding region. Score represents the degree of matching.

[0196] As can be seen from the above process, the quantity detection device can use the Halcon tool to identify the region with a high degree of uniqueness in the image to be labeled as the target label region from a large number of candidate label regions, while taking into account matching efficiency.

[0197] Furthermore, since the measurement detection device can identify more than one target image region in step S104, it is also necessary to subsequently identify multiple candidate marker regions of different sizes based on each target image region. This can further improve the search range and ensure that the final identified target marker region has the highest degree of uniqueness in the entire image to be labeled, and can be significantly distinguished from the graphics in other regions.

[0198] The above describes a method for determining a marked region in one or more embodiments of this application. Based on the same idea, this application also provides a corresponding device for determining a marked region, such as... Figure 5 As shown.

[0199] Figure 5 A schematic diagram of a marking area determination device provided in this application embodiment includes:

[0200] The acquisition module 500 is used to acquire the image to be labeled and identify multiple initial image regions from the image to be labeled.

[0201] The first calculation module 502 is used to calculate the similarity between multiple initial image regions;

[0202] The filtering module 504 is used to filter target image regions from multiple initial image regions based on similarity.

[0203] The segmentation module 506 is used to segment multiple candidate marker regions of different sizes from the target image region;

[0204] The second calculation module 508 is used to calculate the uniqueness evaluation value of the candidate labeling region in the image to be labeled. The uniqueness evaluation value characterizes the degree of uniqueness of the graphic of the candidate labeling region in the image to be labeled.

[0205] The determination module 510 is used to determine the target marked region based on the uniqueness evaluation value of multiple candidate marked regions.

[0206] Optionally, the first calculation module 502 is specifically used to: divide each initial image region into multiple segmented regions, and determine the extraction features corresponding to the initial image region based on the image features of the multiple segmented regions; and calculate the similarity between the multiple initial image regions based on the extraction features corresponding to the initial image regions.

[0207] Optionally, the first calculation module 502 is specifically used to divide the initial image region into multiple segmentation regions in the horizontal direction and / or in the initial image region in the vertical direction.

[0208] Optionally, the first calculation module 502 is specifically used to: for each segmented region, determine the average gray value corresponding to the segmented region based on the gray values ​​of each pixel contained in the segmented region, and determine the standard deviation of the gray value corresponding to the segmented region based on the average gray value corresponding to the segmented region and the gray values ​​of each pixel contained in the segmented region; and determine the average gray value and / or the standard deviation of the gray value corresponding to the segmented region as the image features of the segmented region, so as to determine the extraction features corresponding to the initial image region based on the image features of the segmented region.

[0209] Optionally, the first calculation module 502 is specifically used to: determine at least one set of symmetrical segmentation regions located on both sides of the center line from each segmentation region of the initial image region according to the center line of the initial image region; determine the symmetry features of the initial image region according to the image features of the at least one set of symmetrical segmentation regions; and determine the extraction features corresponding to the initial image region based on the symmetry features, the average gray value, and the standard deviation of the gray value.

[0210] Optionally, the first calculation module 502 is specifically used to: acquire multiple feature fusion strategies for calculating symmetrical features; select at least one set of symmetrical segmentation regions corresponding to each feature fusion strategy from multiple sets of symmetrical segmentation regions; and fuse the image features of at least one set of symmetrical segmentation regions corresponding to each feature fusion strategy to obtain multiple symmetrical features.

[0211] Optionally, the segmentation module 506 is specifically used to slide and capture images of the image to be marked through a preset sliding window at a preset step and according to a preset path to obtain multiple initial image regions, wherein the preset step is smaller than the size of the preset sliding window.

[0212] Optionally, the filtering module 504 is specifically used to: sort the similarity between each initial image region and multiple other initial image regions from largest to smallest, and take the similarity before the first set ranking as the target similarity corresponding to the initial image region; sort the target similarity corresponding to each initial image region from smallest to largest, and determine the initial image region before the second set ranking as the target image region.

[0213] Optionally, the first calculation module 502 is specifically used to calculate the distance value between the extracted features corresponding to each initial image region, and determine the distance value as the similarity between multiple initial image regions;

[0214] The filtering module 504 is specifically used to sort the distance values ​​between each initial image region and multiple other initial image regions from smallest to largest, and take the distance values ​​before the third set position as the target distance values ​​corresponding to the initial image region; sort the target distance values ​​corresponding to each initial image region from largest to smallest, and determine the initial image regions before the fourth set position as the target image regions.

[0215] Optionally, the determining module 510 is specifically used to sort multiple candidate marked regions in descending order of uniqueness evaluation value, and to take the candidate marked region ranked before the fifth set position as the target marked region.

[0216] Optionally, the second calculation module 508 is specifically used to: for each candidate marking region, slide the candidate marking region in the image to be marked to calculate the matching degree between the candidate marking region and each sliding region; sort the multiple matching degrees from largest to smallest, determine the matching degree that is located before the sixth set position and is less than a preset value as the target matching degree of the candidate marking region, and determine the target matching degree as the uniqueness evaluation value of the candidate marking region in the image to be marked;

[0217] The determination module 510 is specifically used to sort the candidate marked regions in ascending order of their target matching degree, and to take the candidate marked region ranked before the seventh set position as the target marked region.

[0218] Optionally, the device also includes:

[0219] Alignment module 512 acquires other images, which are the images to be aligned with the image to be labeled; it determines the image region that matches the target labeling region in the other images, which is used as the alignment region corresponding to the other images; and it aligns the image to be labeled with the other images according to the target labeling region and the alignment region.

[0220] Optionally, the image to be labeled and other images are wafer images;

[0221] The equipment also includes:

[0222] The detection module 514 performs wafer defect detection based on the aligned image to be labeled and other images.

[0223] This application also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 A method for determining marked regions is provided.

[0224] This application also provides Figure 6 The diagram shows a schematic structural representation of the electronic device. Figure 6 At the hardware level, the device includes a processor, internal bus, network interface, memory, and non-volatile storage, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile storage into memory and then executes it to achieve the above. Figure 1 A method for determining a marked region. Of course, in addition to software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0225] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0226] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0227] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0228] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0229] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0230] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0231] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0232] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes ​ The steps of the function specified in one or more boxes.

[0233] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0234] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0235] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0236] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0237] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0238] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0239] The above are merely embodiments of this specification and are not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for determining a marked region, characterized in that, include: Acquire an image to be labeled, and identify multiple initial image regions from the image to be labeled; Calculate the similarity between the multiple initial image regions; Based on the similarity, a target image region is selected from the plurality of initial image regions. For each initial image region, the similarity between the initial image region and a plurality of other initial image regions is sorted from largest to smallest. The similarity ranked before the first set ranking is taken as the target similarity corresponding to the initial image region. The target similarity corresponding to each initial image region is sorted from smallest to largest. The initial image region ranked before the second set ranking is determined as the target image region. Segment the target image region into multiple candidate marker regions of different sizes; Calculate the uniqueness evaluation value of the candidate marker region in the image to be labeled. The uniqueness evaluation value characterizes the degree to which the graphic of the candidate marker region is unique in the image to be labeled. For any candidate marker region, the uniqueness evaluation value corresponding to the candidate marker region is negatively correlated with the matching degree between the candidate marker region and other regions. The target labeled region is determined based on the uniqueness evaluation values ​​of the multiple candidate labeled regions.

2. The method as described in claim 1, characterized in that, Calculating the similarity between the multiple initial image regions specifically includes: For each initial image region, the initial image region is divided into multiple segmented regions, and the extraction features corresponding to the initial image region are determined based on the image features of the multiple segmented regions. The similarity between the multiple initial image regions is calculated based on the extracted features corresponding to the initial image regions.

3. The method as described in claim 2, characterized in that, The initial image region is divided into multiple segmented regions, specifically including: The initial image region is divided into multiple segmented regions in the horizontal direction and / or in the vertical direction.

4. The method as described in claim 2 or 3, characterized in that, Based on the image features of the multiple segmented regions, the extraction features corresponding to the initial image region are determined, specifically including: For each segmented region, the average gray value of the segmented region is determined based on the gray values ​​of each pixel contained in the segmented region, and the standard deviation of the gray value of the segmented region is determined based on the average gray value of the segmented region and the gray values ​​of each pixel contained in the segmented region. The average gray value and / or the standard deviation of the gray value corresponding to the segmented region are determined as the image features of the segmented region, so as to determine the extraction features corresponding to the initial image region based on the image features of the segmented region.

5. The method as described in claim 4, characterized in that, Based on the image features of the segmented region, the extraction features corresponding to the initial image region are determined, specifically including: Based on the center line of the initial image region, at least one set of symmetrical segmented regions located on both sides of the center line are determined from each segmented region of the initial image region; Based on the image features of the at least one set of symmetrical segmented regions, the symmetry features of the initial image region are determined, and the extraction features corresponding to the initial image region are determined based on the symmetry features, the average gray value, and the standard deviation of the gray value.

6. The method as described in claim 5, characterized in that, Based on the image features of the at least one set of symmetrical segmented regions, the symmetry features of the initial image region are determined, specifically including: Obtain multiple feature fusion strategies for computing symmetric features; From multiple sets of symmetrical segmentation regions, select at least one set of symmetrical segmentation regions corresponding to each feature fusion strategy; Based on the feature fusion strategy, the image features of at least one set of symmetrical segmented regions corresponding to each feature are fused to obtain multiple symmetrical features.

7. The method as described in claim 1, characterized in that, Identifying multiple initial image regions from the image to be labeled specifically includes: The image to be labeled is captured by sliding through a preset sliding window with a preset step size and a preset path to obtain multiple initial image regions. The preset step size is smaller than the size of the preset sliding window.

8. The method as described in claim 2, characterized in that, Based on the extracted features corresponding to the initial image regions, the similarity between the multiple initial image regions is calculated, including: Calculate the distance between the extracted features corresponding to each initial image region, and determine the distance value as the similarity between the multiple initial image regions; Based on the similarity, a target image region is selected from the plurality of initial image regions, specifically including: For each initial image region, sort the distance values ​​between the initial image region and multiple other initial image regions from smallest to largest, and take the distance value before the third set position as the target distance value corresponding to the initial image region. Sort the target distance values ​​corresponding to each initial image region from largest to smallest, and determine the initial image region that is located before the fourth set position as the target image region.

9. The method as described in claim 1, characterized in that, Based on the uniqueness evaluation values ​​of the multiple candidate marked regions, the target marked region is determined, specifically including: The candidate marked regions are sorted in descending order of their uniqueness evaluation values, and the candidate marked region ranked before the fifth set position is taken as the target marked region.

10. The method as described in claim 1, characterized in that, Calculating the uniqueness evaluation value of the candidate labeled region in the image to be labeled specifically includes: For each candidate labeling region, the candidate labeling region is slid across the image to be labeled to calculate the matching degree between the candidate labeling region and each slid region; The matching scores are sorted from largest to smallest. The matching scores that are located before the sixth set position and are less than a preset value are determined as the target matching scores of the candidate marking regions. The target matching scores are determined as the uniqueness evaluation value of the candidate marking regions in the image to be marked. The target labeled region is determined based on the uniqueness evaluation values ​​of multiple candidate labeled regions, specifically including: The candidate marked regions are sorted in ascending order of their target matching degree, and the candidate marked regions ranked before the seventh set position are taken as the target marked regions.

11. The method as described in claim 1, characterized in that, The method further includes: Acquire other images, which are images that need to be aligned with the image to be labeled; In the other images, an image region that matches the target marker region is determined as the alignment region corresponding to the other images; The image to be labeled is aligned with the other images based on the target marking region and the alignment region.

12. The method as described in claim 11, characterized in that, The image to be labeled and the other images are wafer images; The method further includes: Wafer defect detection is performed based on the aligned image to be labeled and the other images.

13. A marking area determination device, characterized in that, include: The acquisition module is used to acquire the image to be labeled and identify multiple initial image regions from the image to be labeled; The first calculation module is used to calculate the similarity between the multiple initial image regions; The filtering module is used to filter target image regions from the plurality of initial image regions based on the similarity. For each initial image region, the similarity between the initial image region and a plurality of other initial image regions is sorted from largest to smallest. The similarity ranked before the first set ranking is taken as the target similarity corresponding to the initial image region. The target similarity corresponding to each initial image region is sorted from smallest to largest. The initial image region ranked before the second set ranking is determined as the target image region. A segmentation module is used to segment multiple candidate marker regions of different sizes from the target image region; The second calculation module is used to calculate the uniqueness evaluation value of the candidate marking region in the image to be marked. The uniqueness evaluation value represents the degree of uniqueness of the graphic of the candidate marking region in the image to be marked. For any candidate marking region, the uniqueness evaluation value corresponding to the candidate marking region is negatively correlated with the matching degree between the candidate marking region and other regions. The determination module is used to determine the target marking region based on the uniqueness evaluation value of the multiple candidate marking regions.

14. A computer-readable storage medium, characterized in that, The storage medium has a computer program / instruction, which, when executed by a processor, implements the method described in any one of claims 1 to 12.

15. An electronic device, characterized in that, The invention includes a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that the processor executes the program to implement the method described in any one of claims 1 to 12.

Images