Marked area determination method and device, storage medium and equipment

Abstract

In the marked area determination method and device, the storage medium and the equipment provided by the invention, the to-be-marked image is firstly acquired, the plurality of initial image areas are identified from the to-be-marked image, then the similarity among the plurality of initial image areas is calculated, and the target image area is screened from the plurality of initial image areas based on the similarity. And then, segmenting a plurality of candidate mark regions with different sizes from the target image region to calculate uniqueness evaluation values of the candidate mark regions in the to-be-marked image, so as to determine the target mark region based on the uniqueness evaluation values of the plurality of candidate mark regions. According to the method, the target marking area with obvious distinguishing features can be found from the to-be-marked image through matching of the two stages, so that the cost is reduced, and the accuracy of determining the marking area is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a method, apparatus, storage medium, and device for determining a marked area. Background Art

[0002] In the fields of semiconductor manufacturing, printed circuit board manufacturing, and flat panel display manufacturing, the use of marking points (i.e., marks) to align and position images is involved to detect whether there are defects in panels, PCB boards, wafers, etc. in the manufacturing process, thereby improving the yield rate.

[0003] In practical applications, there are two main ways to set markers: one is to set physical markers on the object, and the other is to add virtual markers to the captured images of panels, PCBs, wafers, etc.

[0004] Although both methods can achieve subsequent image alignment, positioning and other operations, they both have certain problems.

[0005] For the first approach, adding physical markers not only increases costs but also risks damaging the target device. For the second approach, virtual markers are often added to the image based on human experience. This means that these added virtual markers may not have significant distinguishing features compared to other elements in the image, making it impossible to achieve accurate image alignment and positioning in subsequent processes.

[0006] Based on this, the present application provides a method, apparatus, storage medium and device for determining a marked area. Summary of the Invention

[0007] The present application provides a method, apparatus, storage medium and device for determining a marked area to partially solve the above-mentioned problems existing in the prior art.

[0008] This application adopts the following technical solution: This application provides a method for determining a marked area, the method comprising: Acquire an image to be labeled, and identify a plurality of initial image regions from the image to be labeled; Calculate the similarity between multiple initial image regions; Based on the similarity, the target image region is selected from multiple initial image regions; Segmenting multiple candidate marker regions of different sizes from the target image region; Calculating a uniqueness evaluation value of the candidate marking region in the image to be marked, where the uniqueness evaluation value represents the uniqueness of the pattern of the candidate marking region in the image to be marked; Based on the uniqueness evaluation values ​​of the plurality of candidate label regions, a target label region is determined.

[0009] Optionally, calculating similarities between multiple initial image regions specifically includes: For each initial image region, the initial image region is divided to obtain a plurality of segmented regions, and an extraction feature corresponding to the initial image region is determined based on image features of the plurality of segmented regions; The similarities between the multiple initial image regions are calculated based on the extracted features corresponding to the initial image regions.

[0010] Optionally, the initial image area is divided to obtain multiple segmented areas, specifically including: The initial image region is divided into a plurality of segmented regions in a horizontal direction, and / or the initial image region is divided into a plurality of segmented regions in a vertical direction.

[0011] Optionally, determining the extraction features corresponding to the initial image region based on the image features of the plurality of segmented regions specifically includes: For each segmented area, determine the average grayscale value corresponding to the segmented area based on the grayscale value of each pixel contained in the segmented area, and determine the standard deviation of the grayscale value corresponding to the segmented area based on the average grayscale value corresponding to the segmented area and the grayscale value of each pixel contained in the segmented area; The average grayscale value corresponding to the segmented area and / or the grayscale value standard deviation corresponding to the segmented area are determined as the image features of the segmented area, so as to determine the extraction features corresponding to the initial image area according to the image features of the segmented area.

[0012] Optionally, determining the extraction features corresponding to the initial image area based on the image features of the segmented area specifically includes: According to the center line of the initial image area, determining at least one group of symmetrical segmented areas located on both sides of the center line from the segmented areas of the initial image area; The symmetric features of the initial image region are determined according to the image features of at least one group of symmetrical segmented regions, and the extraction features corresponding to the initial image region are determined based on the symmetric features, the average grayscale value and the grayscale value standard deviation.

[0013] Optionally, determining the symmetry features of the initial image region based on image features of at least one set of symmetric segmented regions specifically includes: Obtain multiple feature fusion strategies for calculating symmetric features; From the multiple groups of symmetrical segmentation regions, select at least one group of symmetrical segmentation regions corresponding to each feature fusion strategy; Based on each feature fusion strategy, the image features of at least one group of symmetrical segmented areas corresponding thereto are fused to obtain a plurality of symmetrical features.

[0014] Optionally, identifying multiple initial image regions from the image to be labeled specifically includes: The image to be marked is sampled by sliding through a preset sliding window according to a preset path with a preset step, to obtain a plurality of initial image regions, wherein the preset step is smaller than the size of the preset sliding window.

[0015] Optionally, based on similarity, the target image region is screened from the multiple initial image regions, specifically including: For each initial image region, sort the similarities between the initial image region and multiple other initial image regions from largest to smallest, and use the similarities before the first set ranking as the target similarity corresponding to the initial image region; The target similarities corresponding to the initial image regions are sorted from small to large, and the initial image regions located before the second set ranking are determined as the target image regions.

[0016] Optionally, calculating similarities between the multiple initial image regions based on the extracted features corresponding to the initial image regions includes: Calculating the distance value between the extracted features corresponding to each initial image region, and determining the distance value as the similarity between the multiple initial image regions; Based on the similarity, the target image region is selected from multiple initial image regions, including: For each initial image region, sort the distance values ​​between the initial image region and multiple other initial image regions from small to large, and use the distance value before the third set ranking as the target distance value corresponding to the initial image region; The target distance values ​​corresponding to the initial image regions are sorted from largest to smallest, and the initial image regions located before the fourth set ranking are determined as the target image regions.

[0017] Optionally, determining a target marking region based on uniqueness evaluation values ​​of multiple candidate marking regions specifically includes: The plurality of candidate marking regions are sorted in descending order of uniqueness evaluation values, and the candidate marking region ranked before the fifth set ranking is used as the target marking region.

[0018] Optionally, calculating a uniqueness evaluation value of the candidate marking region in the image to be marked specifically includes: For each candidate marking area, slide the candidate marking area in the image to be marked to calculate the matching degree between the candidate marking area and each sliding area; Sort the multiple matching degrees from largest to smallest, determine the matching degree that is before the sixth set ranking and smaller than a preset value as the target matching degree of the candidate marking region, and determine the target matching degree as the uniqueness evaluation value of the candidate marking region in the image to be marked; Based on the uniqueness evaluation values ​​of multiple candidate marking regions, the target marking region is determined, specifically including: The candidate marking regions are sorted in ascending order according to the target matching degree, and the candidate marking regions ranked before the seventh set ranking are used as target marking regions.

[0019] Optionally, the method further comprises: Obtain other images, where the other images and the image to be marked are images that need to be aligned; Determine an image region that matches the target marked region in the other image as the alignment region corresponding to the other image; Align the image to be marked with other images based on the target marking area and the alignment area.

[0020] Optionally, the image to be marked and the other images are wafer images; The method also includes: Wafer defect detection is performed based on the aligned images to be marked and other images.

[0021] The present application provides a marking area determination device, comprising: an acquisition module, configured to acquire an image to be labeled and identify a plurality of initial image regions from the image to be labeled; A first calculation module is used to calculate the similarity between multiple initial image regions; A screening module, configured to screen a target image region from a plurality of initial image regions based on similarity; A segmentation module is used to segment a plurality of candidate marker regions of different sizes from the target image region; A second calculation module is used to calculate a uniqueness evaluation value of the candidate marking region in the image to be marked, where the uniqueness evaluation value represents the uniqueness of the pattern of the candidate marking region in the image to be marked; The determination module is used to determine a target marking area based on uniqueness evaluation values ​​of multiple candidate marking areas.

[0022] The present application provides a computer-readable storage medium having a computer program / instruction. When the computer program / instruction is executed by a processor, the above-mentioned marking area determination method is implemented.

[0023] The present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, the above-mentioned marking area determination method is implemented.

[0024] The present application provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned marking area determination method when executed by a processor.

[0025] At least one of the above-mentioned technical solutions employed in this application can achieve the following beneficial effects: first, an image to be labeled is acquired, and multiple initial image regions are identified from the image to be labeled. Then, similarities between the multiple initial image regions are calculated, and based on the similarities, a target image region is selected from the multiple initial image regions. Subsequently, multiple candidate labeling regions of different sizes are segmented from the target image region to calculate uniqueness evaluation values ​​for the candidate labeling regions in the image to be labeled. Finally, a target labeling region is determined based on the uniqueness evaluation values ​​of the multiple candidate labeling regions.

[0026] As can be seen from the above method, the present application can first determine several target image regions with significant distinguishing characteristics from the overall perspective of the image to be marked by calculating and comparing the similarities between image regions. The target image regions are significant compared to most image regions. Then, the target image region is segmented to obtain multiple candidate marking regions with different sizes, and the uniqueness evaluation value of the candidate marking regions is calculated to further determine the most unique target marking region from the global perspective of the image to be marked from multiple candidate marking regions of different sizes and originating from different target image regions. This not only does not require the addition of additional physical marking points, thus reducing costs, but also ensures that the determined target marking region can be significantly distinguished from other image elements in the image to be marked, thereby ensuring the accurate alignment and positioning of subsequent images. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of a method for determining a marked area provided in an embodiment of the present application; Figure 2A and Figure 2B Schematic diagram of two image acquisition methods provided in the embodiments of this application; Figure 3A and Figure 3B A schematic diagram of dividing the initial image area in the horizontal and vertical directions provided in an embodiment of the present application; Figure 4A and Figure 4B A schematic diagram of a process for determining multiple candidate marking regions of different sizes provided in an embodiment of the present application; Figure 5 A schematic diagram of a marking area determination device provided in an embodiment of the present application; Figure 6 A schematic diagram of the electronic device structure corresponding to a marking point determination method provided in this application. DETAILED DESCRIPTION

[0028] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0029] In order to avoid the problems caused by the traditional method of determining marking points, in an embodiment of the present application, a target marking area is determined from the image to be marked through a two-stage image matching process. The target marking area can be regarded as a marking point determined from the image to be marked. This not only significantly reduces the cost problem caused by setting physical marking points, but also ensures the accuracy of the determined target marking area.

[0030] The method for determining a marker region provided in the embodiments of the present application is similar to the prior art method for determining virtual marker points. However, unlike the prior art method of manually determining virtual marker points, the method provided in the embodiments of the present application primarily analyzes the image features of multiple sub-image regions within an image to determine a target marker region with significantly distinct image features.

[0031] Furthermore, the marking area determination method provided in the embodiments of the present application can be applied in a variety of fields. For example, the marking area determination method provided in the embodiments of the present application can be applied to measurement equipment in the semiconductor field. Measurement equipment includes various types of measurement functions involved in the entire semiconductor manufacturing process, such as thin film measurement, line width measurement, overlay accuracy measurement, bump detection, and defect detection.

[0032] The measurement and inspection equipment includes components such as a control system, a stage, and a manipulator. The control system includes terminals (such as industrial computers), servers, and motion controllers. The control system's components communicate with each other based on actual needs. Marking areas are generated, and solutions such as image alignment and wafer positioning based on these areas are implemented on a computer or server. The relevant information is displayed on the visual interface of the measurement and inspection equipment, and the motion controller controls the movement of the manipulator and / or stage. The relevant information is then reported to the factory's manufacturing execution system.

[0033] Of course, the marking area determination method provided in the embodiment of the present application can also be implemented on electronic equipment (terminals, servers, etc.) before the quantity detection equipment is put into production, and the obtained marking area is saved in the control system of the quantity detection equipment for subsequent use.

[0034] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0035] Figure 1 A flowchart of a method for determining a marked area provided in an embodiment of the present application includes the following steps: S100: Acquire an image to be labeled, and identify multiple initial image regions from the image to be labeled.

[0036] In an embodiment of the present application, an image of an area to be marked, i.e., an image to be marked, can be obtained. The image to be marked can be obtained by capturing an image of a specified target object. The specific form of the target object depends on the actual application scenario. For example, in the field of semiconductor manufacturing, the target object can refer to a wafer, die, chip, etc. to be inspected; for another example, in the field of printed circuit board manufacturing, the target object can refer to a printed circuit board to be inspected. Other specific forms of the target object will not be given one by one here.

[0037] The marking area determination method provided in the embodiments of the present application can be executed by a variety of entities, including terminal devices such as desktop computers and laptops, clients installed in terminal devices, servers, and quantity detection equipment specifically used for wafer / chip quantity detection. For ease of explanation, the marking area determination method provided in the embodiments of the present application will be described in detail below using the quantity detection equipment as the execution entity.

[0038] After acquiring the above-mentioned image to be marked, it is necessary to identify and divide the image to be marked to determine multiple initial image regions therefrom. The initial image region may include a portion of the image to be marked, wherein the embodiment of the present application may divide the multiple initial image regions in a variety of ways. For example, the quantity detection device may identify the various units contained in the image to be marked, and determine the position of each unit in the image to be marked, and then determine each initial image region of a preset size with the center point of the position of each unit (such as spatially adjacent superpixels with similar grayscale / color) in the image to be marked as the center.

[0039] For another example, the quantity detection device may preset a step, and slide the image to be marked through a preset sliding window according to a preset path to obtain multiple initial image areas.

[0040] There are many ways for the detection device to slide a preset sliding window on the image to be marked to identify multiple initial image areas, such as Figure 2A and Figure 2B shown.

[0041] Figure 2A and Figure 2B Schematic diagram of two image acquisition methods provided in the embodiments of this application.

[0042] exist Figure 2A In the figure, the dotted box on the left is used to represent the sliding window for dividing and identifying the initial image area. Then, Figure 2A In the image to be marked on the right, multiple initial image regions are determined by the sliding window. Figure 2A The initial image regions determined in do not overlap with each other. Therefore, Figure 2A The illustrated method can be understood as performing image sampling in the image to be marked without repetition, so that the initial image regions obtained by the previous and subsequent sampling have no overlapping parts.

[0043] And in Figure 2B In the figure, the dotted box on the left is still used to represent the sliding window for dividing and identifying the initial image area. Figure 2B In the image to be marked on the right, the quantity detection device can slide the sliding window according to a preset step size smaller than the sliding window to divide the initial image areas in the image to be marked. There is an overlapping part between the two initial image areas divided by sliding the sliding window twice.

[0044] In the above two methods, the sliding path of the sliding window can be various. For example, you can start from the upper left corner of the image to be marked, slide from left to right to divide a row of initial image areas, then move the sliding window downward by a set step size, and then slide from left to right until all initial image areas are divided. For another example, you can start from the upper left corner of the image to be marked, slide from top to bottom to divide a column of initial image areas, then move the sliding window to the right by a set step size, and then slide from top to bottom until all initial image areas are divided.

[0045] Of course, except Figure 2A and Figure 2B In addition to the above-mentioned manner, there may be other division manners, which will not be described in detail here. Similarly, in actual applications, other sliding paths may be included in addition to the above-mentioned sliding paths, which will not be described one by one here.

[0046] It should be emphasized that in order to ensure that the target marking area finally determined has significant distinguishability in the entire image to be marked, it is necessary to ensure that the initial image areas divided out cover all image areas in the image to be marked as a whole, and the preset step size needs to be smaller than the preset sliding window size.

[0047] S102: Calculate similarities between multiple initial image regions.

[0048] In embodiments of the present application, the quantity detection device can calculate the similarity between multiple initial image regions in various ways. Similarity can represent the degree of similarity / correlation between image regions. For example, the quantity detection device can determine the grayscale image corresponding to each initial image region (if the image to be labeled is already a grayscale image, this can be used directly). Then, for any two initial image regions, a histogram is used to calculate the joint distribution of the two initial image regions. This joint distribution reflects the frequency of combinations of grayscale values ​​of pixels at the same image location in the two initial image regions. If certain grayscale combinations occur more frequently in the two initial image regions, it indicates a strong correlation between the two initial image regions. Subsequently, based on the joint distribution between the initial image regions, the mutual information value between the initial image regions is calculated, and the similarity between the initial image regions is determined based on the calculated mutual information value. For two initial image regions, the larger the mutual information value between the two initial image regions, the greater the amount of shared information between the two initial image regions, and thus the higher the similarity between the two initial image regions.

[0049] For another example, after dividing each initial image area, the quantity detection device can determine the image features corresponding to the initial image area for each initial image area. Afterwards, the similarity between each initial image area is calculated based on the distance between the corresponding image features of each initial image area. The image features can be specifically determined based on the pixel values ​​of the pixels contained in the image area. Then, the quantity detection device can adopt a variety of methods in the process of determining the image features. For example, the image features can be determined by calculating the average value of the pixel values ​​of the pixels in the initial image area; for another example, the image features can be determined by calculating the gradient of the pixel values ​​in the initial image area. Other methods will not be explained here one by one.

[0050] When determining the image features corresponding to the initial image area, the quantity detection device may further divide the initial image area, thereby determining the image features of the initial image area based on the divided segmented areas.

[0051] Specifically, for each initial image area, the quantity detection device can divide the initial image area into multiple segmented areas. Thereafter, the image features corresponding to each segmented area can be determined based on the pixel values ​​of the pixels in each segmented area. Then, based on the image features corresponding to each segmented area, the extracted features corresponding to the initial image area can be determined. The extracted features may include representative information of the image area in several dimensions.

[0052] The segmented area may include a portion of the initial image area. When segmenting the initial image area, the amount detection device may segment the initial image area in a horizontal direction or in a vertical direction, such as Figure 3A and Figure 3B shown.

[0053] Figure 3A and Figure 3B A schematic diagram of dividing the initial image area in the horizontal and vertical directions provided in an embodiment of the present application.

[0054] Figure 3A The figure shows eight segmented areas obtained by the detection device segmenting the initial image area in the horizontal direction. Figure 3B Shown are eight segmented areas obtained by segmenting the initial image area by the quantity detection device in the vertical direction.

[0055] In the embodiment of the present application, the amount detection device can segment the initial image area only in the horizontal direction, or only in the vertical direction, or segment the initial image area in the horizontal direction and also in the vertical direction. Figure 3A and Figure 3B In the example shown, when the detection device segments the same initial image area in the horizontal and vertical directions, the segmented Figure 3A and Figure 3B The 16 segmented areas are shown.

[0056] After segmenting each segmented area, the quantity detection device can determine the average grayscale value of each segmented area based on the grayscale value of each pixel contained in the segmented area, and determine the grayscale value standard deviation corresponding to the segmented area based on the average grayscale value corresponding to the segmented area and the grayscale value of each pixel contained in the segmented area. The determined average grayscale value and / or grayscale value standard deviation can then be determined as the image feature corresponding to the segmented area.

[0057] The image features of the segmented area can reflect multiple specific local information of the initial image area. In order to extract the information of the initial image area more comprehensively, this application analyzes the symmetry of multiple segmented areas in the initial image area and integrates the features obtained from the symmetry analysis into the extracted features of the initial image area.

[0058] Specifically, the quantity detection device can determine, for each segmented area, at least one group of symmetrical segmented areas located on both sides of the center line from the segmented areas within the initial image area according to the center line of the initial image area in which the segmented area is located, and determine the symmetrical features of the initial image area based on the image features of these symmetrical segmented areas.

[0059] Symmetry features can characterize the degree of symmetry of an image region relative to the centerline. In the aforementioned symmetry feature determination process, since different symmetric regions are located at different distances from the centerline, different feature fusion strategies can be employed for combinations of segmented regions with different symmetries to better determine whether the image contained within the initial image region is symmetric. These feature fusion strategies can be determined based on human experience.

[0060] Specifically, the quantity detection device may obtain a plurality of feature fusion strategies for calculating symmetry features. These feature fusion strategies may be predetermined and stored. When it is necessary to determine the symmetry features, these feature fusion strategies are read out from the storage space.

[0061] Different feature fusion strategies correspond to different groups of symmetrical segmentation regions. For any feature fusion strategy, it may correspond not only to one group of symmetrical segmentation regions but also to multiple groups of symmetrical segmentation regions.

[0062] Therefore, the quantity detection device can select at least one group of symmetrical segmentation areas corresponding to each feature fusion strategy from multiple groups of symmetrical segmentation areas, and then fuse the image features of at least one group of symmetrical segmentation areas corresponding to each feature fusion strategy to obtain multiple symmetrical features.

[0063] For the feature fusion strategies corresponding to different groups of symmetrical segmentation regions, please refer to the following Tables 1 and 2 for details.

[0064] Table 1

[0065] In Table 1 above, the initial image area is divided into 8 segmented areas. HM2 to HM7 are used to represent the average grayscale values ​​corresponding to the second to seventh segmented areas. The segmented areas here can be as follows: Figure 3A and / or Figure 3B As shown, it can also be obtained based on other reasonable segmentation methods. HZ1~HZ5 are symmetry features determined to reflect the symmetry of the image to a certain extent.

[0066] The symmetry feature HZ1 is calculated by calculating the difference between the average grayscale values ​​corresponding to the fifth segment and the average grayscale values ​​corresponding to the fourth segment. A smaller difference indicates that the fourth segment and its corresponding symmetrical region, located on either side of the centerline of the initial image region, are similar in image features, and the initial image region contains highly symmetrical image elements (or, the image elements traversed by the centerline of the initial image region have strong symmetry). Conversely, a difference indicates that the two segmented regions have certain differences in image features, and the image elements in the initial image region have weaker symmetry (or, the image elements traversed by the centerline of the initial image region have weaker symmetry). (HM5 - HM4) × 2 is the feature fusion strategy corresponding to the symmetrical set of segmented regions, the fourth and fifth segments.

[0067] Similarly, the symmetry feature HZ2 is derived by calculating the difference in average grayscale values ​​between the fifth and fourth segments, and the difference in average grayscale values ​​between the sixth and third segments, and then summing these two differences. If the initial image region contains highly symmetrical image elements, the sum will be low, reflecting that the sixth segment and its corresponding symmetrical region (the third segment) on either side of the initial image region's centerline are similar in image features. Furthermore, the fourth segment and its corresponding symmetrical region (the fifth segment) are also similar in image features. This indicates that the initial image region contains highly symmetrical image elements. Conversely, a low sum indicates that the initial image region contains image elements with weak symmetry. HM5-HM4+HM6-HM3 is the feature fusion strategy for the two sets of symmetrical segments: the fourth and fifth segments, and the third and sixth segments.

[0068] The same is true for the symmetry feature HZ3, which will not be explained in detail here. Symmetry features HZ4 and HZ5 can also, to a certain extent, reflect image symmetry. Taking the symmetry feature HZ4 as an example, after averaging the average grayscale value of the fourth segmented region with the average grayscale value of the fifth segmented region, the calculated average value can be compared with the average grayscale value of the fourth segmented region and the average grayscale value of the fifth segmented region, respectively. If the calculated average value is not significantly different from the average grayscale value of the fourth segmented region and the average grayscale value of the fifth segmented region, it indicates that the initial image region contains image elements with strong symmetry. Otherwise, it indicates that the image elements contained in the initial image region have weak symmetry. The same is true for HZ5, which will not be explained in detail here. The feature fusion strategy used to determine the symmetry feature HZ5 is the feature fusion strategy corresponding to the three groups of symmetrical segmented regions: the second and seventh segmented regions, the third and sixth segmented regions, and the fourth and fifth segmented regions.

[0069] Table 2

[0070] In the above Table 2, the initial image area is divided into 8 segmented areas. VM2~VM7 are used to represent the standard deviation of the grayscale values ​​corresponding to the second to seventh segmented areas. The segmented areas here can be as follows: Figure 3A and / or Figure 3B As shown, it can also be obtained based on other reasonable segmentation methods. VZ1~VZ5 are symmetry features determined to reflect the symmetry of the image to a certain extent. For specific explanations, please refer to the explanations of the symmetry features in Table 1, and will not be repeated here.

[0071] The present application can determine the extraction features of the initial image region based on the image features, symmetry features, etc. of the segmented region. The image features of the initial image region directly determined above also belong to a type of extraction features.

[0072] The quantity detection device can use a variety of methods to determine the above extraction features.

[0073] For example, the quantity detection device can perform weighted fusion on the determined average grayscale value, grayscale value standard deviation and symmetry feature of each segmented area for each segmented area to obtain the fusion feature of the segmented area, and then add up the fusion features corresponding to each segmented area in the initial image area to obtain the extraction feature corresponding to the initial image area; for another example, the quantity detection device can re-average the average grayscale values ​​of all segmented areas to obtain the average grayscale value corresponding to the initial image area, average the grayscale value standard deviations of all segmented areas to obtain the average standard deviation corresponding to the initial image area, and average the symmetry features of all segmented areas to obtain the average symmetry feature corresponding to the initial image area, and then fuse the average grayscale value corresponding to the initial image area, the average standard deviation corresponding to the initial image area and the average symmetry feature corresponding to the initial image area to obtain the extraction feature corresponding to the initial image area; for another example, for each initial image area, the quantity detection device can splice the average grayscale value, grayscale value standard deviation and symmetry feature of each segmented area contained in the initial image area to obtain a vector with a dimension smaller than the pixel size of the initial image area, and use the vector as the extraction feature corresponding to the initial image area. Other methods are illustrated here one by one.

[0074] After determining the extraction features corresponding to each of the above-mentioned initial image areas, the similarity between these initial image areas can be calculated, that is, the quantity detection device can calculate the similarity between each initial image area and other initial image areas, so as to perform the first area screening based on the calculated similarity in the subsequent process.

[0075] In an embodiment of the present application, there may be multiple ways for the detection device to calculate the similarity between each initial image area. For example, the similarity between each initial image area may be determined by calculating the Euclidean distance between the corresponding extracted features of the initial image area; for another example, the similarity between each initial image area may be determined by calculating the cosine distance between the corresponding extracted features of the initial image area.

[0076] It should be noted that when calculating distance to determine similarity, the calculated distance value is negatively correlated with the similarity. That is, the larger the distance between the corresponding extracted features of two initial image regions, the less similar the two initial image regions are, and the lower the similarity between the two initial image regions. Conversely, the smaller the distance between the corresponding extracted features of two initial image regions, the more similar the two initial image regions are, and the higher the similarity between the two initial image regions.

[0077] Therefore, in the embodiment of the present application, the measurement device can directly determine the similarity between the initial image regions based on the calculated distance value between the corresponding extracted features. In this case, the distance value and the similarity are numerically equivalent. Of course, the measurement device can also determine the similarity between the initial image regions based on the calculated distance value, such that the distance value and the similarity are negatively correlated.

[0078] The quantity detection device can also calculate the similarity between the initial image regions using other methods. For example, the quantity detection device can input the extracted features corresponding to the initial image regions into a preset model. The preset model, through the feature expression analysis capabilities learned through previous training, determines the degree of correlation between the extracted features in the feature space, and thus outputs the similarity between the initial image regions. Other methods for calculating similarity are not listed here.

[0079] It should be noted that before dividing the image to be labeled into multiple initial image regions, the image to be labeled can be preprocessed. For example, the image to be labeled can be scaled to adjust to a uniform specified pixel size. Another example is that the image to be labeled can be noise filtered to remove noise pixels contained in the image to be labeled. Other preprocessing methods are not listed here.

[0080] S104: Filtering a target image region from the multiple initial image regions based on the similarity.

[0081] For any initial image region, if the similarity between the initial image region and other initial image regions is greater, it means that the commonality in graphic features between the initial image region and other initial image regions is stronger; otherwise, it means that the commonality in graphic features between the initial image region and other initial images is weaker.

[0082] On this basis, the quantity detection device can screen out target image areas with significant differences from multiple initial image areas based on the determined similarity. In an embodiment of the present application, there can be multiple ways to screen target image areas based on similarity. For example, the quantity detection device can sort the similarities between each initial image area and multiple other initial image areas from large to small, and use the similarities before the first set ranking as the target similarity corresponding to the initial image area. Then, the quantity detection device can sort the target similarities corresponding to each initial image area from small to large, and then determine the initial image area before the second set ranking as the screened target image area.

[0083] For another example, after determining the similarity, the detection device can also determine the score corresponding to each initial image area based on the similarity and the texture features of each initial image area, and filter out the target image area from the multiple initial image areas based on the calculated score. Among them, if the similarity corresponding to an initial image area is lower and the texture features are more obvious, the score corresponding to the initial image area is higher. Therefore, the initial image area with the highest score can be filtered as the target image area. Other methods are not explained here one by one.

[0084] As can be seen from the above method, for each initial image region, the detection device actually first determines the largest (or maximum) similarity between that initial image region and other initial image regions, which serves as a proxy for the similarity between the graphics in that initial image region and other graphics in the entire image to be labeled (i.e., the target similarity). It then selects the initial image region with the smallest (or minimum) target similarity as the target image region.

[0085] Therefore, if the target similarity of an initial image region is smaller, it means that the difference between the graphics on the initial image region and other graphics in the entire image to be labeled is greater, and the uniqueness of the initial image region in the entire image to be labeled is stronger.

[0086] Therefore, the target image region selected by the above method is a region with strong uniqueness in the entire image to be marked, and is significantly different from other initial image regions in terms of graphic characteristics.

[0087] Alternatively, the quantity detection device can use the calculated distance value as similarity, and since the distance value and similarity are negatively correlated, when screening target image regions based on similarity, the quantity detection device can also sort the distance values ​​between each initial image region and multiple other initial image regions from smallest to largest, using the distance value before the third set ranking as the target distance value corresponding to the initial image region. Subsequently, the quantity detection device can sort the target distance values ​​corresponding to the initial image regions from largest to smallest, and then determine the initial image region before the fourth set ranking as the target image region.

[0088] Similar to the principle described above, using target distance values ​​to filter target image regions essentially involves filtering out highly unique initial image regions from the image to be labeled. However, since the distance value here is negatively correlated with similarity, the distance values ​​between the initial image region and multiple other initial image regions are sorted from small to large. Based on the sorting results, the smallest (or smallest) distance value between the initial image region and the other initial image regions is determined, which serves as a representation of whether the graphics of the initial image region are similar to other graphics in the entire image to be labeled (i.e., the target distance value). Target image regions are then sorted from large to small by the target distance values, filtering out target image regions from the image to be labeled that are dissimilar in graphical characteristics or have weak similarities with other initial image regions.

[0089] Regardless of which of the above methods is used to determine the target image region, at least one target image region may be determined. In other words, to more accurately determine the most unique region in the image to be labeled, the detection device may select multiple target image regions from the initial image region. The specific number of target image regions to be selected may be determined based on the second and fourth setting rankings determined based on actual needs.

[0090] Of course, the first and third preset rankings can also be determined based on actual needs. Therefore, in actual applications, the detection device does not necessarily need to use the maximum similarity as the target similarity or the minimum distance as the target distance. This allows more suitable target image areas to be screened during the first screening process, thereby more comprehensively covering the most unique areas in the image to be marked, thereby ensuring the accuracy of the subsequent determination of the target marking area.

[0091] S106: Segment the target image region into a plurality of candidate marker regions of different sizes.

[0092] After determining the target image area, the detection device can segment it into multiple candidate marker areas of different sizes. The purpose of this is to determine multiple image areas where distinctly unique graphics may appear within the neighborhood of the target image area, so as to accurately determine the area containing distinctly unique graphics as the target marker area in the subsequent process. Figure 4A and Figure 4B shown.

[0093] Figure 4A and Figure 4B A schematic diagram of the process of determining multiple candidate marking areas of different sizes provided in an embodiment of the present application.

[0094] Figure 4AThe dotted box area in the image to be marked is the determined target image area. The detection device can use the target image area as a reference to determine multiple candidate marking areas of different sizes in the image to be marked. Figure 4B The multiple candidate marking areas of different sizes in are the areas covered by the multiple dotted boxes in the image to be marked.

[0095] In embodiments of the present application, the quantity detection device can determine multiple candidate marker regions of different sizes in various ways. For example, the quantity detection device can first determine the center point of the target image region, and then scale the target image region around the center point, thereby determining multiple candidate marker regions that have the same center point as the target image region but different sizes.

[0096] For another example, the measurement device can first determine the center point of the target image area and then move the center point within a preset range to obtain multiple center point positions. The measurement device can then determine multiple candidate marking areas of different sizes based on these center point positions. The sizes of these candidate marking areas can be adjusted within a certain range based on the size of the target image area.

[0097] For another example, the detection device may slide the target image area around within a certain range, and then scale the size of the image area obtained by sliding to different degrees, thereby obtaining multiple candidate marking areas of different sizes.

[0098] Of course, in actual applications, the detection device can also determine multiple candidate marking areas of different sizes through other methods, which will not be explained one by one here.

[0099] S108: Calculate a uniqueness evaluation value of the candidate marking region in the image to be marked.

[0100] The uniqueness evaluation value represents the degree of uniqueness of the candidate marking region within the image to be marked. This value can be calculated in a variety of ways. For example, a Fourier transform can be performed on the candidate marking region and the image to be marked to obtain their respective frequency domain representations. Frequency spectrum information can then be extracted from the frequency domain representations. Difference analysis (e.g., mean square error, KL divergence) can then be performed on the information related to the candidate marking region and the image to be marked to obtain the uniqueness evaluation value of the candidate marking region.

[0101] For another example, the quantity detection device may also determine the uniqueness evaluation value of the candidate marker image in the image to be marked by calculating the degree of match between the candidate marker region and other regions. In a specific process, the quantity detection device may slide each candidate marker region in the image to be marked to calculate the degree of match between the candidate marker region and each sliding region. The calculated degrees of match may then be sorted from largest to smallest, and the degree of match that is before the sixth set ranking and is less than a preset value is determined as the target degree of match for the candidate marker region.

[0102] The sixth setting ranking and preset value here can be flexibly set based on the solution design and practical needs. For example, the preset value can be used to eliminate the matching degree calculated with the candidate marking area itself, and can also be used to eliminate the matching degree that may bring redundancy to the application accuracy.

[0103] Among them, the above-mentioned method of calculating the matching degree can be implemented in a similar way to calculating the similarity of image features. Then, the higher the matching degree, the more similar the candidate marking area is to the sliding area, and the lower the uniqueness of the candidate marking area in the image to be marked (which can be represented by the uniqueness evaluation value), that is, the matching degree is negatively correlated with the uniqueness evaluation value.

[0104] Therefore, the quantity detection device may determine the determined target matching degree as the uniqueness evaluation value of the candidate marking area in the image to be marked.

[0105] Other methods of calculating uniqueness evaluation values ​​are not described here one by one.

[0106] S110: Determine a target marking region based on the uniqueness evaluation values ​​of the plurality of candidate marking regions.

[0107] There are many ways to filter the target marking area from multiple candidate marking areas. For example, you can first preliminarily filter out some uniqueness evaluation values ​​(such as filtering out the uniqueness evaluation values ​​corresponding to the candidate marking areas located in specific areas of the wafer image (such as the edge area, the area scanned by the camera later), and filtering out the uniqueness evaluation values ​​outside the acceptable range (determined by engineers based on theoretical and practical experience, such as the uniqueness evaluation value is less than a certain threshold value according to the design drawing, and the acceptable range is determined by the threshold value; for example, the acceptable range is reasonably set to improve the processing speed)), and then sort the remaining uniqueness evaluation values ​​from large to small, and determine the candidate marking areas corresponding to the uniqueness evaluation values ​​greater than the preset ranking as the target marking areas; for another example, you can directly sort the uniqueness evaluation values ​​of each candidate marking area in order from large to small, and use the candidate marking areas ranked before the fifth set ranking as the target marking areas.

[0108] For another example, when the above-mentioned target matching degree is used as the uniqueness evaluation value of the candidate marking area in the image to be marked, the quantity detection device can sort the candidate marking areas in order from small to large according to the target matching degree, and use the candidate marking areas ranked before the seventh set ranking as the target marking areas.

[0109] Because regions with a high degree of uniqueness actually have a low (or minimal) degree of match with other sliding regions, the target marking regions determined in ascending order of target matching degrees are those with a high (or highest) degree of uniqueness in the image to be marked. The number of target marking regions selected can be one or more, depending on the specific value of the seventh setting ranking. The specific values ​​of the sixth and seventh setting rankings can be determined based on actual needs.

[0110] As can be seen from the above method, a two-stage matching process can be used to identify target marking regions with distinct distinguishing characteristics within the image to be marked. Specifically, several candidate marking regions with distinct distinguishing characteristics are first identified across the entire image to be marked. Then, from these candidate marking regions, a target marking region that is clearly distinct from the other candidate marking regions is identified. This not only eliminates the need for adding additional physical marking points on the target device, reducing costs, but also ensures that the identified target marking region is distinct from other image elements in the image to be marked, thereby ensuring accurate alignment and positioning of subsequent images.

[0111] Furthermore, the target marking area determined by the quantity detection device in the above manner is equivalent to the marking point determined in the image to be marked. On this basis, the quantity detection device can perform image alignment operations based on the target marking area determined in the image to be marked.

[0112] Specifically, the quantity detection device can obtain another image to be aligned with the image to be marked. Then, through image region matching, it can identify an image region in the other image that matches the target marking region, and use this region as the alignment region for the other image. The quantity detection device can then align the image to be marked with the other image based on the target marking region and the alignment region.

[0113] The image region matching method used in image alignment can be the same as the method for determining the matching degree described above, and will not be described in detail here. When the image alignment is completed by the measurement detection device, subsequent operations can be performed based on the alignment results.

[0114] For example, when the image to be marked and the other image are wafer images, the measurement inspection equipment can perform wafer defect detection based on the aligned image to be marked and the other image. Specifically, when the image to be marked is a wafer image without defects, it can be used as a template image. By aligning the template image with the other image, the template image and the other image can be compared to detect whether the wafer corresponding to the other image has defects.

[0115] For another example, when the image to be marked is a wafer image, the measurement device can further determine the image coordinates of the target marking area in the image to be marked. The measurement device can then determine whether the position of the wafer corresponding to the image to be marked on the machine needs to be adjusted by calculating the difference between the image coordinates of the target marking area and the theoretical coordinates. The theoretical coordinates of the target marking area can refer to the image coordinates of the target marking area in the image to be marked when the wafer is correctly placed on the machine.

[0116] Furthermore, the measurement detection device can control a designated device to adjust the position of the wafer on the machine according to the difference in the determined coordinates. This can be achieved by adjusting the angle, adjusting the position of the wafer center, etc.

[0117] The designated device mentioned above can include various types, such as a robotic arm or a carrier for picking up wafers. Based on this, when the designated device is a robotic arm, the measurement and detection device can plan the robotic arm's operating path based on the determined coordinate differences, controlling the robotic arm to pick up the wafer and reposition it at the planned location. If the designated device is a carrier, the measurement and detection device can fine-tune the carrier based on the determined coordinate differences to adjust the position and / or angle of the wafer.

[0118] In addition, the amount detection device can achieve the above process through a preset tool in the process of determining the target marking area. The preset tool here can refer to a tool such as Halcon.

[0119] In the above process of determining the candidate marking area, the detection device can actually first determine multiple initial candidate marking areas of different sizes, and then determine multiple candidate marking areas of different sizes based on these initial candidate marking areas. The obtained candidate marking areas can be determined on the basis of fine-tuning the position or rotating the angle of the initial candidate marking areas.

[0120] Specifically, when using the halcon tool to perform image region matching to determine the target marking area, you can enter the following fields in the halcon tool: create_shape_model(Template::NumLevels,AngleStart,AngleExtent,AngleStep,Optimization, Metric,Contrast,MinContrast : ModelID).

[0121] The purpose of this field is to determine multiple candidate marking areas of different sizes in the image to be marked by the halcon tool. Therefore, Template is used to indicate that the candidate marking areas that meet the requirements are to be determined, and the fields subsequent to Template can be understood as the condition fields used to determine the candidate marking areas.

[0122] NumLevels is used to indicate the maximum pyramid level. This is actually used to reflect whether the candidate marker area needs to be determined in the initial candidate marker area. For example, when NumLevels=1, it means that the candidate marker area is determined only based on the initial candidate marker area, and the initial candidate marker area itself is not scaled in size.

[0123] AngleStart is used to indicate the minimum rotation arc, and AngleExtent is used to indicate the rotation arc range. This is mainly used to reflect the rotation range when determining the candidate marker area based on the initial candidate marker area. For example, when AngleStart=-0.1 (-5.7°) and AngleExtent=0.2 (11.5°), it is used to indicate that the halcon tool will rotate the center point of the initial candidate marker area within the range of -5.7° to 11.5° to determine the candidate marker area corresponding to the initial candidate marker area.

[0124] AngleStep is used to represent the rotation arc step, that is, the step size of each rotation when rotating the initial candidate marker area through the center point of the initial candidate marker area. For example, when AngleStep='auto', the halcon tool automatically selects the rotation step size to balance computational efficiency and accuracy.

[0125] Optimization is used to indicate the optimization method used. Optimization is primarily used to determine how much image detail needs to be retained or reflected in the candidate marker area that is ultimately found. For example, when Optimization='auto', the Halcon tool automatically selects the optimization method. The choice of different optimization methods can result in differences in image details such as the number of image feature points contained in the candidate marker area and whether low-contrast edges are filtered out.

[0126] Metric is used to represent the grayscale characteristics considered when determining candidate marker regions. It can reflect whether the grayscale characteristics of the final candidate marker region are consistent with those of the initial candidate marker region. Generally, it is necessary to ensure that the grayscale characteristics of the final candidate marker region are consistent with those of the initial candidate marker region.

[0127] Contrast is used to indicate the contrast threshold of the pyramid level, which is mainly used to constrain the degree of association between the final candidate marker area and the initial candidate marker area. For example, when Contrast=[14,29,5], 14 means that the final candidate marker area needs to contain edge points or contour points with a contrast greater than or equal to 14 in the initial candidate marker area, 29 means that the final candidate marker area needs to contain edge points or contour points with a contrast less than or equal to 29 in the initial candidate marker area, and 5 means that the final candidate marker area needs to contain at least 5 unit pixel areas in the initial candidate marker area.

[0128] MinContrast is used to indicate the minimum contrast, and is used to limit the minimum contrast of edge points or contour points of interest when searching for candidate marker regions. For details, please refer to the example used in the above explanation of Contrast.

[0129] ModelID is used to indicate the identification information corresponding to the determined initial candidate marking area.

[0130] The quantity detection device can use the halcon tool to determine multiple candidate marking areas of different sizes through the above fields. Afterwards, the quantity detection device can further use the halcon tool to determine the target marking area from these candidate marking areas.

[0131] For example, you can enter the following fields in the halcon tool: find_shape_model(Image::ModelID,AngleStart,AngleExtent,MinScore,NumMatches,MaxOverlap, SubPixel, NumLevels, Greediness: Row, Column, Angle, Score).

[0132] ModelID is the identification information corresponding to the candidate marking area determined in the above manner.

[0133] AngleStart and AngleExtent indicate that when matching the candidate marker area with the sliding area, it can be rotated within a certain angle range, and the matching process is completed through the rotated area.

[0134] MinScore actually specifies which sliding areas the candidate marker area matches with and which matching scores need to be retained. For example, when MinScore = 0, it means that the matching scores calculated between the candidate marker area and each sliding area need to be retained.

[0135] NumMatches specifies the number of regions that need to be selected from each sliding region to match the candidate marker region. For example, when NumMatches=2, it means that the two sliding regions that best match the candidate marker region need to be selected from each sliding region.

[0136] MaxOverlap specifies how to select matching regions from among the sliding regions based on the degree of region overlap. For example, when MaxOverlap = 0.1, if two sliding regions have a high degree of match with the candidate marking region, but the overlap between the two sliding regions exceeds 10%, then one of the sliding regions will be selected for return.

[0137] SubPixel is used to indicate the accuracy of the best matching area when filtering out the area that best matches the candidate marker area (including the candidate marker area itself) from each sliding area. For example, when SubPixel='none', it means that in this case only the integer coordinates of the best matching area will be returned, and the accuracy will not be accurate to sub-pixel.

[0138] NumLevels is used to indicate the strategy adopted in the matching process. In other words, the number of image pyramid levels (ie, NumLevel) actually specifies how many layers of pyramid images need to be controlled for search during the matching process.

[0139] The matching process based on NumLevel can be roughly regarded as first finding several sliding areas in the image to be marked that may match the candidate marking area. These sliding areas are often larger in size than the candidate marking area. Then, the area that best matches the candidate marking area can be found within these sliding areas. The final determined area is also comparable in size to the candidate marking area.

[0140] Therefore, the value of NumLevel reflects the number of pyramid image layers used in the matching process. For example, when NumLevel=3, it means that several sliding regions that may match the candidate marker region are first matched. Then, within these sliding regions, several regions that are closer matches to the candidate marker region are searched. Finally, within these closer matches, the region that is closest matches to the candidate marker region is searched again. The entire process involves three searches, so NumLevel=3.

[0141] When NumLevel=0, the halcon tool will automatically select the appropriate number of pyramid image layers for the matching process.

[0142] Greediness is used to indicate the greedy strategy used during the matching process. The greediness used is related to its value. For example, when Greediness is close to 1 (such as 0.9), it indicates that a certain degree of matching error is acceptable in exchange for faster matching speed. When Greediness is close to 0, the algorithm will be more conservative and try more possibilities to ensure that it finds a sliding area that matches the candidate marked area.

[0143] Row and Column represent the image coordinates of the final matched sliding area, while Angle refers to the matching angle used to obtain the most closely matched sliding area. Score indicates the degree of matching.

[0144] From the above process, it can be seen that the quantity detection device can use the halcon tool to determine the area with a strong degree of uniqueness in the image to be marked as the target marking area from a large number of candidate marking areas while taking into account the matching efficiency.

[0145] Furthermore, the detection device may determine more than one target image region in step S104. Subsequently, multiple candidate marking regions of varying sizes may need to be determined based on each target image region. This further increases the search range and ensures that the target marking region ultimately determined is the most unique within the entire image to be marked, and can be clearly distinguished from graphics in other regions.

[0146] The above is a marking area determination method provided in one or more embodiments of the present application. Based on the same idea, the present application also provides a corresponding marking area determination device, such as Figure 5 shown.

[0147] Figure 5 A schematic diagram of a marking area determination device provided in an embodiment of the present application includes: An acquisition module 500 is configured to acquire an image to be labeled and identify a plurality of initial image regions from the image to be labeled; A first calculation module 502 is used to calculate the similarity between multiple initial image regions; A screening module 504 is configured to screen a target image region from a plurality of initial image regions based on similarity; a segmentation module 506 for segmenting a target image region into a plurality of candidate marker regions of different sizes; A second calculation module 508 is configured to calculate a uniqueness evaluation value of the candidate marking region in the image to be marked, where the uniqueness evaluation value represents the degree of uniqueness of the pattern of the candidate marking region in the image to be marked; The determination module 510 is configured to determine a target marking region based on uniqueness evaluation values ​​of the plurality of candidate marking regions.

[0148] Optionally, the first calculation module 502 is specifically used to divide each initial image area into multiple segmented areas, and determine the extraction features corresponding to the initial image area based on the image features of the multiple segmented areas; and calculate the similarity between the multiple initial image areas based on the extraction features corresponding to the initial image area.

[0149] Optionally, the first calculation module 502 is specifically configured to divide the initial image region into a plurality of segmented regions in a horizontal direction, and / or divide the initial image region into a plurality of segmented regions in a vertical direction.

[0150] Optionally, the first calculation module 502 is specifically used to determine, for each segmented area, the average grayscale value corresponding to the segmented area based on the grayscale value of each pixel contained in the segmented area, and determine the standard deviation of the grayscale value corresponding to the segmented area based on the average grayscale value corresponding to the segmented area and the grayscale value of each pixel contained in the segmented area; determine the average grayscale value corresponding to the segmented area and / or the standard deviation of the grayscale value corresponding to the segmented area as the image feature of the segmented area, so as to determine the extraction feature corresponding to the initial image area based on the image feature of the segmented area.

[0151] Optionally, the first calculation module 502 is specifically used to determine, based on the center line of the initial image area, at least one group of symmetrical segmented areas located on both sides of the center line from each segmented area of ​​the initial image area; determine the symmetrical features of the initial image area based on the image features of at least one group of symmetrical segmented areas, and determine the extraction features corresponding to the initial image area based on the symmetrical features, the average grayscale value and the grayscale value standard deviation.

[0152] Optionally, the first calculation module 502 is specifically used to obtain multiple feature fusion strategies for calculating symmetrical features; select at least one group of symmetrical segmentation regions corresponding to each feature fusion strategy from multiple groups of symmetrical segmentation regions; and based on each feature fusion strategy, fuse the image features of at least one group of symmetrical segmentation regions corresponding thereto to obtain multiple symmetrical features.

[0153] Optionally, the segmentation module 506 is specifically configured to slide the image to be marked through a preset sliding window according to a preset path at a preset step to obtain a plurality of initial image regions, where the preset step is smaller than a size of the preset sliding window.

[0154] Optionally, the screening module 504 is specifically used to, for each initial image area, sort the similarities between the initial image area and multiple other initial image areas from large to small, and use the similarities before a first set ranking as the target similarity corresponding to the initial image area; sort the target similarities corresponding to each initial image area from small to large, and determine the initial image area before a second set ranking as the target image area.

[0155] Optionally, the first calculation module 502 is specifically configured to calculate distance values ​​between the extracted features corresponding to the initial image regions, and determine the distance values ​​as similarities between the multiple initial image regions; The screening module 504 is specifically used to, for each initial image area, sort the distance values ​​between the initial image area and multiple other initial image areas from small to large, and use the distance value before the third set ranking as the target distance value corresponding to the initial image area; sort the target distance values ​​corresponding to each initial image area from large to small, and determine the initial image area before the fourth set ranking as the target image area.

[0156] Optionally, the determining module 510 is specifically configured to sort the multiple candidate marking regions in descending order of uniqueness evaluation values, and select the candidate marking region ranked before the fifth set ranking as the target marking region.

[0157] Optionally, the second calculation module 508 is specifically configured to, for each candidate marking region, slide the candidate marking region in the image to be marked to calculate a matching degree between the candidate marking region and each sliding region; sort the multiple matching degrees from largest to smallest, determine a matching degree that is before a sixth set ranking and is smaller than a preset value as a target matching degree of the candidate marking region, and determine the target matching degree as a uniqueness evaluation value of the candidate marking region in the image to be marked; The determining module 510 is specifically configured to sort the candidate marking regions in ascending order of the target matching degrees, and select the candidate marking regions ranked before the seventh set ranking as the target marking regions.

[0158] Optionally, the device further includes: The alignment module 512 obtains other images, and the other images and the image to be marked are the images to be aligned; determines the image area that matches the target marking area in the other images as the alignment area corresponding to the other images; and aligns the image to be marked with the other images based on the target marking area and the alignment area.

[0159] Optionally, the image to be marked and the other images are wafer images; The equipment also includes: The detection module 514 performs wafer defect detection based on the aligned images to be marked and other images.

[0160] The present application also provides a computer-readable storage medium, which stores a computer program that can be used to execute the above Figure 1 A method for determining a marked area is provided.

[0161] This application also provides Figure 6 The schematic structure diagram of the electronic device shown in FIG. Figure 6 At the hardware level, the device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 A method for determining a marked area. Of course, in addition to software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0162] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using physical hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly performed using software called a "logic compiler." This is similar to the software compilers used during program development. Before compilation, the original code must be written in a specific programming language, called a Hardware Description Language (HDL). There are many types of HDL, including ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that simply by programming a method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0163] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the memory control logic. Those skilled in the art will also appreciate that, in addition to implementing the controller purely in computer-readable program code, the controller can also be implemented in the form of logic gates, switches, an application-specific integrated circuit, a programmable logic controller, an embedded microcontroller, etc. by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the means for implementing the various functions included therein can also be considered as structures within the hardware component. Alternatively, the means for implementing the various functions can be considered both a software module implementing the method and a structure within the hardware component.

[0164] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0165] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0166] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0168] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0170] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0171] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0172] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0173] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0174] This specification may be described in the general context of computer-executable instructions, such as program modules, executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media, including storage devices.

[0175] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0176] The above are merely examples of the present invention and are not intended to limit the present invention. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A method for determining a marked area, characterized in that: include: Acquire an image to be marked, and identify a plurality of initial image regions from the image to be marked; Calculating similarities between the multiple initial image regions; Based on the similarity, screening a target image region from the multiple initial image regions; Segmenting a plurality of candidate marker regions of different sizes from the target image region; Calculating a uniqueness evaluation value of the candidate marking region in the image to be marked, wherein the uniqueness evaluation value represents a uniqueness degree of the pattern of the candidate marking region in the image to be marked; A target marker region is determined based on the uniqueness evaluation values ​​of the plurality of candidate marker regions.

2. The method according to claim 1, wherein Calculating the similarity between the multiple initial image regions specifically includes: For each initial image region, the initial image region is divided to obtain a plurality of segmented regions, and an extraction feature corresponding to the initial image region is determined based on image features of the plurality of segmented regions; The similarities between the multiple initial image regions are calculated based on the extracted features corresponding to the initial image regions.

3. The method according to claim 2, wherein The initial image area is divided into multiple segmented areas, including: The initial image region is divided into a plurality of segmented regions in a horizontal direction, and / or the initial image region is divided into a plurality of segmented regions in a vertical direction.

4. The method according to claim 2 or 3, wherein: Determining, based on the image features of the plurality of segmented regions, an extraction feature corresponding to the initial image region, specifically includes: For each segmented area, determine the average grayscale value corresponding to the segmented area based on the grayscale value of each pixel contained in the segmented area, and determine the standard deviation of the grayscale value corresponding to the segmented area based on the average grayscale value corresponding to the segmented area and the grayscale value of each pixel contained in the segmented area; The average grayscale value corresponding to the segmented area and / or the grayscale value standard deviation corresponding to the segmented area are determined as the image features of the segmented area, so as to determine the extraction features corresponding to the initial image area according to the image features of the segmented area.

5. The method according to claim 4, wherein Determine the extraction features corresponding to the initial image region based on the image features of the segmented region, specifically including: According to the center line of the initial image area, determining at least one group of symmetrical segmented areas located on both sides of the center line from the segmented areas of the initial image area; The symmetric features of the initial image region are determined according to the image features of the at least one group of symmetrical segmented regions, and the extraction features corresponding to the initial image region are determined based on the symmetric features, the average grayscale value and the grayscale value standard deviation.

6. The method according to claim 5, wherein Determining the symmetry features of the initial image region according to the image features of the at least one group of symmetric segmented regions specifically includes: Obtain multiple feature fusion strategies for calculating symmetric features; From the multiple groups of symmetrical segmentation regions, select at least one group of symmetrical segmentation regions corresponding to each feature fusion strategy; Based on each feature fusion strategy, the image features of at least one group of symmetrical segmented areas corresponding thereto are fused to obtain a plurality of symmetrical features.

7. The method according to claim 1, wherein Identifying a plurality of initial image regions from the image to be marked specifically includes: The image to be marked is sampled by sliding a preset sliding window along a preset path with a preset step, to obtain a plurality of initial image regions, wherein the preset step is smaller than the size of the preset sliding window.

8. The method according to claim 1, wherein Based on the similarity, screening a target image region from the multiple initial image regions specifically includes: For each initial image region, sort the similarities between the initial image region and multiple other initial image regions from largest to smallest, and use the similarities before the first set ranking as the target similarity corresponding to the initial image region; The target similarities corresponding to the initial image regions are sorted from small to large, and the initial image regions located before the second set ranking are determined as the target image regions.

9. The method according to claim 2, wherein Calculating similarities between the multiple initial image regions based on the extracted features corresponding to the initial image regions includes: Calculating distance values ​​between the extracted features corresponding to the respective initial image regions, and determining the distance values ​​as similarities between the plurality of initial image regions; Based on the similarity, screening a target image region from the multiple initial image regions specifically includes: For each initial image region, sort the distance values ​​between the initial image region and multiple other initial image regions from small to large, and use the distance value before the third set ranking as the target distance value corresponding to the initial image region; The target distance values ​​corresponding to the initial image regions are sorted from largest to smallest, and the initial image regions located before the fourth set ranking are determined as the target image regions.

10. The method according to claim 1, wherein Determining a target marking region based on the uniqueness evaluation values ​​of the multiple candidate marking regions specifically includes: The plurality of candidate marker regions are sorted in descending order of uniqueness evaluation values, and a candidate marker region ranked before a fifth set ranking is used as a target marker region.

11. The method according to claim 1, wherein Calculating a uniqueness evaluation value of the candidate marking region in the image to be marked specifically includes: For each candidate marking region, slide the candidate marking region in the image to be marked to calculate the matching degree between the candidate marking region and each sliding region; sorting the plurality of matching degrees from largest to smallest, determining a matching degree that is before a sixth set ranking and is smaller than a preset value as a target matching degree of the candidate marking region, and determining the target matching degree as a uniqueness evaluation value of the candidate marking region in the image to be marked; Determining a target marking region based on the uniqueness evaluation values ​​of the plurality of candidate marking regions specifically includes: The candidate marking regions are sorted in ascending order according to the target matching degree, and the candidate marking regions ranked before the seventh set ranking are used as target marking regions.

12. The method according to claim 1, wherein The method further comprises: Acquire another image, where the other image and the image to be marked are images that need to be aligned; Determining an image region matching the target marked region in the other image as an alignment region corresponding to the other image; The image to be marked is aligned with the other images according to the target marking area and the alignment area.

13. The method according to claim 12, wherein: The image to be marked and the other images are wafer images; The method further comprises: Wafer defect detection is performed based on the aligned image to be marked and the other images.

14. A marking area determination device, characterized in that: include: an acquisition module, configured to acquire an image to be labeled and identify a plurality of initial image regions from the image to be labeled; A first calculation module, configured to calculate similarities between the multiple initial image regions; a screening module, configured to screen a target image region from the plurality of initial image regions based on the similarity; a segmentation module, configured to segment the target image region into a plurality of candidate marker regions of different sizes; A second calculation module is configured to calculate a uniqueness evaluation value of the candidate marking region in the image to be marked, wherein the uniqueness evaluation value represents a uniqueness degree of the pattern of the candidate marking region in the image to be marked; A determination module is configured to determine a target marking region based on the uniqueness evaluation values ​​of the plurality of candidate marking regions.

15. A computer-readable storage medium, characterized in that The storage medium contains a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

16. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 13 is implemented.

Images