Image alignment method, computer equipment and computer storage medium

Abstract

The embodiment of the invention discloses an image alignment method, computer equipment and a computer storage medium. Taking the sub-region as a template image, taking an image obtained by externally expanding each edge of the sub-region by a preset size as an alignment image, and performing template matching on the template image and the alignment image according to a template matching algorithm to obtain a plurality of alignment pixel points and a correlation coefficient corresponding to each alignment pixel point; and according to the correlation coefficient of the plurality of alignment pixel points and the gray value of each alignment pixel point, determining the coordinate of a target alignment pixel point of the sub-region. And determining the alignment coordinates of the target die image according to the coordinates of the target alignment pixel points of the plurality of sub-regions. Since the alignment coordinate is calculated based on the pixel distribution condition and the periodic change condition of each sub-region of the target die image, the target die image and the reference image can be accurately aligned based on the alignment coordinate, the influence of the periodic pattern of the die image on the image alignment can be avoided, and the alignment precision of the die image is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of die image processing, and specifically to an image alignment method, a computer device, and a computer storage medium. Background Art

[0002] When performing defect detection on a die, the camera images the die to obtain a die image. The die image contains both periodic and non-periodic patterns. The periodic pattern is a pattern formed by crisscrossing shapes of different grayscales, and the non-periodic pattern is a pattern with uniform grayscale. Figure 1 As shown, the overall grayscale of the left pattern is uniform, which is a non-periodic pattern; the right pattern has lines of different grayscales interlaced in the horizontal direction, which is a periodic pattern.

[0003] The camera cannot control the motion accuracy of the image when imaging the die, and the periodic pattern has pattern repetitiveness, which leads to misalignment between the template image and the die image during die defect detection. In addition, due to the presence of non-periodic patterns, the image alignment is biased, resulting in more false detections in the non-periodic pattern area. Summary of the invention

[0004] The embodiments of the present application provide an image alignment method, a computer device, and a computer storage medium for determining accurate alignment coordinates of a die image to improve the alignment accuracy of the die image.

[0005] A first aspect of an embodiment of the present application provides an image alignment method, the method comprising:

[0006] Acquire a target die image to be processed, divide a target area in the target die image into a plurality of sub-areas, and separate an edge of the target area from an edge of the target die image by a preset size;

[0007] The following steps are performed for each sub-region where a periodic pattern exists:

[0008] The sub-region is used as a template image, and each side of the sub-region is expanded by the preset size to obtain an image as a registration image;

[0009] Performing template matching on the template image and the alignment image according to a template matching algorithm to obtain a plurality of alignment pixels and a correlation coefficient corresponding to each of the alignment pixels, wherein the correlation coefficient is used to characterize the similarity between the alignment region of the alignment image and the template image when the alignment image and the template image are aligned based on the alignment pixels;

[0010] Determine the coordinates of the target alignment pixel point of the sub-area according to the correlation coefficients respectively corresponding to the plurality of alignment pixel points and the grayscale value of each alignment pixel point;

[0011] When the coordinates of the target alignment pixels of the multiple sub-areas are obtained, the alignment coordinates of the target die image are determined according to the coordinates of the target alignment pixels of the multiple sub-areas.

[0012] A second aspect of an embodiment of the present application provides a computer device, the computer device comprising:

[0013] an acquisition unit, configured to acquire a target die image to be processed, and divide a target area in the target die image into a plurality of sub-areas, wherein an edge of the target area is spaced from an edge of the target die image by a preset size;

[0014] A computing unit, configured to perform the following steps for each sub-region having a periodic pattern:

[0015] The sub-region is used as a template image, and each side of the sub-region is expanded by the preset size to obtain an image as a registration image;

[0016] Performing template matching on the template image and the alignment image according to a template matching algorithm to obtain a plurality of alignment pixels and a correlation coefficient corresponding to each of the alignment pixels, wherein the correlation coefficient is used to characterize the similarity between the alignment region of the alignment image and the template image when the alignment image and the template image are aligned based on the alignment pixels;

[0017] Determine the coordinates of the target alignment pixel point of the sub-area according to the correlation coefficients respectively corresponding to the plurality of alignment pixel points and the grayscale value of each alignment pixel point;

[0018] A determination unit is used to determine the alignment coordinates of the target die image according to the coordinates of the target alignment pixel points of the multiple sub-areas when obtaining the coordinates of the target alignment pixel points of the multiple sub-areas.

[0019] A third aspect of an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0020] A fourth aspect of an embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method of the first aspect.

[0021] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0022] The following steps are performed for each sub-region of the target die image where a periodic pattern exists: the sub-region is used as a template image, each side of the sub-region is expanded by a preset size to obtain an image as an alignment image, the template image is matched with the alignment image according to a template matching algorithm, a plurality of alignment pixels and a correlation coefficient corresponding to each alignment pixel are obtained, the coordinates of the target alignment pixels of the sub-region are determined according to the correlation coefficients corresponding to the plurality of alignment pixels and the grayscale values ​​of the alignment pixels, and the alignment coordinates of the target die image are determined according to the coordinates of the target alignment pixels of the plurality of sub-regions. Since the alignment coordinates of the target die image are calculated based on the pixel distribution and periodic variation of each sub-region of the target die image, the target die image and the reference image can be accurately aligned based on the alignment coordinates, and the influence of the periodic pattern of the die image on the image alignment can be avoided, so that the die image alignment is more accurate during die defect detection, and the accuracy of die defect detection is ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is an exemplary die image in the embodiment of the present application;

[0024] Figure 2 A schematic diagram of a flow chart of an image alignment method in an embodiment of the present application;

[0025] Figure 3 is an exemplary target die image in the embodiment of the present application;

[0026] Figure 4 for Figure 3 A schematic diagram of a method of dividing multiple sub-regions of a target die image is shown;

[0027] Figure 5 for Figure 4 Schematic diagram of a method for determining an alignment image and a template image corresponding to one of the sub-regions of the target die image;

[0028] Figure 6 is another exemplary die image in the embodiment of the present application;

[0029] Figure 7 This is a schematic diagram of the structure of a computer device in an embodiment of the present application;

[0030] Figure 8 This is another structural diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The embodiments of the present application provide an image alignment method, a computer device, and a computer storage medium for determining accurate alignment coordinates of a die image to improve the alignment accuracy of the die image.

[0032] The following describes the image alignment method in the embodiment of the present application:

[0033] See also Figure 2 In the embodiment of the present application, an image alignment method includes:

[0034] 201. Obtain a target die image to be processed, divide a target area in the target die image into a plurality of sub-areas, and perform steps 202 to 204 for each sub-area having a periodic pattern;

[0035] The method of this embodiment can be applied to a computer device, which can be a terminal or a server, or other devices with data processing and computing capabilities. A camera device can be used to image the die in the wafer to obtain a die image. There are both periodic patterns and non-periodic patterns in the die image. The periodic pattern is a pattern formed by crisscrossing shapes of different grayscales, and the non-periodic pattern is a pattern with uniform grayscale. Figure 1 As shown, the overall grayscale of the left pattern is uniform, which is a non-periodic pattern; the right pattern has lines of different grayscales interlaced in the horizontal direction, which is a periodic pattern.

[0036] When the camera images the die, it cannot control the motion accuracy of the image capture and the periodic pattern has pattern repeatability, which leads to misalignment between the template image and the die image during die defect detection. In addition, due to the presence of non-periodic patterns, image alignment deviation occurs, resulting in more false detections in the non-periodic pattern area. Therefore, this embodiment processes the die image to determine its accurate alignment pixel points, and the target die image is the die image for which accurate alignment pixel points need to be determined.

[0037] After the target die image is obtained, the target area in the target die image is determined by a preset size interval between the edge of the target die image and the target die image, that is, the edge of the target area is spaced from the edge of the target die image by a preset size. Figure 3 As shown, after obtaining the target die image, a target area is demarcated along the edge of the target die image and separated from the edge by a preset size, and the edge of the target area is separated from the edge of the target die image by the preset size. The preset size can be any size in pixels, such as the edge of the target area is separated from the edge of the target die image by 50 pixels, or by another number of pixels.

[0038] The target area is used to determine the accurate alignment pixel points of the target die image. The target area can be divided into multiple sub-areas, and the alignment pixel points of each sub-area are determined according to steps 202 to 204. Figure 4 As shown, Figure 3 The target area shown is divided into 5×5 sub-areas, and alignment pixels are determined for each sub-area.

[0039] 202. Use the sub-region as a template image, and expand each side of the sub-region by the preset size to obtain an image as a registration image;

[0040] For example, based on Figure 4 For the sub-region (2, 0), each side of this sub-region is expanded by the above preset size based on the target die image to form a new image, as shown below: Figure 5 The whole image shown in FIG. 1 is used as the alignment image of the (2,0) sub-region image. Since the edge of the target region is separated from the edge of the target die image by a preset size, and the upper edge of the (2,0) sub-region is the upper edge of the target region, the alignment image (i.e. Figure 5 The upper edge of the entire image is coincident with the upper edge of the target die image.

[0041] 203. Perform template matching on the template image and the alignment image according to a template matching algorithm to obtain a plurality of alignment pixel points and a correlation coefficient corresponding to each of the alignment pixel points;

[0042] In this embodiment, the template matching algorithm may be any template matching algorithm, for example, it may be a template matching algorithm such as OpenCV, scikit-learn, etc. When performing template matching between a template image and its alignment image based on the template matching algorithm, the alignment pixel points based on which the alignment image and the template image are aligned may be determined, and a correlation coefficient between the two may be calculated based on the alignment pixel points. The correlation coefficient is used to characterize the similarity between the alignment area of ​​the alignment image and the template image when the template image and its alignment image are aligned based on the alignment pixel points.

[0043] Among them, since the size of the alignment image is larger than the template image, the alignment area of ​​the alignment image is the area where multiple pixels in the alignment image that are aligned with the pixels of the template image are located when the template image and the alignment image are aligned based on the alignment pixel points. Since the template image and its alignment image are both composed of pixels, that is, both are pixel matrices, calculating the correlation coefficient is to calculate the correlation coefficient between the pixel matrix corresponding to the template image and the pixel matrix corresponding to the alignment area of ​​the alignment image.

[0044] For example, the pixel corresponding to the upper left vertex of the template image can be used as the reference point, and the pixel corresponding to the upper left vertex of the alignment image can be used as the starting point. The upper left vertex pixel of the template image and the upper left vertex pixel of the alignment image can be aligned and the correlation coefficient can be calculated. Then, the template image can be moved horizontally and vertically to align its upper left vertex pixel with other pixels of the alignment image in sequence and the correlation coefficient can be calculated. The alignment is completed when the lower right vertex pixel of the template image and the lower right vertex pixel of the alignment image are aligned in a certain alignment. Figure 5 In the example shown, assume that the edges of the (2, 0) sub-region are expanded by 50 pixels to obtain Figure 5 The final alignment pixel range is a rectangular range with the upper left vertex pixel of the alignment image as the vertex and a side length of 101×101, that is, Figure 5 The alignment pixel range shown is 101×101 alignment pixel points.

[0045] The correlation coefficient may be calculated by calculating the mean value of the pixel values ​​of all pixels in the alignment area of ​​the alignment image that is aligned with the template image, recorded as imgMean, and subtracting the pixel value of each pixel from imgMean to obtain a pixel difference matrix, recorded as imgDiff; calculating the mean value of the pixel values ​​of all pixels in the template image, recorded as tempMean, and subtracting the pixel value of each pixel from tempMean to obtain a pixel difference matrix, recorded as tempDiff. Calculate the correlation between imgDiff and tempDiff to obtain the correlation coefficient corresponding to the current alignment pixel.

[0046] Of course, other algorithms regarding the correlation between matrices may also be used to calculate the correlation coefficients corresponding to the aligned pixel points, and this embodiment does not limit this.

[0047] 204. Determine the coordinates of the target alignment pixel point in the sub-area according to the correlation coefficients respectively corresponding to the plurality of alignment pixel points and the grayscale value of each alignment pixel point;

[0048] The computer device may establish a rectangular coordinate system in advance and place the target die image in the rectangular coordinate system to determine the coordinates of each pixel point.

[0049] After obtaining multiple alignment pixels and the correlation coefficient corresponding to each alignment pixel, the coordinates of the target alignment pixel in the sub-region can be determined according to the correlation coefficients corresponding to the multiple alignment pixels and the grayscale values ​​of each alignment pixel. Figures 3 to 5After obtaining multiple alignment pixels in the (2, 0) sub-region and the grayscale value of each alignment pixel, the coordinates of the target alignment pixel in the (2, 0) sub-region can be determined according to the correlation coefficients corresponding to the multiple alignment pixels in the (2, 0) sub-region and the grayscale values ​​of each alignment pixel.

[0050] Therefore, by executing steps 202 to 204 for each sub-region respectively, the coordinates of the target alignment pixel points of each sub-region can be determined.

[0051] 205. When the coordinates of the target alignment pixels of the multiple sub-regions are obtained, determine the alignment coordinates of the target die image according to the coordinates of the target alignment pixels of the multiple sub-regions;

[0052] When determining the coordinates of the target alignment pixel points of each sub-region, the alignment coordinates of the target die image may be further determined according to the coordinates of the target alignment pixel points of the plurality of sub-regions of the target die image.

[0053] During defect detection of the target die image, the target die image and the reference image used for auxiliary detection can be aligned based on the alignment coordinates of the target die image. Since the alignment coordinates of the target die image are calculated based on the pixel distribution and periodic changes of each sub-region of the target die image, the target die image and the reference image can be accurately aligned based on the alignment coordinates, which can avoid the influence of the periodic pattern of the die image on the image alignment, thereby making the die image alignment more accurate during die defect detection and ensuring the accuracy of die defect detection.

[0054] based on Figure 2 In the embodiment shown, in a preferred implementation manner, when determining the coordinates of the target alignment pixel point of the sub-region in step 204, it is possible to determine whether there are periodic patterns in the x-axis direction and the y-axis direction of the alignment image according to the correlation coefficients respectively corresponding to the multiple alignment pixel points of the sub-region. If there is a periodic pattern in the x-axis direction of the alignment image but not in the y-axis direction, the target x-coordinate is determined according to the grayscale value of the alignment pixel point in any row of the alignment pixel point matrix formed by the multiple alignment pixel points of the sub-region, and the preset value is used as the target y-coordinate, and the target x-coordinate and the target y-coordinate constitute the coordinates of the target alignment pixel point of the sub-region.

[0055] If there is no periodic pattern in the x-axis direction of the alignment image but there is a periodic pattern in the y-axis direction, the target y coordinate is determined according to the grayscale value of the alignment pixel point in any column of the alignment pixel point matrix, and the preset value is used as the target x coordinate. The target x coordinate and the target y coordinate constitute the coordinates of the target alignment pixel point in the sub-area.

[0056] For example, using Figures 3 to 5 The example shown, Figure 5 The alignment pixel range shown is an alignment pixel matrix composed of multiple alignment pixel points in the (2, 0) sub-region. The target alignment pixel point of the sub-region can be determined based on the grayscale value of each pixel in the alignment pixel matrix.

[0057] The preset value may be any value as long as it is used to indicate that the preset value does not participate in the subsequent calculation process of the alignment coordinates of the target die image. For example, the preset value may be set to -1, indicating that this value does not participate in the subsequent calculation process of the alignment coordinates of the target die image.

[0058] Therefore, the target alignment pixel points of each sub-area can be determined in this way. The target alignment pixel points of the sub-area are calculated based on the pixel distribution and periodic change of the sub-area. Therefore, the target alignment pixel points of the sub-area can be used as the accurate alignment pixel points of the sub-area during image alignment, providing an accurate calculation basis for determining the alignment coordinates of the subsequent target die image.

[0059] When determining the target x coordinate according to the grayscale value of the aligned pixel point in any row of the aligned pixel point matrix, the x coordinate of the pixel point corresponding to the grayscale value extreme value of any row in the aligned pixel point matrix can be determined as the target x coordinate.

[0060] Continue to use Figures 3 to 5 In the example shown, the extreme value of the row y=50 in the alignment pixel matrix can be taken. The determination method can be to obtain the grayscale values ​​of all pixels in the current row. The width of the alignment pixel matrix is ​​101, that is, there are 101 grayscale values. Calculation starts from the third pixel. If the current pixel value is greater than the average of the first two pixel values ​​and the average of the last two pixel values, the current point is an extreme point. Calculate to the 99th pixel, a total of 97 values. Assuming that m extreme points are obtained, take the x coordinate of the extreme point closest to x=50 among these m extreme points as the target x coordinate.

[0061] Of course, in addition to using the x coordinate of the pixel point with the extreme gray value as the target x coordinate, the x coordinate of the pixel point corresponding to the median gray value in the current row may also be used as the target x coordinate, which is not limited in this embodiment.

[0062] Similarly, when determining the target y coordinate according to the grayscale value of the alignment pixel point in any column of the alignment pixel point matrix, the y coordinate of the pixel point corresponding to the grayscale value extreme value of any column in the alignment pixel point matrix can be determined as the target y coordinate.

[0063] Continue to use Figures 3 to 5In the example shown, the extreme value of the column x=50 in the alignment pixel matrix can be taken. The determination method can be to obtain the grayscale values ​​of all pixels in the current column. The length of the alignment pixel matrix is ​​101, that is, there are 101 grayscale values. Calculation starts from the third pixel. If the current pixel value is greater than the average of the first two pixel values ​​and the average of the last two pixel values, the current point is an extreme point. Calculate to the 99th pixel, a total of 97 values. Assuming that m extreme points are obtained, take the y coordinate of the extreme point closest to y=50 among these m extreme points as the target y coordinate.

[0064] Therefore, the coordinates of the target alignment pixel points of each sub-area can be quickly calculated and determined by the above method, which further facilitates the calculation process of the subsequent alignment coordinates of the target die image.

[0065] In this embodiment, a preferred implementation method for determining whether there are periodic patterns in the x-axis direction and the y-axis direction of the alignment image may be to calculate the deviation degree coefficient corresponding to the correlation coefficient of each column in the alignment pixel matrix. If the deviation degree coefficient of each column is less than a first preset threshold, it is determined that there is a periodic pattern in the x-axis direction of the alignment image, and it is determined that there is no periodic pattern in the y-axis direction of the alignment image. Calculate the deviation degree coefficient corresponding to the correlation coefficient of each row in the alignment pixel matrix. If the deviation degree coefficient of each row is less than the first preset threshold, it is determined that there is a periodic pattern in the y-axis direction of the alignment image, and it is determined that there is no periodic pattern in the x-axis direction of the alignment image.

[0066] For example, projection is performed along the y direction of the alignment pixel matrix, and the mean of the correlation coefficients corresponding to each column of alignment pixels is calculated. The mean is subtracted from the correlation coefficients corresponding to all alignment pixels in the column and the absolute value is obtained. All the results after subtraction are accumulated, and the accumulated value is the deviation degree coefficient of the column. If the accumulated value is less than a preset threshold (such as 0.01), it is considered that there is a periodic change in the current x direction, and it is considered that there is no periodic change in the current y direction.

[0067] Of course, the above-mentioned accumulated value may be divided by the number of aligned pixels in this column, and the obtained quotient (i.e., average deviation) may be compared with the preset threshold; or the sum of squares of the differences between the correlation coefficients corresponding to all aligned pixels in the column and the above-mentioned mean of the correlation coefficients may be calculated, and this sum of squares may be used as the deviation degree coefficient of the column, and then compared with the preset threshold. This embodiment does not limit the calculation method of the deviation degree coefficient corresponding to the correlation coefficient of each column or each row, as long as the deviation degree coefficient can characterize the deviation level of multiple correlation coefficients of the row or column.

[0068] Therefore, this method can be used to quickly determine whether there is a periodic pattern in each mode of the alignment image, so as to facilitate the subsequent rapid calculation of the coordinates of the target alignment pixel points in the sub-area according to the determination result.

[0069] based on Figure 2 In the embodiment shown, in a preferred implementation manner, before step 202, the discrete degree coefficients of the grayscale values ​​of all pixels in each sub-region may be calculated respectively. If the discrete degree coefficient is less than the second preset threshold, the sub-region is determined to be a non-periodic pattern, and the sub-region with only the non-periodic pattern does not need to determine the corresponding target alignment pixel point, that is, the sub-region has no target alignment pixel point; if the discrete degree coefficient is greater than the second preset threshold, it indicates that the sub-region contains a periodic pattern, and step 202 and subsequent steps may be executed to determine the target alignment pixel point of the sub-region.

[0070] by Figure 4 For example, for the (0, 0) sub-region, the grayscale value mean of the sub-region image can be calculated, recorded as img_mean; the maximum and minimum grayscale values ​​of the sub-region image are determined, recorded as img_min and img_max; the discrete degree coefficient val is calculated, val = (img_max-img_min) / img_mean; if val>1.5 (i.e. ±2db), then the sub-region needs to calculate the corresponding target alignment pixel point, that is, execute step 202 and subsequent steps; otherwise, it is not necessary, indicating that there is no target alignment pixel point in the sub-region, and start calculating the next sub-region.

[0071] Therefore, when determining the alignment coordinates of the target die image according to the coordinates of the target alignment pixels of the multiple sub-regions in step 205, if the number of the target alignment pixels of the multiple sub-regions of the target die image is zero, it means that all the sub-regions of the target die image are flat regions, that is, there are no periodic patterns. At this time, the alignment coordinates of the target die image can be determined as the origin coordinates of the coordinate system, that is, (0, 0). Figure 4 For example, if there are no target alignment pixels in all sub-areas in the target die image, the alignment coordinates of the target die image are (0, 0).

[0072] If the number of target alignment pixels in multiple sub-areas of the target die image is greater than zero and less than a preset number, the average of the accumulated values ​​of the x-coordinates of the target alignment pixels in the multiple sub-areas is used as the x-coordinate of the alignment coordinates of the target die image, and the average of the accumulated values ​​of the y-coordinates of the target alignment pixels in the multiple sub-areas is used as the y-coordinate of the alignment coordinates of the target die image.

[0073] For example, if the number of target alignment pixels in multiple sub-regions of the target die image is 1-3, it means that there are currently 1-3 sub-regions with periodic patterns. At this time, the x-coordinates and y-coordinates of all target alignment pixels can be accumulated and averaged. Among them, if the value of the x-coordinate or y-coordinate of the target alignment pixel is the above-mentioned preset value, such as -1, the value of the x-coordinate or y-coordinate does not participate in the accumulation of the coordinate values.

[0074] If the number of target alignment pixels in multiple sub-regions of the target die image is greater than the preset number, then in the array composed of all x-coordinates of the target alignment pixels in the multiple sub-regions of the target die image, the distance set of each value is calculated, the distance set is the set of differences between the value and all values, and all elements in the set are sorted by size, and the element in the distance set ranked in the preset order is used as the target distance value of the value, and the x-coordinate corresponding to the minimum target distance value is used as the x-coordinate of the alignment coordinate of the target die image. In addition, in the array composed of all y-coordinates of the target alignment pixels in the multiple sub-regions of the target die image, the distance set of each value is calculated, and the element in the distance set ranked in the preset order is used as the target distance value of the value, and the y-coordinate corresponding to the minimum target distance value is used as the y-coordinate of the alignment coordinate of the target die image.

[0075] For example, the coordinates of the target aligned pixels of multiple sub-regions are (2, -1), (2, 8), (1, 6), (-1, 7), (2, 8) and (2, 4). For the x direction, the array of all x coordinates is [2, 2, 1, -1, 2, 2], where negative values ​​are not included in the calculation. The difference between each value in the array and all values ​​is calculated to obtain the distance set of each value. For example, the multiple elements in the distance set of the first value "2" in the array are {0, 0, 1, empty, 0, 0}. They are sorted from large to small (or from small to large), and the distance set obtained is {1, 0, 0, 0, 0}. The third smallest element (i.e. "0") is selected as the target distance value of the first value "2" in the array, recorded as dist0. Similarly, the target distance value of each value in the array can be calculated, recorded as dist1, dist2, empty, dist4 and dist5. Take the minimum value among dist0, dist1, dist2, dist4 and dist5, and use the x coordinate corresponding to the minimum value as the x coordinate of the alignment coordinate of the target die image.

[0076] Similarly, for the y direction, the array of all y coordinates is [-1, 8, 6, 7, 8, 4], where negative values ​​are not included in the calculation. Calculate the difference between each value in the array and all values ​​to obtain the distance set of each value. For example, the distance set of the second value "8" in the array has multiple elements {empty, 0, 2, 1, 0, 4}. Sort them from large to small (or from small to large), and the distance set obtained is {4, 2, 1, 0, 0}. Select the third smallest element (i.e. "1") as the target distance value of the second value "8" in the array, recorded as dist1. Similarly, the target distance value of each value in the array can be calculated, recorded as dist2, dist3, dist4 and dist5 respectively. Take the minimum value among dist1, dist2, dist3, dist4 and dist5, and use the y coordinate corresponding to the minimum value as the y coordinate of the alignment coordinate of the target die image.

[0077] Finally, through the above calculation operation, it can be determined that the alignment coordinates of the target die image are (2, 8).

[0078] It should be noted that if the x-coordinates or y-coordinates of the coordinates of the multiple target alignment pixels in the multiple sub-areas are all the preset values, for example, all -1, it means that there is no valid information in this direction that can be used to determine the alignment coordinates, that is, a pure cycle. In this case, the corresponding x-value or y-value can be set to 0, and no alignment is required in this direction. Figure 6 As shown, the die image has a repetitive period in the x direction and no alignment is required in the y direction.

[0079] Therefore, the x-coordinate and y-coordinate of the alignment coordinates of the target die image can be accurately calculated by the above method, reducing the image alignment deviation caused by the periodic change of the die image pattern on the die image alignment, thereby improving the accuracy of the die image alignment, and further improving the accuracy of die image defect detection.

[0080] The above describes the image alignment method in the embodiment of the present application. The following describes the computer device in the embodiment of the present application. Figure 7 In the embodiment of the present application, one embodiment of the computer device includes:

[0081] An acquisition unit 701 is used to acquire a target die image to be processed, and divide a target area in the target die image into a plurality of sub-areas, wherein an edge of the target area is spaced from an edge of the target die image by a preset size;

[0082] The calculation unit 702 is configured to perform the following steps for each sub-region having a periodic pattern:

[0083] The sub-region is used as a template image, and each side of the sub-region is expanded by the preset size to obtain an image as a registration image;

[0084] Performing template matching on the template image and the alignment image according to a template matching algorithm to obtain a plurality of alignment pixels and a correlation coefficient corresponding to each of the alignment pixels, wherein the correlation coefficient is used to characterize the similarity between the alignment region of the alignment image and the template image when the alignment image and the template image are aligned based on the alignment pixels;

[0085] Determine the coordinates of the target alignment pixel point of the sub-area according to the correlation coefficients respectively corresponding to the plurality of alignment pixel points and the grayscale value of each alignment pixel point;

[0086] The determination unit 703 is configured to determine the alignment coordinates of the target die image according to the coordinates of the target alignment pixel points of the multiple sub-regions when obtaining the coordinates of the target alignment pixel points of the multiple sub-regions.

[0087] In a preferred implementation of this embodiment, the calculation unit 702 is specifically used for:

[0088] Determining whether there is a periodic pattern in the x-axis direction and the y-axis direction of the alignment image according to the correlation coefficients respectively corresponding to the plurality of alignment pixel points;

[0089] If there is a periodic pattern in the x-axis direction of the alignment image but no periodic pattern in the y-axis direction, a target x-coordinate is determined according to the grayscale value of an alignment pixel point in any row of the alignment pixel point matrix formed by the plurality of alignment pixel points, and a preset value is used as a target y-coordinate, and the target x-coordinate and the target y-coordinate constitute the coordinates of the target alignment pixel point of the sub-area;

[0090] If there is no periodic pattern in the x-axis direction of the alignment image but there is a periodic pattern in the y-axis direction, the target y coordinate is determined according to the grayscale value of the alignment pixel point in any column of the alignment pixel point matrix, and the preset value is used as the target x coordinate. The target x coordinate and the target y coordinate constitute the coordinates of the target alignment pixel point of the sub-area.

[0091] In a preferred implementation of this embodiment, the calculation unit 702 is specifically used for:

[0092] In any row of the aligned pixel matrix, determine the x coordinate of the pixel point corresponding to the grayscale value extreme value of the row as the target x coordinate;

[0093] The determining the target y coordinate according to the grayscale value of the aligned pixel point in any column of the aligned pixel point matrix includes:

[0094] In any column of the aligned pixel matrix, the y coordinate of the pixel point corresponding to the grayscale value extreme value of the column is determined as the target y coordinate.

[0095] In a preferred implementation of this embodiment, the calculation unit 702 is specifically used for:

[0096] Calculating the deviation degree coefficient corresponding to the correlation coefficient of each column in the alignment pixel matrix, and if the deviation degree coefficient of each column is less than a first preset threshold, determining that there is a periodic pattern in the x-axis direction of the alignment image, and determining that there is no periodic pattern in the y-axis direction of the alignment image;

[0097] Calculate the deviation degree coefficient corresponding to the correlation coefficient of each row in the alignment pixel point matrix; if the deviation degree coefficient of each row is less than the first preset threshold, determine that there is a periodic pattern in the y-axis direction of the alignment image, and determine that there is no periodic pattern in the x-axis direction of the alignment image.

[0098] In a preferred implementation of this embodiment, the determining unit 703 is specifically configured to:

[0099] If the number of target alignment pixels in the multiple sub-areas is zero, determining the alignment coordinates of the target die image as the origin coordinates;

[0100] If the number of target alignment pixel points in the multiple sub-areas is greater than zero and less than a preset number, the average of the accumulated values ​​of the x-coordinates of the target alignment pixel points in the multiple sub-areas is used as the x-coordinate of the alignment coordinates of the target die image, and the average of the accumulated values ​​of the y-coordinates of the target alignment pixel points in the multiple sub-areas is used as the y-coordinate of the alignment coordinates of the target die image.

[0101] In a preferred implementation of this embodiment, if the number of target alignment pixels in the multiple sub-areas is greater than the preset number, the determining unit 703 is further configured to:

[0102] In an array composed of all x-coordinates of target alignment pixels of the multiple sub-areas, a distance set of each value is calculated, the distance set being a set of differences between the value and all values, and all elements in the set are sorted by size, and the element in the distance set ranked in a preset order is used as the target distance value of the value, and the x-coordinate corresponding to the minimum target distance value is used as the x-coordinate of the alignment coordinate of the target die image; and,

[0103] In the array composed of all y coordinates of the target alignment pixel points of the multiple sub-areas, the distance set of each value is calculated, and the element ranked in a preset order in the distance set is used as the target distance value of the value, and the y coordinate corresponding to the minimum target distance value is used as the y coordinate of the alignment coordinate of the target die image.

[0104] In a preferred implementation of this embodiment, before taking the sub-region as a template image and expanding each edge of the sub-region by the preset size to obtain an image as the alignment image, the calculation unit 702 is further configured to:

[0105] Calculate the discrete degree coefficient of the grayscale values ​​of all pixels in the sub-region;

[0106] If the dispersion coefficient is less than a second preset threshold, determining that the sub-region is a non-periodic pattern;

[0107] If the discrete degree coefficient is greater than the second preset threshold, the sub-region is used as a template image, and each side of the sub-region is expanded by the preset size to obtain an image as a registration image.

[0108] In this embodiment, the operations performed by each unit in the computer device are the same as those described above. Figure 2 The description in the illustrated embodiment is similar and will not be repeated here.

[0109] During defect detection of the target die image, the target die image and the reference image used for auxiliary detection can be aligned based on the alignment coordinates of the target die image. Since the alignment coordinates of the target die image are calculated based on the pixel distribution and periodic changes of each sub-region of the target die image, the target die image and the reference image can be accurately aligned based on the alignment coordinates, which can avoid the influence of the periodic pattern of the die image on the image alignment, thereby making the die image alignment more accurate during die defect detection and ensuring the accuracy of die defect detection.

[0110] The computer device in the embodiment of the present application is described below. Figure 8 In the embodiment of the present application, one embodiment of the computer device includes:

[0111] The computer device 800 may include one or more central processing units (CPU) 801 and a memory 805 , wherein the memory 805 stores one or more application programs or data.

[0112] The memory 805 may be a volatile storage or a persistent storage. The program stored in the memory 805 may include one or more modules, each of which may include a series of instruction operations in the computer device. Furthermore, the central processing unit 801 may be configured to communicate with the memory 805 and execute a series of instruction operations in the memory 805 on the computer device 800.

[0113] The computer device 800 may also include one or more power supplies 802, one or more wired or wireless network interfaces 803, one or more input and output interfaces 804, and / or one or more operating systems, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0114] The CPU 801 can execute the aforementioned Figure 2 The operations performed by the computer device in the illustrated embodiment will not be described in detail here.

[0115] The present application also provides a computer storage medium, wherein one embodiment includes: the computer storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the aforementioned Figure 2 The operations performed by the computer device in the illustrated embodiment.

[0116] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0117] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0120] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. An image alignment method, It is characterized in that The method comprises: Acquire a target die image to be processed, divide a target area in the target die image into a plurality of sub-areas, and separate an edge of the target area from an edge of the target die image by a preset size; The following steps are performed for each sub-region where a periodic pattern exists: The sub-region is used as a template image, and each side of the sub-region is expanded by the preset size to obtain an image as a registration image; Performing template matching on the template image and the alignment image according to a template matching algorithm to obtain a plurality of alignment pixels and a correlation coefficient corresponding to each of the alignment pixels, wherein the correlation coefficient is used to characterize the similarity between the alignment region of the alignment image and the template image when the alignment image and the template image are aligned based on the alignment pixels; Determine the coordinates of the target alignment pixel point of the sub-area according to the correlation coefficients respectively corresponding to the plurality of alignment pixel points and the grayscale value of each alignment pixel point; When the coordinates of the target alignment pixels of the multiple sub-areas are obtained, the alignment coordinates of the target die image are determined according to the coordinates of the target alignment pixels of the multiple sub-areas.

2. The method according to claim 1, It is characterized in that The determining the coordinates of the target alignment pixel point of the sub-area according to the correlation coefficients respectively corresponding to the plurality of alignment pixel points and the grayscale value of each alignment pixel point comprises: Determining whether there is a periodic pattern in the x-axis direction and the y-axis direction of the alignment image according to the correlation coefficients respectively corresponding to the plurality of alignment pixel points; If there is a periodic pattern in the x-axis direction of the alignment image but no periodic pattern in the y-axis direction, a target x-coordinate is determined according to the grayscale value of an alignment pixel point in any row of the alignment pixel point matrix formed by the plurality of alignment pixel points, and a preset value is used as a target y-coordinate, and the target x-coordinate and the target y-coordinate constitute the coordinates of the target alignment pixel point of the sub-area; If there is no periodic pattern in the x-axis direction of the alignment image but there is a periodic pattern in the y-axis direction, the target y coordinate is determined according to the grayscale value of the alignment pixel point in any column of the alignment pixel point matrix, and the preset value is used as the target x coordinate. The target x coordinate and the target y coordinate constitute the coordinates of the target alignment pixel point of the sub-area.

3. The method according to claim 2, It is characterized in that The step of determining the target x-coordinate according to the grayscale values ​​of the alignment pixels in any row of the alignment pixel matrix formed by the plurality of alignment pixels comprises: In any row of the aligned pixel matrix, determine the x coordinate of the pixel point corresponding to the grayscale value extreme value of the row as the target x coordinate; The determining the target y coordinate according to the grayscale value of the aligned pixel point in any column of the aligned pixel point matrix includes: In any column of the aligned pixel matrix, the y coordinate of the pixel point corresponding to the grayscale value extreme value of the column is determined as the target y coordinate.

4. The method according to claim 2, It is characterized in that The determining whether there is a periodic pattern in the x-axis direction and the y-axis direction of the alignment image according to the correlation coefficients respectively corresponding to the plurality of alignment pixel points includes: Calculating the deviation degree coefficient corresponding to the correlation coefficient of each column in the alignment pixel matrix, and if the deviation degree coefficient of each column is less than a first preset threshold, determining that there is a periodic pattern in the x-axis direction of the alignment image, and determining that there is no periodic pattern in the y-axis direction of the alignment image; Calculate the deviation degree coefficient corresponding to the correlation coefficient of each row in the alignment pixel point matrix; if the deviation degree coefficient of each row is less than the first preset threshold, determine that there is a periodic pattern in the y-axis direction of the alignment image, and determine that there is no periodic pattern in the x-axis direction of the alignment image.

5. The method according to claim 1, It is characterized in that The determining the alignment coordinates of the target die image according to the coordinates of the target alignment pixel points of the multiple sub-areas includes: If the number of target alignment pixels in the multiple sub-areas is zero, determining the alignment coordinates of the target die image as the origin coordinates; If the number of target alignment pixel points in the multiple sub-areas is greater than zero and less than a preset number, the average of the accumulated values ​​of the x-coordinates of the target alignment pixel points in the multiple sub-areas is used as the x-coordinate of the alignment coordinates of the target die image, and the average of the accumulated values ​​of the y-coordinates of the target alignment pixel points in the multiple sub-areas is used as the y-coordinate of the alignment coordinates of the target die image.

6. The method according to claim 5, It is characterized in that If the number of target alignment pixels in the plurality of sub-areas is greater than the preset number, the method further includes: In an array composed of all x-coordinates of target alignment pixels of the multiple sub-areas, a distance set of each value is calculated, the distance set being a set of differences between the value and all values, and all elements in the set are sorted by size, and the element in the distance set ranked in a preset order is used as the target distance value of the value, and the x-coordinate corresponding to the minimum target distance value is used as the x-coordinate of the alignment coordinate of the target die image; and, In the array composed of all y coordinates of the target alignment pixel points of the multiple sub-areas, the distance set of each value is calculated, and the element ranked in a preset order in the distance set is used as the target distance value of the value, and the y coordinate corresponding to the minimum target distance value is used as the y coordinate of the alignment coordinate of the target die image.

7. The method according to claim 1, It is characterized in that Before taking the sub-region as a template image and expanding each side of the sub-region by the preset size to obtain an image as a registration image, the method further includes: Calculate the discrete degree coefficient of the grayscale values ​​of all pixels in the sub-region; If the dispersion coefficient is less than a second preset threshold, determining that the sub-region is a non-periodic pattern; If the discrete degree coefficient is greater than the second preset threshold, the sub-region is used as a template image, and each side of the sub-region is expanded by the preset size to obtain an image as a registration image.

8. A computer device, It is characterized in that The computer device comprises: an acquisition unit, configured to acquire a target die image to be processed, and divide a target area in the target die image into a plurality of sub-areas, wherein an edge of the target area is spaced from an edge of the target die image by a preset size; A computing unit, configured to perform the following steps for each sub-region having a periodic pattern: The sub-region is used as a template image, and each side of the sub-region is expanded by the preset size to obtain an image as a registration image; Performing template matching on the template image and the alignment image according to a template matching algorithm to obtain a plurality of alignment pixels and a correlation coefficient corresponding to each of the alignment pixels, wherein the correlation coefficient is used to characterize the similarity between the alignment region of the alignment image and the template image when the alignment image and the template image are aligned based on the alignment pixels; Determine the coordinates of the target alignment pixel point of the sub-area according to the correlation coefficients respectively corresponding to the plurality of alignment pixel points and the grayscale value of each alignment pixel point; A determination unit is used to determine the alignment coordinates of the target die image according to the coordinates of the target alignment pixel points of the multiple sub-areas when obtaining the coordinates of the target alignment pixel points of the multiple sub-areas.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer storage medium, It is characterized in that The computer storage medium stores instructions, and when the instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 7.

Images