Image alignment method, computer device and computer storage medium

Abstract

Embodiments of the present application disclose an image alignment method, a computer device and a computer storage medium. A sub-region is taken as a template image, and an image obtained by extending each side of the sub-region by a preset size is taken as an alignment image. The template image and the alignment image are matched according to a template matching algorithm, a plurality of alignment pixel points and a correlation coefficient corresponding to each alignment pixel point are obtained, and the coordinates of a target alignment pixel point of the sub-region are determined according to the correlation coefficient of the plurality of alignment pixel points and the gray value of each alignment pixel point. The alignment coordinates of a target die image are determined according to the coordinates of the target alignment pixel points of a plurality of sub-regions. Since the alignment coordinates are calculated based on the pixel distribution and periodic variation of each sub-region of the target die image, the target die image and the reference image can be accurately aligned based on the alignment coordinates, the influence of the periodic pattern of the die image on image alignment can be avoided, and thus the alignment accuracy of the die image is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of die image processing, and in particular to an image alignment method, a computer device and a computer storage medium. BACKGROUND

[0002] When performing defect detection on a die, a camera images the die to obtain a die image. The die image contains both a periodic pattern and a non-periodic pattern. The periodic pattern is a pattern formed by shapes of different gray levels intersecting vertically and horizontally, and the non-periodic pattern is a pattern of uniform gray level. As shown in FIG. 1, the left pattern as a whole has a uniform gray level, which is a non-periodic pattern; the right pattern has lines of different gray levels intersecting in the horizontal direction, which is a periodic pattern. Figure 1 When the camera images the die, the motion accuracy of image shooting cannot be controlled, and the periodic pattern has pattern repetitiveness, which causes the template image and the die image to be misaligned when performing defect detection on the die, resulting in misalignment. Moreover, due to the existence of the non-periodic pattern, the image alignment deviates, thereby causing more false detections in the non-periodic pattern region.

[0003] When the camera images the die, the motion accuracy of image shooting cannot be controlled, and the periodic pattern has pattern repetitiveness, which causes the template image and the die image to be misaligned when performing defect detection on the die, resulting in misalignment. Moreover, due to the existence of the non-periodic pattern, the image alignment deviates, thereby causing more false detections in the non-periodic pattern region. SUMMARY

[0004] Embodiments of the present application provide an image alignment method, a computer device and a computer storage medium, which are used to determine accurate alignment coordinates of a die image, so as to improve the alignment accuracy of the die image.

[0005] In a first aspect, an image alignment method is provided, and the method comprises the following steps:

[0006] obtaining a target die image to be processed, dividing a target region in the target die image into a plurality of sub-regions, and spacing the edges of the target region from the edges of the target die image by a preset size;

[0007] respectively performing the following steps on each of the sub-regions having a periodic pattern:

[0008] taking the sub-region as a template image, and taking an image obtained by outwardly expanding each edge of the sub-region by the preset size as an alignment image;

[0009] performing template matching on the template image and the alignment image according to a template matching algorithm, to obtain a plurality of alignment pixel points and a correlation coefficient corresponding to each of the alignment pixel points, the correlation coefficient being used to represent the similarity between an alignment region of the alignment image and the template image when the alignment image is aligned based on the alignment pixel points;

[0010] determine the coordinates of the target alignment pixel points of the sub-regions according to the correlation coefficients corresponding to the target alignment pixel points of the sub-regions and the gray values of the target alignment pixel points of the sub-regions.

[0011] When the coordinates of the target alignment pixel points of the sub-regions are obtained, the alignment coordinates of the target die image are determined according to the coordinates of the target alignment pixel points of the sub-regions.

[0012] The second aspect of the embodiment of the present application provides a computer device, which comprises:

[0013] The acquisition unit is configured to acquire a target die image to be processed, divide a target region in the target die image into a plurality of sub-regions, and set a distance between an edge of the target region and an edge of the target die image as a preset size.

[0014] The calculation unit is configured to perform the following steps on each of the sub-regions in which the periodic pattern exists:

[0015] The sub-region is taken as a template image, and an image obtained by extending each edge of the sub-region by the preset size is taken as an alignment image.

[0016] The template image and the alignment image are matched according to a template matching algorithm, a plurality of alignment pixel points and a correlation coefficient corresponding to each of the alignment pixel points are obtained, and the correlation coefficient is used to represent the similarity between an alignment region of the alignment image and the template image when the alignment image is aligned with the template image based on the alignment pixel points.

[0017] The coordinates of the target alignment pixel points of the sub-regions are determined according to the correlation coefficients corresponding to the target alignment pixel points of the sub-regions and the gray values of the target alignment pixel points of the sub-regions.

[0018] The determination unit is configured to, when the coordinates of the target alignment pixel points of the sub-regions are obtained, determine the alignment coordinates of the target die image according to the coordinates of the target alignment pixel points of the sub-regions.

[0019] The third aspect of the embodiment of the present application provides a computer device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0020] The fourth aspect of the embodiment of the present application provides a computer storage medium, and the computer storage medium stores instructions, which, when executed on a computer, cause the computer to execute the method of the first aspect.

[0021] As can be seen from the above technical solutions, the embodiment of the present application has the following advantages:

[0022] For each sub-region of the target die image containing a periodic pattern, the following steps are performed: The sub-region is used as a template image. Each edge of the sub-region is expanded by a preset size to obtain an image for alignment. The template image and the alignment image are matched using a template matching algorithm to obtain multiple alignment pixels and their corresponding correlation coefficients. The coordinates of the target alignment pixels in the sub-region are determined based on the correlation coefficients and grayscale values ​​of each alignment pixel. Finally, the alignment coordinates of the target die image are determined based on the coordinates of the target alignment pixels in the multiple sub-regions. Since the alignment coordinates of the target die image are calculated based on the pixel distribution and periodic changes of each sub-region of the target die image, the target die image and the reference image can be accurately aligned based on these alignment coordinates. This avoids the influence of the periodic pattern of the die image on image alignment, thus making die image alignment more accurate during die defect detection and ensuring the accuracy of die defect detection. Attached Figure Description

[0023] Figure 1 This is an exemplary die image in an embodiment of this application;

[0024] Figure 2 This is a flowchart illustrating the image alignment method in an embodiment of this application;

[0025] Figure 3 This is an exemplary target die image in an embodiment of this application;

[0026] Figure 4 for Figure 3 A schematic diagram illustrating one method of dividing a target die image into multiple sub-regions;

[0027] Figure 5 for Figure 4 A schematic diagram illustrating how the alignment image and template image are determined for one sub-region of the target die image shown.

[0028] Figure 6 This is another exemplary die image in the embodiments of this application;

[0029] Figure 7 This is a schematic diagram of the structure of a computer device in an embodiment of this application;

[0030] Figure 8 This is another schematic diagram of the structure of the computer device in the embodiments of this application. Detailed Implementation

[0031] The embodiment of the present application provides an image alignment method, a computer device and a computer storage medium, which are used for determining accurate alignment coordinates of a die image, so as to improve the alignment accuracy of the die image.

[0032] The image alignment method in the embodiment of the present application is described below.

[0033] Referring to Figure 2 An embodiment of the image alignment method in the embodiment of the present application comprises the following steps.

[0034] 201, obtaining a target die image to be processed, dividing a target region in the target die image into a plurality of sub-regions, and performing steps 202 to 204 on each sub-region with a periodic pattern.

[0035] The method of the embodiment can be applied to a computer device, which can be a terminal or a server, or other devices with data processing and computing capabilities. A camera device can be used to image a die in a wafer to obtain a die image. The die image simultaneously contains a periodic pattern and a non-periodic pattern. The periodic pattern is a pattern formed by shapes of different gray levels intersecting vertically and horizontally, and the non-periodic pattern is a pattern with uniform gray levels. As shown in the left side of FIG. 1, the overall gray level of the left side pattern is uniform, which is a non-periodic pattern. The right side pattern is a periodic pattern with lines of different gray levels intersecting in the horizontal direction. Figure 1

[0036] When a camera images a die, the motion accuracy of image shooting cannot be controlled, and the periodic pattern has pattern repetition, which causes alignment errors between a template image and a die image during die defect detection, resulting in misalignment. In addition, due to the existence of the non-periodic pattern, the image alignment deviates, thereby causing more false detections in the non-periodic pattern region. Therefore, the die image is processed to determine accurate alignment pixels thereof in the embodiment. The target die image is a die image for which accurate alignment pixels need to be determined.

[0037] After obtaining the target die image, a target region in the target die image is determined at a distance of a preset size from the edge of the target die image, that is, the edge of the target region is at a distance of the preset size from the edge of the target die image. As shown in FIG. 2, after obtaining the target die image, a target region is demarcated along the edge of the target die image at a distance of a preset size from the edge, and the edge of the target region is at a distance of the preset size from the edge of the target die image. The preset size can be any size in pixel units, such as a distance of 50 pixels or other pixel numbers. Figure 3

[0038] ​​The target region is used to determine the accurate alignment pixel points of the target die image. The target region can be divided into a plurality of sub-regions, and the alignment pixel points of each sub-region can be determined according to steps 202 to 204. Figure 4 As shown in FIG. 5, the target region shown in FIG. 4 can be divided into 5x5 sub-regions, and the alignment pixel points of each sub-region can be determined. Figure 3 As shown in FIG. 5, the target region shown in FIG. 4 can be divided into 5x5 sub-regions, and the alignment pixel points of each sub-region can be determined.

[0039] 202. The sub-region is taken as a template image, and an image obtained by outwardly expanding each edge of the sub-region by the preset size is taken as an alignment image;

[0040] For example, based on the target die image, each edge of the (2, 0) sub-region is outwardly expanded by the preset size, thereby forming a new image, and obtaining the whole image as shown in FIG. 6. Figure 4 As shown in FIG. 6, the new image is taken as the alignment image of the (2, 0) sub-region image. Since the edges of the target region are spaced apart from the edges of the target die image by the preset size, and the upper edge of the (2, 0) sub-region is the upper edge of the target region, the upper edge of the alignment image (i.e., the whole image shown in FIG. 6) obtained by outwardly expanding each edge of the (2, 0) sub-region is coincident with the upper edge of the target die image. Figure 5 Figure 5 203. The template matching algorithm is used to perform template matching on the template image and the alignment image, to obtain a plurality of alignment pixel points and a correlation coefficient corresponding to each alignment pixel point;

[0041] In this embodiment, the template matching algorithm can be any template matching algorithm, for example, OpenCV, scikit-learn, etc. When the template matching algorithm is used to perform template matching on the template image and the alignment image, the alignment pixel point on which the alignment of the alignment image and the template image is based can be determined, and the correlation coefficient between the two can be calculated based on the alignment pixel point. The correlation coefficient is used to represent the similarity between the alignment region of the alignment image and the template image when the alignment of the alignment image and the template image is based on the alignment pixel point.

[0042] Since the size of the alignment image is greater than that of the template image, the alignment region of the alignment image is the region in which the pixels of the template image are aligned with the pixels of the alignment image when the alignment of the alignment image and the template image is based on the alignment pixel point. Since the template image and its alignment image are both composed of pixels, i.e., both are pixel matrices, calculating the correlation coefficient is equivalent to calculating the correlation coefficient between the pixel matrix corresponding to the template image and the pixel matrix corresponding to the alignment region of the alignment image.

[0043] Since the size of the alignment image is greater than that of the template image, the alignment region of the alignment image is the region in which the pixels of the template image are aligned with the pixels of the alignment image when the alignment of the alignment image and the template image is based on the alignment pixel point. Since the template image and its alignment image are both composed of pixels, i.e., both are pixel matrices, calculating the correlation coefficient is equivalent to calculating the correlation coefficient between the pixel matrix corresponding to the template image and the pixel matrix corresponding to the alignment region of the alignment image.

[0044] ​For example, the pixel corresponding to the top-left corner of the template image can be taken as a reference point, and the pixel corresponding to the top-left corner of the alignment image can be taken as a starting point. The top-left corner pixel of the template image is aligned with the top-left corner pixel of the alignment image, and the correlation coefficient is calculated. Then, the template image is moved along the horizontal direction and the vertical direction, so that the top-left corner pixel of the template image is sequentially aligned with other pixels of the alignment image, and the correlation coefficient is calculated. The alignment is stopped when the bottom-right corner pixel of the template image is aligned with the bottom-right corner pixel of the alignment image in a certain alignment, and the alignment is completed. As shown in the example of FIG. 11, assuming that the alignment image obtained after the edges of the (2, 0) sub-region are expanded by 50 pixels is as shown in FIG. 12, the final alignment pixel point range is a rectangular range with the top-left corner pixel of the alignment image as a vertex and a side length of 101x101, that is, as shown in FIG. 13, that is, 101x101 alignment pixel points are obtained. Figure 5 Figure 5 Figure 5

[0045] The manner of calculating the correlation coefficient can be as follows: the mean value of the pixel values of all the pixels in the alignment region of the alignment image aligned with the template image is calculated and denoted as imgMean, and the pixel value of each pixel is subtracted from imgMean to obtain a pixel difference matrix denoted as imgDiff; the mean value of the pixel values of all the pixels of the template image is calculated and denoted as tempMean, and the pixel value of each pixel is subtracted from tempMean to obtain a pixel difference matrix denoted as tempDiff. The correlation of imgDiff and tempDiff is calculated to obtain the correlation coefficient corresponding to the current alignment pixel point.

[0046] Of course, other algorithms related to the correlation between matrices can also be used to calculate the correlation coefficient corresponding to each alignment pixel point, and the present embodiment is not limited in this regard.

[0047] 204. determining the coordinates of the target alignment pixel point of the sub-region according to the correlation coefficients corresponding to the plurality of alignment pixel points and the gray values of the alignment pixel points;

[0048] The computer device can pre-establish a rectangular coordinate system, and place the target die image in the rectangular coordinate system to determine the coordinates of each pixel point.

[0049] After the plurality of alignment pixel points and the correlation coefficients corresponding to each alignment pixel point are obtained, the coordinates of the target alignment pixel point of the sub-region can be determined according to the correlation coefficients corresponding to the plurality of alignment pixel points and the gray values of the alignment pixel points. As shown in FIG. 14, the target alignment pixel point of the (2, 0) sub-region is (2, 0). Figures 3 to 5 ​​​After obtaining the multiple alignment pixels of the (2, 0) sub-region and the gray values of each alignment pixel, the coordinates of the target alignment pixel of the (2, 0) sub-region can be determined according to the correlation coefficients corresponding to the multiple alignment pixels of the (2, 0) sub-region and the gray values of the alignment pixels.

[0050] Therefore, the steps 202 to 204 are performed on each sub-region respectively, and the coordinates of the target alignment pixel of each sub-region can be determined.

[0051] 205、When the coordinates of the target alignment pixels of the multiple sub-regions are obtained, the alignment coordinates of the target die image are determined according to the coordinates of the target alignment pixels of the multiple sub-regions.

[0052] When the coordinates of the target alignment pixels of each sub-region are determined, the alignment coordinates of the target die image can be further determined according to the coordinates of the target alignment pixels of the multiple sub-regions of the target die image.

[0053] In the defect detection of the target die image, the target die image and the reference image used for assisting the detection can be aligned based on the alignment coordinates of the target die image. Since the alignment coordinates of the target die image are calculated based on the pixel distribution and the periodic change of each sub-region of the target die image, the target die image and the reference image can be accurately aligned based on the alignment coordinates, and the influence of the periodic pattern of the die image on the image alignment can be avoided, so that the die image alignment is more accurate in the die defect detection, and the accuracy of the die defect detection is ensured.

[0054] Based on Figure 2 In a preferred embodiment of the embodiment shown in the figure, when the coordinates of the target alignment pixel of the sub-region are determined in step 204, whether there is a periodic pattern in the x-axis direction and the y-axis direction of the alignment image can be determined according to the correlation coefficients corresponding to the multiple alignment pixels of the sub-region. If there is a periodic pattern in the x-axis direction of the alignment image and there is no periodic pattern in the y-axis direction, the target x-coordinate is determined according to the gray values of the alignment pixels in any row of the alignment pixel matrix formed by the multiple alignment pixels of the sub-region, and a preset value is taken as the target y-coordinate, and the target x-coordinate and the target y-coordinate constitute the coordinates of the target alignment pixel of the sub-region.

[0055] If there is no periodic pattern in the x-axis direction of the alignment image and there is a periodic pattern in the y-axis direction, the target y-coordinate is determined according to the gray values of the alignment pixels in any column of the alignment pixel matrix, and a preset value is taken as the target x-coordinate, and the target x-coordinate and the target y-coordinate constitute the coordinates of the target alignment pixel of the sub-region.

[0056] For example, along withFigures 3 to 5 The example shown, Figure 5 The alignment pixel point range shown is also the alignment pixel point matrix composed of multiple alignment pixel points of the (2, 0) sub-region. The target alignment pixel point of the sub-region can be determined based on the gray value of each pixel point in the alignment pixel point matrix.

[0057] The preset value can be any numerical value, as long as it indicates that the preset value does not participate in the subsequent calculation of the alignment coordinates of the target die image. For example, the preset value can be -1, indicating that this value does not participate in the subsequent calculation of the alignment coordinates of the target die image.

[0058] Therefore, the target alignment pixel point of each sub-region can be determined in this way. The target alignment pixel point of the sub-region is calculated based on the pixel distribution and periodic variation of the sub-region. Therefore, the target alignment pixel point of the sub-region can be used as the accurate alignment pixel point of the sub-region during image alignment, providing an accurate calculation basis for the subsequent determination of the alignment coordinates of the target die image.

[0059] When determining the target x coordinate based on the gray value of the alignment pixel point in any row of the alignment pixel point matrix, the x coordinate of the pixel corresponding to the extreme value of the gray value of the row in the alignment pixel point matrix can be determined as the target x coordinate.

[0060] As used Figures 3 to 5 In the example shown, the extreme value of the y=50 row in the alignment pixel point matrix can be obtained. The determination method can be as follows: the gray values of all pixels in the current row are obtained, the width of the alignment pixel point matrix is 101, i.e., there are 101 gray values, and the x coordinate of the extreme value point is calculated from the third pixel point. If the current pixel value is greater than the average of the values of the previous two pixels and also greater than the average of the values of the two subsequent pixels, the current point is the extreme value point. After calculating the 99th pixel point, a total of 97 values are obtained. Assuming that m extreme value points are obtained, the x coordinate of the extreme value point closest to x=50 in the m extreme value points is taken as the target x coordinate.

[0061] Of course, in addition to taking the x coordinate of the pixel point with the extreme value of the gray value as the target x coordinate, the x coordinate of the pixel point corresponding to the median value of the gray value in the current row can also be taken as the target x coordinate, which is not limited in the present embodiment.

[0062] Similarly, when determining the target y coordinate based on the gray value of the alignment pixel point in any column of the alignment pixel point matrix, the y coordinate of the pixel point corresponding to the extreme value of the gray value of the column in the alignment pixel point matrix can be determined as the target y coordinate.

[0063] As used Figures 3 to 5In the example shown, the extreme value of the column x=50 in the pixel matrix can be obtained, and the extreme value can be determined in the following manner. The gray values of all pixels in the current column are obtained, and the length of the pixel matrix is 101, i.e., there are 101 gray values. Starting from the third pixel, if the current pixel value is greater than the average of the values of the previous two pixels and is also greater than the average of the values of the next two pixels, the current pixel is an extreme value. The y coordinate of the extreme value closest to y=50 in the m extreme values obtained by calculating the 99th pixel is taken as the target y coordinate.

[0064] Therefore, the target pixel coordinates of each sub-region can be quickly calculated and determined in the above manner, and the calculation process of the target die image alignment coordinates is further facilitated.

[0065] In this embodiment, whether there is a periodic pattern in the x-axis direction and the y-axis direction of the alignment image is determined. In one preferred embodiment, the deviation degree coefficient corresponding to the correlation coefficient of each column in the pixel matrix is calculated. If the deviation degree coefficient of each column is less than a first preset threshold, it is determined that there is a periodic pattern in the x-axis direction of the alignment image, and it is determined that there is no periodic pattern in the y-axis direction of the alignment image. The deviation degree coefficient corresponding to the correlation coefficient of each row in the pixel matrix is calculated. If the deviation degree coefficient of each row is less than the first preset threshold, it is determined that there is a periodic pattern in the y-axis direction of the alignment image, and it is determined that there is no periodic pattern in the x-axis direction of the alignment image.

[0066] For example, the projection is performed along the y direction of the pixel matrix, the average of the correlation coefficients corresponding to each column of pixels is calculated, and the absolute value of the difference between the average and the correlation coefficient corresponding to each pixel in the column is taken. The accumulated value of all the results after subtraction is the deviation degree coefficient of the column. If the accumulated value is less than a preset threshold (such as 0.01), it is considered that there is a periodic change in the current x direction, and it is considered that there is no periodic change in the current y direction.

[0067] Of course, the accumulated value can be divided by the number of pixels in the column to obtain the quotient (i.e., the average deviation), which is compared with the preset threshold. Alternatively, the sum of the squares of the differences between the correlation coefficients corresponding to all pixels in the column and the average of the correlation coefficients is calculated, and the sum is taken as the deviation degree coefficient of the column, which is compared with the preset threshold. The calculation method of the deviation degree coefficient corresponding to the correlation coefficient of each column or each row is not limited in this embodiment, as long as the deviation degree coefficient can represent the deviation level of the correlation coefficients of the row or the column.

[0068] Therefore, this method can quickly determine whether there are periodic patterns in various ways of aligning images, which facilitates the rapid calculation of the coordinates of the target alignment pixels in the sub-region based on the judgment results.

[0069] based on Figure 2 In a preferred embodiment of the illustrated example, before step 202, the dispersion coefficient of the grayscale values ​​of all pixels in each sub-region can be calculated. If the dispersion coefficient is less than a second preset threshold, the sub-region is determined to be a non-periodic pattern. Sub-regions with only non-periodic patterns do not need to determine the corresponding target alignment pixels, that is, the sub-region has no target alignment pixels. If the dispersion coefficient is greater than the second preset threshold, it indicates that the sub-region contains a periodic pattern. At this time, step 202 and subsequent steps can be executed to determine the target alignment pixels of the sub-region.

[0070] by Figure 4 For example, for the (0,0) sub-region, the mean gray value of the image in this sub-region can be calculated, denoted as img_mean; the maximum and minimum gray values ​​of the image in this sub-region can be determined, denoted as img_min and img_max; the dispersion coefficient val can be calculated, val = (img_max - img_min) / img_mean; if val > 1.5 (i.e. ±2db), then the corresponding target alignment pixel point needs to be calculated for this sub-region, i.e., step 202 and subsequent steps are executed; otherwise, it is not necessary, indicating that there is no target alignment pixel point in this sub-region, and the calculation of the next sub-region begins.

[0071] Therefore, in step 205, when determining the alignment coordinates of the target die image based on the coordinates of the target alignment pixels in multiple sub-regions, if the number of target alignment pixels in multiple sub-regions of the target die image is zero, it indicates that all sub-regions of the target die image are flat regions, i.e., without periodic patterns. In this case, the alignment coordinates of the target die image can be determined as the coordinates of the origin of the coordinate system, i.e., (0, 0). Still using... Figure 4 For example, if there are no target alignment pixels in any sub-region of the target die image, then the alignment coordinates of the target die image are (0, 0).

[0072] If the number of target alignment pixels in multiple sub-regions of the target die image is greater than zero and less than a preset number, then the average of the cumulative x-coordinates of the target alignment pixels in multiple sub-regions is used as the x-coordinate of the alignment coordinate of the target die image, and the average of the cumulative y-coordinates of the target alignment pixels in multiple sub-regions is used as the y-coordinate of the alignment coordinate of the target die image.

[0073] For example, if the number of target alignment pixel points of the plurality of sub-regions of the target die image is 1-3, it indicates that there are 1-3 sub-regions with periodic patterns. At this time, the x coordinates and y coordinates of all target alignment pixel points can be accumulated and averaged respectively. If the x coordinate or y coordinate of the target alignment pixel point is the preset value, for example, -1, the value of the x coordinate or y coordinate does not participate in the accumulation of the coordinate value.

[0074] If the number of target alignment pixel points of the plurality of sub-regions of the target die image is greater than the preset number, in the array composed of all x coordinates of the target alignment pixel points of the plurality of sub-regions of the target die image, the distance set of each value is calculated, the distance set is the set of the difference between the value and all values, and all elements in the set are sorted by size, and the element ranked in the preset sequence in the distance set is taken as the target distance value of the value, and the x coordinate corresponding to the minimum target distance value is taken as the x coordinate of the alignment coordinate of the target die image. And in the array composed of all y coordinates of the target alignment pixel points of the plurality of sub-regions of the target die image, the distance set of each value is calculated, and the element ranked in the preset sequence in the distance set is taken as the target distance value of the value, and the y coordinate corresponding to the minimum target distance value is taken as the y coordinate of the alignment coordinate of the target die image.

[0075] For example, the coordinates of the target alignment pixel points of the plurality of sub-regions are (2, -1), (2, 8), (1, 6), (-1, 7), (2, 8) and (2, 4) respectively. For the x direction, the array composed of all x coordinates is [2, 2, 1, -1, 2, 2], and the negative value does not participate in the calculation. The difference between each value and all values in the array is calculated respectively to obtain the distance set of each value. For example, the plurality of elements in the distance set of the first value "2" in the array are {0, 0, 1, empty, 0, 0}, which are sorted from large to small (or from small to large), and the distance set obtained is {1, 0, 0, 0, 0}. The third smallest element (i.e. "0") is selected as the target distance value of the first value "2" in the array, which is recorded as dist0. In this way, the target distance value of each value in the array can be calculated, which is recorded as dist1, dist2, empty, dist4 and dist5 respectively. The minimum value of dist0, dist1, dist2, dist4 and dist5 is taken, and the x coordinate corresponding to the minimum value is taken as the x coordinate of the alignment coordinate of the target die image.

[0076] Similarly, for the y direction, the array of all y coordinates is [-1, 8, 6, 7, 8, 4], and the negative value is not involved in the calculation. The distance set of each value in the array is calculated by calculating the difference between each value and all values, for example, the distance set of the second value "8" in the array has multiple elements {empty, 0, 2, 1, 0, 4}, and the distance set is sorted in descending order (or ascending order), and the third smallest element (i.e. "1") is selected as the target distance value of the second value "8" in the array, which is recorded as dist1. Similarly, the target distance values of each value in the array can be calculated, and are recorded as dist2, dist3, dist4 and dist5 respectively. The minimum value of dist1, dist2, dist3, dist4 and dist5 is taken, and the y coordinate corresponding to the minimum value is taken as the y coordinate of the target die image.

[0077] Finally, the alignment coordinates of the target die image are determined to be (2, 8) through the above calculation operations.

[0078] It should be noted that if the x coordinates or y coordinates of the multiple target alignment pixel points of the multiple sub-regions are all the preset values, for example, all are -1, it means that there is no effective information in this direction that can be used to determine the alignment coordinates, i.e. pure periodicity. In this case, the corresponding x value or y value is set to 0, and no alignment is needed in this direction. For example, as shown in the figure, the die image has a repeating period in the x direction, and no alignment is needed in the y direction. Figure 6

[0079] Therefore, the x coordinate and y coordinate of the alignment coordinates of the target die image can be accurately calculated through the above method, the image alignment deviation caused by the periodic change of the pattern of the die image is reduced, the accuracy of the die image alignment is improved, and the accuracy of the die image defect detection is further improved.

[0080] The image alignment method in the embodiments of the present application is described above, and the computer device in the embodiments of the present application is described below. Please refer to Figure 7 An embodiment of the computer device in the embodiments of the present application includes:

[0081] The acquisition unit 701 is configured to acquire a target die image to be processed, divide a target region in the target die image into multiple sub-regions, and set the edge of the target region to be spaced apart from the edge of the target die image by a preset size.

[0082] The calculation unit 702 is configured to perform the following steps for each sub-region with a periodic pattern:

[0083] ​The sub-region is taken as a template image, and an image obtained by extending each side of the sub-region by the preset size is taken as an alignment image;

[0084] The template image and the alignment image are matched according to a template matching algorithm, to obtain a plurality of alignment pixel points and a correlation coefficient corresponding to each alignment pixel point, the correlation coefficient being used to represent a similarity between an alignment region of the alignment image and the template image when the alignment image is aligned with the template image based on the alignment pixel point;

[0085] The coordinates of the target alignment pixel point of the sub-region are determined according to the correlation coefficient corresponding to each alignment pixel point and a gray value of each alignment pixel point.

[0086] The determination unit 703 is configured to determine the alignment coordinates of the target die image according to the coordinates of the target alignment pixel point of each of the plurality of sub-regions.

[0087] In a preferred embodiment of the present embodiment, the calculation unit 702 is specifically configured to:

[0088] The correlation coefficient corresponding to each alignment pixel point is used to determine whether a periodic pattern exists in an x-axis direction and a y-axis direction of the alignment image.

[0089] If the periodic pattern exists in the x-axis direction and does not exist in the y-axis direction of the alignment image, a target x-coordinate is determined according to a gray value of an alignment pixel point in any row of an alignment pixel point matrix formed by the plurality of alignment pixel points, and a target y-coordinate is a preset value, and the target x-coordinate and the target y-coordinate constitute the coordinates of the target alignment pixel point of the sub-region.

[0090] If the periodic pattern does not exist in the x-axis direction and exists in the y-axis direction of the alignment image, a target y-coordinate is determined according to a gray value of an alignment pixel point in any column of the alignment pixel point matrix, and a target x-coordinate is a preset value, and the target x-coordinate and the target y-coordinate constitute the coordinates of the target alignment pixel point of the sub-region.

[0091] In a preferred embodiment of the present embodiment, the calculation unit 702 is specifically configured to:

[0092] The x-coordinate of the alignment pixel point corresponding to the gray value extreme of the row is determined as the target x-coordinate.

[0093] The target y-coordinate is determined according to the gray value of the alignment pixel point in any column of the alignment pixel point matrix, and the target x-coordinate is a preset value, and the target x-coordinate and the target y-coordinate constitute the coordinates of the target alignment pixel point of the sub-region.

[0094] In the any column of the matrix of the alignment pixel points, the y coordinate of the pixel point corresponding to the extreme value of the gray value of the column is determined as the target y coordinate.

[0095] In a preferred embodiment of the present embodiment, the calculation unit 702 is specifically configured to:

[0096] calculate the deviation degree coefficient corresponding to the correlation coefficient of each column in the matrix of the alignment pixel points, and if the deviation degree coefficient of each column is less than a first preset threshold, it is determined that the x axis direction of the alignment image exists a periodic pattern, and it is determined that the y axis direction of the alignment image does not exist a periodic pattern.

[0097] calculate the deviation degree coefficient corresponding to the correlation coefficient of each row in the matrix of the alignment pixel points, and if the deviation degree coefficient of each row is less than the first preset threshold, it is determined that the y axis direction of the alignment image exists a periodic pattern, and it is determined that the x axis direction of the alignment image does not exist a periodic pattern.

[0098] In a preferred embodiment of the present embodiment, the determination unit 703 is specifically configured to:

[0099] If the number of the target alignment pixel points of the plurality of sub-regions is zero, the alignment coordinates of the target die image are determined as the origin coordinates.

[0100] If the number of the target alignment pixel points of the plurality of sub-regions is greater than zero and less than a preset number, the mean value of the cumulative value of the x coordinates of the target alignment pixel points of the plurality of sub-regions is taken as the x coordinate of the alignment coordinates of the target die image, and the mean value of the cumulative value of the y coordinates of the target alignment pixel points of the plurality of sub-regions is taken as the y coordinate of the alignment coordinates of the target die image.

[0101] In a preferred embodiment of the present embodiment, if the number of the target alignment pixel points of the plurality of sub-regions is greater than the preset number, the determination unit 703 is further configured to:

[0102] In the array composed of all the x coordinates of the target alignment pixel points of the plurality of sub-regions, a distance set of each value is calculated, the distance set is a set of the difference values of the value and all values and all elements in the set are sorted by size, and the element ranked in a preset sequence in the distance set is taken as the target distance value of the value, and the x coordinate corresponding to the minimum target distance value is taken as the x coordinate of the alignment coordinates of the target die image; and

[0103] In an array composed of all y coordinates of the target alignment pixel points of the plurality of sub-regions, the distance set of each value is calculated, and an element ranked in a preset order in the distance set is taken as a target distance value of the value, and a y coordinate corresponding to the minimum target distance value is taken as a y coordinate of the alignment coordinate of the target die image.

[0104] In a preferred embodiment of the present embodiment, before taking the sub-region as a template image and taking an image obtained by outwardly expanding each edge of the sub-region by the preset size as an alignment image, the calculation unit 702 is further configured to:

[0105] calculate a dispersion degree coefficient of the gray value of all pixel points in the sub-region.

[0106] if the dispersion degree coefficient is less than a second preset threshold, determine that the sub-region is a non-periodic pattern.

[0107] if the dispersion degree coefficient is greater than the second preset threshold, execute the taking the sub-region as a template image and the taking the image obtained by outwardly expanding each edge of the sub-region by the preset size as an alignment image.

[0108] In the present embodiment, the operations performed by each unit in the computer device are similar to those described in the foregoing Figure 2 embodiment, and thus are not described herein again.

[0109] In the defect detection of the target die image, the target die image and the reference image used for assisting the detection can be aligned based on the alignment coordinate of the target die image. Since the alignment coordinate of the target die image is calculated based on the pixel distribution condition and the periodic variation condition of each sub-region of the target die image, the target die image and the reference image can be accurately aligned based on the alignment coordinate, and the influence of the periodic pattern of the die image on the image alignment can be avoided, so that the die image alignment is more accurate in the die defect detection, and the accuracy of the die defect detection is ensured.

[0110] The computer device in the present embodiment will be described below. Please refer to Figure 8 , one embodiment of the computer device in the present embodiment includes:

[0111] The computer device 800 can include one or more central processing units (CPUs) 801 and a memory 805 in which one or more application programs or data are stored.

[0112] The memory 805 can be volatile memory or non-volatile memory. The program stored in the memory 805 can include one or more modules, each of which can include a series of instruction operations in the computer device. Further, the central processor 801 can be configured to communicate with the memory 805 to execute the series of instruction operations in the memory 805 on the computer device 800.

[0113] The computer device 800 can further include one or more power supplies 802, one or more wired or wireless network interfaces 803, one or more input / output interfaces 804, and / or one or more operating systems, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0114] The central processor 801 can execute the operations of the computer device in the above-described embodiments, which will not be described here in detail. Figure 2

[0115] The embodiments of the present application also provide a computer storage medium, one embodiment of which includes: the computer storage medium stores instructions, which, when executed on a computer, cause the computer to perform the operations of the computer device in the above-described embodiments. Figure 2

[0116] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the above-described method embodiments, which will not be described here in detail.

[0117] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0118] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment scheme. ​​

[0119] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0120] When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions, can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, read-only memory), a random access memory (RAM, random access memory), a magnetic disk or an optical disk, and various other media that can store program codes.

Claims

1. An image alignment method, characterized in that, The method includes: Obtain the target die image to be processed, divide the target region in the target die image into multiple sub-regions, and the edge of the target region is spaced from the edge of the target die image by a preset size; Perform the following steps for each of the sub-regions exhibiting a periodic pattern: The sub-region is used as a template image, and the image obtained by expanding each side of the sub-region by the preset size is used as the alignment image; The template image and the matching image are matched according to the template matching algorithm to obtain multiple matching pixels and the correlation coefficient corresponding to each matching pixel. The correlation coefficient is used to characterize the similarity between the matching region of the matching image and the template image when the matching image and the template image are matched based on the matching pixels. The coordinates of the target aligned pixel in the sub-region are determined based on the correlation coefficients corresponding to the plurality of aligned pixels and the grayscale values ​​of each aligned pixel. When obtaining the coordinates of the target alignment pixels in the multiple sub-regions, the alignment coordinates of the target die image are determined based on the coordinates of the target alignment pixels in the multiple sub-regions.

2. The method according to claim 1, characterized in that, Determining the coordinates of the target aligned pixel in the sub-region based on the correlation coefficients corresponding to the plurality of aligned pixels and the grayscale values ​​of each aligned pixel includes: Based on the correlation coefficients corresponding to the plurality of aligned pixels, it is determined whether there are periodic patterns in the x-axis and y-axis directions of the aligned image; If the alignment image has a periodic pattern in the x-axis direction but not in the y-axis direction, the target x-coordinate is determined based on the gray value of the alignment pixel in any row of the alignment pixel matrix formed by the multiple alignment pixels, and a preset value is used as the target y-coordinate. The target x-coordinate and the target y-coordinate constitute the coordinates of the target alignment pixel in the sub-region. If there is no periodic pattern in the x-axis direction but a periodic pattern in the y-axis direction of the alignment image, the target y coordinate is determined based on the gray value of the alignment pixel in any column of the alignment pixel matrix, and a preset value is used as the target x coordinate. The target x coordinate and the target y coordinate constitute the coordinates of the target alignment pixel in the sub-region.

3. The method according to claim 2, characterized in that, Determining the target x-coordinate based on the grayscale value of the corresponding pixel in any row of the matrix of corresponding pixels formed by the plurality of corresponding pixels includes: The x-coordinate of the pixel corresponding to the extreme gray value of any row in the alignment pixel matrix is ​​determined as the target x-coordinate. Determining the target y-coordinate based on the grayscale value of the corresponding pixel in any column of the corresponding pixel matrix includes: The target y-coordinate is determined by identifying the y-coordinate of the pixel corresponding to the extreme gray value in any column of the alignment pixel matrix.

4. The method according to claim 2, characterized in that, The step of determining whether there is a periodic pattern in the x-axis and y-axis directions of the aligned image based on the correlation coefficients corresponding to the plurality of aligned pixels includes: Calculate the deviation coefficient corresponding to the correlation coefficient of each column in the alignment pixel matrix. If the deviation coefficient of each column is less than the first preset threshold, it is determined that there is a periodic pattern in the x-axis direction of the alignment image and that there is no periodic pattern in the y-axis direction of the alignment image. Calculate the deviation coefficient corresponding to the correlation coefficient of each row in the alignment pixel matrix. If the deviation coefficient of each row is less than the first preset threshold, it is determined that there is a periodic pattern in the y-axis direction of the alignment image and that there is no periodic pattern in the x-axis direction of the alignment image.

5. The method according to claim 1, characterized in that, Determining the alignment coordinates of the target die image based on the coordinates of the target alignment pixels in the plurality of sub-regions includes: If the number of target alignment pixels in the plurality of sub-regions is zero, then the alignment coordinates of the target die image are determined to be the origin coordinates; If the number of target alignment pixels in the multiple sub-regions is greater than zero and less than a preset number, then the average of the accumulated x-coordinates of the target alignment pixels in the multiple sub-regions is taken as the x-coordinate of the alignment coordinate of the target die image, and the average of the accumulated y-coordinates of the target alignment pixels in the multiple sub-regions is taken as the y-coordinate of the alignment coordinate of the target die image.

6. The method according to claim 5, characterized in that, If the number of target alignment pixels in the plurality of sub-regions is greater than the preset number, the method further includes: In an array consisting of all x-coordinates of the target alignment pixels in the plurality of sub-regions, a distance set is calculated for each value. This distance set is the set of differences between the current value and all other values, and all elements in the set are sorted by size. The element in the distance set at a preset position is taken as the target distance value for that value, and the x-coordinate corresponding to the smallest target distance value is taken as the x-coordinate of the alignment coordinates of the target pixel image. In the array of all y coordinates of the target alignment pixels in the multiple sub-regions, the distance set for each value is calculated, and the element in the distance set that is ranked in a preset order is taken as the target distance value of that value. The y coordinate corresponding to the smallest target distance value is taken as the y coordinate of the alignment coordinate of the target die image.

7. The method according to claim 1, characterized in that, Before using the sub-region as a template image and expanding each side of the sub-region by the preset size to obtain the corresponding image, the method further includes: Calculate the dispersion coefficient of the grayscale values ​​of all pixels in the sub-region; If the dispersion coefficient is less than the second preset threshold, then the sub-region is determined to be an aperiodic pattern; If the dispersion coefficient is greater than the second preset threshold, then the process of using the sub-region as a template image and expanding each edge of the sub-region by the preset size to obtain an image as a alignment image is executed.

8. A computer device, characterized in that, The computer device includes: The acquisition unit is used to acquire a target die image to be processed, divide the target region in the target die image into multiple sub-regions, and the edge of the target region is spaced from the edge of the target die image by a preset size. The calculation unit is configured to perform the following steps for each of the sub-regions exhibiting a periodic pattern: The sub-region is used as a template image, and the image obtained by expanding each side of the sub-region by the preset size is used as the alignment image; The template image and the matching image are matched according to the template matching algorithm to obtain multiple matching pixels and the correlation coefficient corresponding to each matching pixel. The correlation coefficient is used to characterize the similarity between the matching region of the matching image and the template image when the matching image and the template image are matched based on the matching pixels. The coordinates of the target aligned pixel in the sub-region are determined based on the correlation coefficients corresponding to the plurality of aligned pixels and the grayscale values ​​of each aligned pixel. The determining unit is used to determine the alignment coordinates of the target die image based on the coordinates of the target alignment pixels in the multiple sub-regions when obtaining the coordinates of the target alignment pixels in the multiple sub-regions.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium stores instructions that, when executed on the computer, cause the computer to perform the method as described in any one of claims 1 to 7.

Images