CUDA-based golden template graph generation method and readable storage medium

Abstract

The application discloses a CUDA-based Golden template graph generation method, comprising the following steps: step one, scanning multiple same product graphs based on field equipment, and manually determining a template as a reference graph; step two, based on the data extracted in the previous step and the reference template graph, transmitting to the video memory; step three, based on CUDA, a fast sub-pixel down-sampling template matching model with rotation for data and template graphs is proposed, which runs in the GPU, realizes high-speed image processing through the parallel architecture of CUDA, quickly solves the positioning and extraction of all Golden template graphs, quickly and accurately realizes template graph fusion, and automatically separates noise, defects and other problems.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a Golden template graph generation method based on CUDA. BACKGROUND

[0002] The Golden template graph is mainly applied to the field of precise or complex image data scenes such as semiconductors. In the actual production process, the Golden graph is used to automatically match and position the target to be detected, and to realize the detection and positioning of defects. The Golden template graph has the characteristics of large picture data, few samples, and unfixed position angle, and has the requirements of fast calculation speed, high accuracy, etc. to meet the production scene of high-speed calculation.

[0003] At present, the generation of the Golden template graph is mainly based on the template matching technology of CUDA pyramid sub-pixel downsampling and the multi-layer image clustering fusion technology. The general method of traditional Golden graph generation is mainly based on CPU to manually extract sample graphs and then integrate picture data. The existing method has the following defects:

[0004] 1) The operation process is relatively complex, the number of operations is large, the processing speed is slow, and the sample graph collection requires high position accuracy. The intelligent degree is not high, which has a great influence on the increasingly rapid production demand.

[0005] 2) The processing of large Golden template picture data is not ideal, and it is difficult to realize automatic high-speed accurate extraction.

[0006] 3) Generally, CPU is used for picture fusion, and manual input of threshold value is required, which results in low calculation efficiency of large graph.

[0007] Therefore, the present application provides a Golden template graph generation method based on CUDA, which uses automatic threshold value for noise filtering and one-key generation of Golden template graph, which is more convenient to operate and meets the needs of high-speed production environment. SUMMARY

[0008] The present application aims to provide a Golden template graph generation method based on CUDA to solve the problems of slow processing speed, high sample graph collection requirement, and information loss in the sampling process leading to low accuracy of the final result in the existing Golden template graph generation method, as well as the low matching efficiency in the traditional template matching method.

[0009] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a Golden template graph generation method based on CUDA, characterized by comprising the following steps:

[0010] Step one: reference map determination, based on field device scanning multiple copies of the same product map, manually determine one of the templates as a reference map;

[0011] Step two, the extracted data, and the reference map determined in step one are transmitted to the video memory;

[0012] Step three, based on the CUDA architecture, a fast sub-pixel down-sampling template matching model with rotation is established for data and template map;

[0013] The formula of the pyramid sub-pixel down-sampling model with rotation is as follows:

[0014]

[0015]

[0016] In the formula, W and H represent the width and height of the template image respectively, org and res represent the original image and the down-sampled rotated image respectively, x and y represent the pixel position, θ represents the rotation angle (positive direction clockwise), n is the sampling number, x org , y org is the pixel position of the original image, which is a sub-pixel, and its pixel value is obtained by interpolation of the region it belongs to. In order to facilitate CUDA multi-thread parallel operation, the improved interpolation function based on BiCubic is as follows:

[0017]

[0018] In the formula, x represents the position difference between the pixel point and the sub-pixel,

[0019] The pixel weight at different distances can be obtained from this formula, and the calculation is carried out by bringing it into the sub-pixel gray value formula. This formula satisfies the interpolation calculation under any region radius, and the specific formula is as follows:

[0020]

[0021] In the formula, r represents the radius of the interpolation region, (x float , y float ) is the sub-pixel position, (x int , y int ) is the down-rounded pixel position, and f(x,y) represents the pixel value.

[0022] The formula of the improved correlation coefficient template matching model is as follows:

[0023]

[0024] In the formula, I(x, y) represents a detection image pixel position, T(x', y') represents a template image pixel position, w, h represents a template image width and height, represents a pixel mean value of a region at a current detection position, represents a template image pixel mean value;

[0025] Step four, clustering based on a CUDA architecture;

[0026] Step five, according to the clustering result obtained in step four, a maximum, minimum, and mean value graph in the clustering range is calculated, that is, a Golden template graph.

[0027] Preferably, in step four, the clustering operation includes the following steps:

[0028] a) calculating a mean value and a variance and a standard deviation of a data set, expanding or reducing the standard deviation as a clustering dynamic threshold according to a variance-to-standard deviation ratio threshold;

[0029] b) when a threshold range of a target class centroid is met (where the threshold is the threshold obtained in the previous process), the data set is classified into the target class, and the target class centroid position is updated;

[0030] c) looping through all data sets until all data sets are clustered, analyzing the size of all clusters, filtering noise according to the mean value size, and obtaining a final clustering result.

[0031] A readable storage medium, the readable storage medium stores computer program instructions, the computer program instructions are run by a processor, and the steps in the above method are executed.

[0032] Compared with the prior art, the beneficial effects of the present application are:

[0033] (1) The CUDA and the pyramid sub-pixel down-sampling model with rotation are combined for the first time, compared with the traditional method, the calculation accuracy is greatly improved, the loss of the image is minimized, and the calculation efficiency is greatly improved based on the CUDA for independent parallel calculation of each pixel.

[0034] (2) The sub-pixel down-sampling model is combined with the improved correlation coefficient template matching method, the pyramid is matched from top to bottom, the process from coarse matching to fine matching reduces unnecessary calculation amount and improves calculation efficiency.

[0035] (3) The improved pixel clustering method based on CUDA is proposed, the error caused by manual configuration parameters is optimized, and the Golden template graph production is quickly realized by automatically filtering noise and defects. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1A schematic diagram of a CUDA-based pyramid sub-pixel down-sampling matching model implementation method in the present application;

[0037] Figure 2 A schematic diagram of a clustering implementation process for a certain pixel position in the CUDA-based improved pixel clustering method proposed in the embodiment of the present application;

[0038] Figure 3 A sampling original image;

[0039] Figure 4 A sampling effect diagram of a traditional down-sampling method;

[0040] Figure 5 A sampling effect diagram of a sampling method using the present scheme;

[0041] Figure 6 A first original noise-containing defect image;

[0042] Figure 7 A Golden image obtained by clustering and denoising the first original noise-containing defect image using the method of the present scheme;

[0043] Figure 8 A second original noise-containing defect image;

[0044] Figure 9 A Golden image obtained by clustering and denoising the second original noise-containing defect image using the method of the present scheme. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0046] Please refer to Figure 1 The present application provides a technical solution: the method is mainly run in a GPU, and high-speed image processing is achieved through the parallel architecture of CUDA, Golden all template map positioning extraction is quickly solved, template map fusion is quickly and accurately achieved, and problems such as automatic separation of noise and defects are solved, and the main process includes:

[0047] Step one, based on the scanning of multiple identical product images by the on-site device, manually determine a template as a reference image;

[0048] Step two, based on the data extracted in the previous step and the reference template image, transmit to the video memory;

[0049] Step three, based on CUDA, proposes a fast sub-pixel down-sampling template matching model with rotation for data and template images;

[0050] a) The pyramid sub-pixel down-sampling model with rotation can not only realize multi-angle data and template detection, but also avoid the problem of information loss in the down-sampling process.

[0051]

[0052]

[0053] where W and H represent the width and height of the template image respectively, org and res represent the original image and the down-sampled rotated image respectively, x and y represent the pixel position, θ represents the rotation angle (positive direction clockwise), n is the sampling order, x org , y org is the pixel position of the original image, which is generally a sub-pixel, and its pixel value is obtained by interpolation of the region it belongs to. In order to facilitate the multi-thread parallel operation of CUDA, the improved interpolation function based on BiCubic is as follows:

[0054]

[0055] where x represents the position difference between the pixel and the sub-pixel. The pixel weight at different distances can be obtained from this formula, and the calculation is carried out by inputting the sub-pixel gray value formula. This formula satisfies the interpolation calculation under any region radius, and the specific formula is as follows:

[0056]

[0057] where r represents the radius of the interpolation region, (x float , y float ) is the sub-pixel position, (x int , y int ) is the down-rounded pixel position, and f(x,y) represents the pixel value.

[0058] b) The improved correlation coefficient template matching model formula is as follows:

[0059]

[0060] where I(x,y) represents the pixel position of the detection image, T(x',y') represents the pixel position of the template image, w and h represent the width and height of the template image, represents the pixel mean value of the region at the current detection position, represents the pixel mean value of the template image. In the CUDA calculation process, there is no need to traverse all pixel positions. Reasonable step size can be added according to the data size to reduce the calculation amount, which has little effect on the detection effect.

[0061] Step four, based on CUDA architecture, an improved clustering method for multi-template image fusion is proposed, which takes the dynamic threshold multiple of the variance and standard deviation of the data set as the clustering threshold. This method not only reduces the calculation amount, but also does not need to set the tolerance parameter artificially, and has a good inhibitory effect on noise and defects. The improved clustering method in this scheme is:

[0062] 1. Calculate the mean, variance and standard deviation of the data set, and use the threshold of the ratio of variance to standard deviation to expand or reduce the standard deviation as the dynamic clustering threshold.

[0063] 2. When the centroid of the target class is within the threshold range (where the threshold is the threshold obtained in the previous process), it is classified as the class, and the centroid position of the class is updated.

[0064] 3. Loop through all data sets until all data sets are clustered, analyze the size of all clusters, filter noise by mean size, and get the final clustering result.

[0065] Step five, according to the clustering result obtained in the previous step, calculate the maximum, minimum and mean image in the clustering range, which is the Golden template image.

[0066] Compared with the traditional method, this scheme first proposes a method of combining CUDA with a pyramid sub-pixel down-sampling model containing rotation. Compared with the traditional method, it not only greatly improves the calculation accuracy and reduces the loss of the image to the minimum, but also performs parallel calculation on each pixel based on CUDA, greatly improving the calculation efficiency. At the same time, the sub-pixel down-sampling model is combined with the improved correlation coefficient template matching method, and the matching method from top to bottom of the pyramid is adopted. The process from coarse matching to fine matching reduces unnecessary calculation and improves calculation efficiency. An improved pixel clustering method based on CUDA is proposed, which optimizes the error caused by manual parameter configuration, and quickly realizes the automatic filtering of noise and defects in the Golden template image production.

[0067] In order to verify the computing efficiency of the present application, 25 image data of 4852*12480 are used to make the Goden graph of 4504*4758 in the experiment, and the computing efficiency experimental data in Table 1 can be known that the computing efficiency is greatly improved compared with CPU due to the characteristics of GPU multi-thread parallel computing. The pyramid sub-pixel down-sampling matching model based on CUDA in the present application greatly reduces the detection range in the pyramid downward matching process, simplifies the unnecessary detection position, and gradually improves the matching position with the downward layer superposition, so that a more efficient detection algorithm is finally realized. The down-sampling effect is greatly improved in efficiency compared with the traditional down-sampling method. The clustering algorithm based on CUDA in the present application is greatly improved in clustering computing efficiency due to the characteristics of GPU.

[0068] Table 1 computing efficiency experimental data

[0069]

[0070] In order to verify the effect of the down-sampling method used in the present application, the 4-layer down-sampling effect of the original graph is compared. The traditional down-sampling based on OPENCV appears obvious blur and loses the detail texture feature information. The down-sampling method used in the present application, although the image quality is reduced, the overall image is relatively clear, and the down-sampling with angle is supported. The comparison and analysis show that the sub-pixel down-sampling algorithm with rotation in the present application is faster and better. Specifically, the comparison can be made as follows: Figures 3 to 5 The resolution of the original graph is 2252*2379, the image data is large and the image is clear. After using the traditional sampling method, the resolution is reduced to 141*149, the detail texture is lost and the image is in a blurred state. After using the sampling method of the present application, the resolution is 140*14, the image is reduced, but the overall image is relatively clear.

[0071] In order to verify the clustering effect, two groups of graphs before and after clustering are selected for comparison, Figure 6 is the first original noise defect graph; Figure 7 is the Golden graph after clustering and denoising of the first original noise defect graph by using the method of the present application; Figure 8 is the second original noise defect graph; Figure 9 is the Golden graph after clustering and denoising of the second original noise defect graph by using the method of the present application. The comparison result shows that the noise and defects are successfully removed by the clustering method of the present application, and the removal effect is good, which meets the actual production demand.

[0072] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary of the principles and application of the present application. Numerous modifications and adaptions can be effected without departing from the spirit and scope of the present application, which is not limited to the exact construction and arrangement described. It is intended, therefore, to cover all modifications and adaptions that fall within the scope of the claims and their equivalents.

Claims

1. A CUDA-based Golden Template Map generation method, characterized in that, Comprising the following steps: Step one: reference map determination, based on field device scanning multiple copies of the same product map, manually determine one of the templates as a reference map; Step two, the extracted data, and the reference map determined in step one are transmitted to the video memory; Step three, based on the CUDA architecture, a fast data, template map is established with a rotating sub-pixel down-sampling template matching model; Wherein, the formula of the rotating pyramid sub-pixel down-sampling model is as follows: ； In the formula, W and H represent the width and height of the template image respectively, org and res represent the original image and the down-sampled and rotated image respectively, x and y represent the pixel position, θ represents the rotation angle, wherein the clockwise direction is the positive direction, and n is the sampling order, is the pixel position of the original image, the pixel position is a sub-pixel, and the pixel value is obtained by interpolation based on the region, in order to facilitate CUDA multi-thread parallel operation, the improved interpolation function based on BiCubic is as follows: ； Wherein, x represents the position difference between the pixel point and the sub-pixel, From this formula, the pixel weight under different distances can be obtained, and the sub-pixel gray value formula is calculated, which satisfies the interpolation calculation under any area radius, and the specific formula is as follows: ； wherein r represents an interpolation region radius size, (x float ,y float ) is a sub-pixel position, (x int ,y int ) is a down-integer pixel position, and represents a pixel value; The improved correlation coefficient template matching model formula is as follows: ； where I(x, y) represents the detected image pixel position, T(x', y') represents the template image pixel position, w, h represents the template image width and height, represents the pixel mean of the region at the current detection position, represents the template image pixel mean; Step four, clustering based on CUDA architecture; Step five, according to the clustering result obtained in step four, the maximum, minimum and mean value map in the clustering range are calculated, that is, the Golden template map.

2. The CUDA-based Golden Template Map generation method of claim 1, wherein, In step four, the clustering operation comprises the following steps: a) Calculate the mean, variance and standard deviation of the data set, and expand or reduce the standard deviation as the clustering dynamic threshold according to the ratio threshold of variance and standard deviation; b) When the centroid of the target class is within the threshold range, that is, the threshold obtained in the last process, it is classified into this class, and the centroid position of this class is updated; c) Loop through all data sets until all data sets are clustered, analyze the size of all clusters, filter noise according to the mean size, and obtain the final clustering result.

3. A readable storage medium, characterized by, The readable storage medium stores computer program instructions, and the computer program instructions are run by a processor to execute the steps in the method of any one of claims 1-2.

Images