Improved template matching grid point detection method

Abstract

The invention discloses an improved template matching grid point detection method. The method comprises the steps of S1, unit grid selection and image graying; s2, cutting the template image; s3, carrying out image down-sampling; s4, performing first-stage template grid point matching; s5, restoring the coordinates and cutting the image; and S6, two-stage template matching and fine adjustment. According to the improved template matching grid point detection method provided by the invention, when the variance matrix is calculated, the template image and the target image are subjected to normalization operation and then the corresponding pixel variance is calculated, so that the influence of illumination intensity is reduced; besides, a two-stage detection mode is adopted, target matching is carried out on a down-sampled image, points obtained through first-stage matching are restored to an original image, then small-range fine adjustment is carried out on grid points of a first-stage result on the original image, and the reasoning time is greatly shortened while the precision is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of grid point detection, and in particular to an improved template matching grid point detection method. Background Art

[0002] Grid point detection technology is used to detect and analyze regularly arranged point structures. Accurately detecting grid points can help identify and locate defects, facilitate quality control, and aid design and manufacturing. Combining computer vision, machine learning, and deep learning techniques, grid point detection significantly improves detection accuracy and efficiency.

[0003] The application of grid point detection technology is driving the industry towards automation and intelligence in areas such as textile production and quality control, monitoring and analyzing structural health conditions such as building deformation and cracks, inspecting solder joints on circuit boards in electronics manufacturing, and detecting cell grid points. With continued technological advancement, grid point detection will play a vital role in even more areas.

[0004] In the existing technology, the use of deep learning methods to detect grid points has the following disadvantages: the scene changes greatly and cannot adapt to various situations; the cloth grid image is large and the inference takes a long time.

[0005] In the existing technology, the template matching method used to search for grid points has the following disadvantages: low accuracy due to the influence of noise or different light intensity on the same fabric; single image reasoning takes a long time. Summary of the Invention

[0006] The present invention provides an improved template matching grid point detection method to solve the technical problems existing in the above-mentioned prior art.

[0007] To achieve the above object, the present invention provides an improved template matching grid point detection method, which includes:

[0008] S1: Unit grid selection and image grayscale

[0009] Select a unit grid A in the middle of the original image img, determine the four grid points of the unit grid A, calculate the width grid_w and height grid_h of the unit grid A, and grayscale the original image img to obtain the grayscale image img_gray.

[0010] The image coordinate system is established with the upper left corner of the grayscale image img_gray as the origin, the horizontal right direction as the positive direction of the x-axis, and the direction rotated 90° clockwise from the positive direction of the x-axis as the y-axis square;

[0011] S2: Crop template image

[0012] Centered on the grid point at the upper left corner of the unit grid A, crop an image with a width of crop_ratio×grid_w and a height of crop_ratio×grid_h from the grayscale image img_gray and use it as the template image template_gray.

[0013] The point B in the upper left corner of the grayscale image img_gray is recorded as lt_pt, and the deviation between the unit grid A and point B is (xo_bias, yo_bias);

[0014] S3: Image downsampling

[0015] Downsample the grayscale image img_gray and the template image template_gray to downsample_ratio times, and get img_gray_resize and template_gray_resize respectively.

[0016] The downsampled point B is lt_pt_resize(lt_pt_x_resize, lt_pt_y_resize), the width and height of the unit grid A are grid_w_resize and grid_h_resize respectively, and the deviation between the unit grid A and point B is (xo_bias_resize, yo_bias_resize);

[0017] S4: One-stage template matching grid points

[0018] Generate an all-zero pixel map C of the same size as img_gray_resize to record whether the grid points in img_gray_resize have been visited. The all-zero pixel map C is used to record the corresponding grid points that have been visited in img_gray_resize as visited.

[0019] For img_gray_resize, take lt_pt_resize as the starting point and take the matched grid point as the center in each subsequent cropping, slide successively in the x-axis direction of the image with a step size of grid_w_resize and in the y-axis direction of the image with a step size of grid_h_resize, and crop an image with a width and height of crop_ratio1×grid_w_resize and crop_ratio1×grid_h_resize on img_gray_resize, recorded as target_crop_img. The grid point matched at each sliding in the process is the grid point that has been visited.

[0020] In the all-zero pixel map C, the corresponding grid points are set to 1, and the template grid points are updated using the improved template matching algorithm for target_crop_img and template_gray_resize, and all grid areas on img_gray_resize are visited in sequence;

[0021] S5: Coordinate restoration and image cropping

[0022] Restore the N visited grid point coordinates in img_gray_resize to the corresponding coordinates in the grayscale image img_gray. On the grayscale image img_gray, with the N visited grid point coordinates as the center, crop an image that is padding_bias pixels larger than the top, bottom, left, and right edges of the template image template_gray, and record it as target_crop_img1;

[0023] S6: Two-stage template matching fine-tuning

[0024] The N target_crop_img1 and template image template_gray are processed using the improved template matching algorithm to obtain the best grid points in N small ranges, namely template matching points, and restore them to the coordinates in the grayscale image img_gray.

[0025] In one embodiment of the present invention, the crop_ratio is set to 0.3.

[0026] In one embodiment of the present invention, downsample_ratio is set to 2.

[0027] In one embodiment of the present invention, crop_ratio1 is set to 0.4.

[0028] In one embodiment of the present invention, padding_bias is set to 2.

[0029] In one embodiment of the present invention, the improved template matching algorithm steps are as follows:

[0030] S1′: Input the target image target_crop_img and the template image template_gray_resize, and normalize template_gray_resize to obtain the normalized template image template_gray_resize_norm;

[0031] S2′: Slide the normalized template image template_gray_resize_norm in the x-axis and y-axis directions of the normalized target image target_crop_img with a step size of 1, and calculate the variance of the corresponding pixels to obtain the variance matrix;

[0032] S3′: Use a 3x3 Gaussian kernel to slide in the variance matrix with a step size of 1, calculate the sum of the dot products of the variance matrix and the kernel function corresponding pixels, and obtain the filtered variance matrix;

[0033] S4′: Find the minimum variance value and its corresponding coordinates (match_x, match_y) in the filtered variance matrix. If the minimum variance value is less than the threshold val_thresh, the point corresponding to the coordinates (match_x, match_y) is considered the best matching point.

[0034] The coordinates of the grid points matched in the filter variance matrix are (x_grid_resize, y_grid_resize), where:

[0035] x_grid_resize=lt_pt_x_resize+match_x+xo_bias_resize+1

[0036] y_grid_resize=lt_pt_y_resize+match_y+yo_bias_resize+1

[0037] where 1 is the offset of the 3×3 Gaussian kernel.

[0038] In one embodiment of the present invention, val_thresh is set to 1.7.

[0039] The improved template matching grid point detection method provided by the present invention has the following beneficial technical effects:

[0040] (1) Since the area near the real grid point can have a small variance value with the template image, the present invention uses a 3×3 Gaussian kernel to effectively eliminate the isolated false detection points (or noise points) generated by template matching;

[0041] (2) When calculating the variance matrix, the present invention first performs a normalization operation on both the template image and the target image before calculating the corresponding pixel variance to reduce the influence of light intensity;

[0042] (3) The present invention adopts a two-stage detection method, first performing target matching on the downsampled image, restoring the points matched in the first stage to the original image, and then performing small-scale fine-tuning on the grid points of the first stage results on the original image, thereby greatly shortening the inference time while ensuring accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 It is a structural diagram of an improved template matching grid point detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0046] The present invention provides an improved template matching grid point detection method, such as Figure 1 FIG. 1 is a schematic structural diagram of an improved template matching grid point detection method according to an embodiment of the present invention. The improved template matching grid point detection method provided by the present invention includes:

[0047] S1: Unit grid selection and image grayscale

[0048] Select a unit grid A in the middle of the original image img, determine the four grid points of the unit grid A, calculate the width grid_w and height grid_h of the unit grid A, and grayscale the original image img to obtain the grayscale image img_gray.

[0049] The image coordinate system is established with the upper left corner of the grayscale image img_gray as the origin, the horizontal right direction as the positive direction of the x-axis, and the direction rotated 90° clockwise from the positive direction of the x-axis as the y-axis square;

[0050] S2: Crop template image

[0051] Centered on the grid point at the upper left corner of the unit grid A, crop an image with a width of crop_ratio×grid_w and a height of crop_ratio×grid_h from the grayscale image img_gray and use it as the template image template_gray.

[0052] The point B in the upper left corner of the grayscale image img_gray is recorded as lt_pt, and the deviation between the unit grid A and point B is (xo_bias, yo_bias);

[0053] S3: Image downsampling

[0054] Downsample the grayscale image img_gray and the template image template_gray to downsample_ratio times, and get img_gray_resize and template_gray_resize respectively.

[0055] The downsampled point B is lt_pt_resize(lt_pt_x_resize, lt_pt_y_resize), the width and height of the unit grid A are grid_w_resize and grid_h_resize respectively, and the deviation between the unit grid A and point B is (xo_bias_resize, yo_bias_resize);

[0056] S4: One-stage template matching grid points

[0057] Generate an all-zero pixel map C of the same size as img_gray_resize to record whether the grid points in img_gray_resize have been visited. The all-zero pixel map C is used to record the corresponding grid points that have been visited in img_gray_resize as visited.

[0058] For img_gray_resize, take lt_pt_resize as the starting point and take the matched grid point as the center in each subsequent cropping, slide successively in the x-axis direction of the image with a step size of grid_w_resize and in the y-axis direction of the image with a step size of grid_h_resize, and crop an image with a width and height of crop_ratio1×grid_w_resize and crop_ratio1×grid_h_resize on img_gray_resize, recorded as target_crop_img. The grid point matched at each sliding in the process is the grid point that has been visited.

[0059] In the all-zero pixel map C, the corresponding grid points are set to 1, and the template grid points are updated using the improved template matching algorithm for target_crop_img and template_gray_resize, and all grid areas on img_gray_resize are visited in sequence;

[0060] S5: Coordinate restoration and image cropping

[0061] Restore the N visited grid point coordinates in img_gray_resize to the corresponding coordinates in the grayscale image img_gray. On the grayscale image img_gray, with the N visited grid point coordinates as the center, crop an image that is padding_bias pixels larger than the top, bottom, left, and right edges of the template image template_gray, and record it as target_crop_img1;

[0062] S6: Two-stage template matching fine-tuning

[0063] The N target_crop_img1 and template image template_gray are processed using the improved template matching algorithm to obtain the best grid points in N small ranges, namely template matching points, and restore them to the coordinates in the grayscale image img_gray.

[0064] In one embodiment of the present invention, the crop_ratio is set to 0.3.

[0065] In one embodiment of the present invention, downsample_ratio is set to 2.

[0066] In one embodiment of the present invention, crop_ratio1 is set to 0.4.

[0067] In one embodiment of the present invention, padding_bias is set to 2.

[0068] In one embodiment of the present invention, the improved template matching algorithm steps are as follows:

[0069] S1′: Input the target image target_crop_img and the template image template_gray_resize, and normalize template_gray_resize to obtain the normalized template image template_gray_resize_norm;

[0070] S2′: Slide the normalized template image template_gray_resize_norm in the x-axis and y-axis directions of the normalized target image target_crop_img with a step size of 1, and calculate the variance of the corresponding pixels to obtain the variance matrix;

[0071] S3′: Use a 3x3 Gaussian kernel to slide in the variance matrix with a step size of 1, calculate the sum of the dot products of the variance matrix and the kernel function corresponding pixels, and obtain the filtered variance matrix;

[0072] S4′: Find the minimum variance value and its corresponding coordinates (match_x, match_y) in the filtered variance matrix. If the minimum variance value is less than the threshold val_thresh, the point corresponding to the coordinates (match_x, match_y) is considered the best matching point.

[0073] The coordinates of the grid points matched in the filter variance matrix are (x_grid_resize, y_grid_resize), where:

[0074] x_grid_resize=lt_pt_x_resize+match_x+xo_bias_resize+1

[0075] y_grid_resize=lt_pt_y_resize+match_y+yo_bias_resize+1

[0076] where 1 is the offset of the 3×3 Gaussian kernel.

[0077] In one embodiment of the present invention, val_thresh is set to 1.7.

[0078] like Figure 1 As shown, Figure 1 4 images involved in the embodiment of the present invention are shown in FIG. 4 , where the origin of each image is in the upper left corner, wherein:

[0079] img_gray_resize: convert the original image to grayscale and then resize;

[0080] target_crop_img: The area to search for a single grid point, which is cropped on img_gray_resize;

[0081] template_gray_resize_norm: a fixed template image cropped around the initial point after marking on img_gray_resize;

[0082] 3×3 Gaussian kernel: a 3×3 Gaussian kernel matrix;

[0083] lt_pt_x_resize and lt_pt_y_resize are the pixel distances in the x and y directions from the upper left corner of target_crop_img to the upper left corner of img_gray_resize;

[0084] match_x and match_y are the pixel distances in the x and y directions from the upper left corner of template_gray_resize_norm to the upper left corner of target_crop_img;

[0085] xo_bias_resize and yo_bias_resize are the pixel distances in the x and y directions from the upper left corner of the 3×3 Gaussian kernel to the upper left corner of template_gray_resize_norm;

[0086] The x and y pixel distances between the grid point and the upper left corner of the 3×3 Gaussian kernel are 1 and 1 respectively.

[0087] The improved template matching grid point detection method provided by the present invention has the following beneficial technical effects:

[0088] (1) Since the area near the real grid point can have a small variance value with the template image, the present invention uses a 3×3 Gaussian kernel to effectively eliminate the isolated false detection points (or noise points) generated by template matching;

[0089] (2) When calculating the variance matrix, the present invention first performs a normalization operation on both the template image and the target image before calculating the corresponding pixel variance to reduce the influence of light intensity;

[0090] (3) The present invention adopts a two-stage detection method, first performing target matching on the downsampled image, restoring the points matched in the first stage to the original image, and then performing small-scale fine-tuning on the grid points of the first stage results on the original image, thereby greatly shortening the inference time while ensuring accuracy.

[0091] Those skilled in the art will appreciate that the accompanying drawings are merely schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0092] Those skilled in the art will appreciate that the modules in the apparatuses of the embodiments may be distributed in the apparatuses of the embodiments as described in the embodiments, or may be located in one or more apparatuses different from the embodiments with corresponding changes. The modules in the above embodiments may be combined into one module or further divided into multiple sub-modules.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An improved template matching grid point detection method, characterized in that: include: S1: Unit grid selection and image grayscale Select a unit grid A in the middle of the original image img, determine the four grid points of the unit grid A, calculate the width grid_w and height grid_h of the unit grid A, and grayscale the original image img to obtain the grayscale image img_gray. The image coordinate system is established with the upper left corner of the grayscale image img_gray as the origin, the horizontal right direction as the positive direction of the x-axis, and the direction rotated 90° clockwise from the positive direction of the x-axis as the y-axis square; S2: Crop template image Centered on the grid point at the upper left corner of the unit grid A, crop an image with a width of crop_ratio×grid_w and a height of crop_ratio×grid_h from the grayscale image img_gray and use it as the template image template_gray. The point B in the upper left corner of the grayscale image img_gray is recorded as lt_pt, and the deviation between the unit grid A and point B is (xo_bias, yo_bias); S3: Image downsampling Downsample the grayscale image img_gray and the template image template_gray to downsample_ratio times, and get img_gray_resize and template_gray_resize respectively. The downsampled point B is lt_pt_resize(lt_pt_x_resize, lt_pt_y_resize), the width and height of the unit grid A are grid_w_resize and grid_h_resize respectively, and the deviation between the unit grid A and point B is (xo_bias_resize, yo_bias_resize); S4: One-stage template matching grid points Generate an all-zero pixel map C of the same size as img_gray_resize to record whether the grid points in img_gray_resize have been visited. The all-zero pixel map C is used to record the corresponding grid points that have been visited in img_gray_resize as visited. For img_gray_resize, take lt_pt_resize as the starting point and take the matched grid point as the center in each subsequent cropping, slide successively in the x-axis direction of the image with a step size of grid_w_resize and in the y-axis direction of the image with a step size of grid_h_resize, and crop an image with a width and height of crop_ratio1×grid_w_resize and crop_ratio1×grid_h_resize on img_gray_resize, recorded as target_crop_img. The grid point matched at each sliding in the process is the grid point that has been visited. In the all-zero pixel map C, the corresponding grid points are set to 1, and the template grid points are updated using the improved template matching algorithm for target_crop_img and template_gray_resize, and all grid areas on img_gray_resize are visited in sequence; S5: Coordinate restoration and image cropping Restore the N visited grid point coordinates in img_gray_resize to the corresponding coordinates in the grayscale image img_gray. On the grayscale image img_gray, with the N visited grid point coordinates as the center, crop an image that is padding_bias pixels larger than the top, bottom, left, and right edges of the template image template_gray, and record it as target_crop_img1; S6: Two-stage template matching fine-tuning The N target_crop_img1 and template image template_gray are processed using the improved template matching algorithm to obtain the best grid points in N small ranges, namely template matching points, and restore them to the coordinates in the grayscale image img_gray.

2. The improved template matching grid point detection method according to claim 1, characterized in that: The crop_ratio is set to 0.

3.

3. The improved template matching grid point detection method according to claim 1, characterized in that: The downsample_ratio is set to 2.

4. The improved template matching grid point detection method according to claim 1, characterized in that: The crop_ratio1 is set to 0.

4.

5. The improved template matching grid point detection method according to claim 1, characterized in that: padding_bias is set to 2.

6. The improved template matching grid point detection method according to claim 1, characterized in that: The steps of the improved template matching algorithm are as follows: S1′: Input the target image target_crop_img and the template image template_gray_resize, and normalize template_gray_resize to obtain the normalized template image template_gray_resize_norm; S2′: Slide the normalized template image template_gray_resize_norm in the x-axis and y-axis directions of the normalized target image target_crop_img with a step size of 1, and calculate the variance of the corresponding pixels to obtain the variance matrix; S3′: Use a 3x3 Gaussian kernel to slide in the variance matrix with a step size of 1, calculate the sum of the dot products of the variance matrix and the kernel function corresponding pixels, and obtain the filtered variance matrix; S4′: Find the minimum variance value and its corresponding coordinates (match_x, match_y) in the filtered variance matrix. If the minimum variance value is less than the threshold val_thresh, the point corresponding to the coordinates (match_x, match_y) is considered the best matching point. The coordinates of the grid points matched in the filter variance matrix are (x_grid_resize, y_grid_resize), where: x_grid_resize=lt_pt_x_resize+match_x+xo_bias_resize+1 y_grid_resize=lt_pt_y_resize+match_y+yo_bias_resize+1 where 1 is the offset of the 3×3 Gaussian kernel.

7. The improved template matching grid point detection method according to claim 6, characterized in that: val_thresh is set to 1.7.

Images