Deformable template matching method for printing detection
By employing methods such as localized template selection, triangular mesh construction, and optical flow optimization, the problem of high-speed and high-precision template matching on flexible printing materials was solved, achieving efficient and accurate deformation correction and meeting the inspection requirements of high-speed printing production lines.
Patent Information
- Application Number
- CN202511045435.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-29
- Publication Date
- 2025-11-14
AI Technical Summary
Existing technologies struggle to achieve high-speed, high-precision template matching on flexible printing materials, leading to false alarms and missed detections in defect detection. Furthermore, the computational load is large and the time consumption is long, failing to meet the detection requirements of high-speed printing production lines.
By employing a method of local positioning template selection, triangular mesh construction, offset calculation and interpolation, and optical flow optimization, sub-pixel accuracy deformation correction is achieved through local template matching and triangular mesh technology combined with optical flow optimization.
It achieves efficient and accurate template matching, reduces false alarms and missed alarms, improves computing speed, meets the real-time detection needs of high-speed printing production lines, and enhances the adaptability and flexibility of the method in areas with weak texture.
Smart Images
Figure CN120953642A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of printing inspection, specifically a deformable template matching method for printing inspection. Background Technology
[0002] In high-speed printing production lines, template matching is a prerequisite for defect detection, overprinting, and dimensional inspection. Paper substrates have almost no expansion or contraction, so rigid templates are sufficient. However, flexible materials such as plastic films can experience unpredictable local stretching under tension and temperature. If a sub-pixel deformation mapping between the template and the image to be inspected cannot be established in real time, subsequent defect detection will result in a large number of false alarms and missed alarms due to alignment errors.
[0003] Existing deformable matching schemes are mainly divided into three categories. Gray-scale-based optical flow or deformable meshes require pixel-by-pixel iteration, and even with GPU acceleration, it is still difficult to achieve millisecond-level accuracy. Feature-point-based descriptors such as SIFT / SURF have redundant degrees of freedom, and after RANSAC filtering, only global affine or perspective transformations can be obtained, which cannot characterize local stretching, and the computational load is large. Edge-based schemes such as HalconPatFlex and Cognex PatMax have poor stability in weak texture regions, and they optimize the entire image contour by contour, taking tens of milliseconds, which is difficult to meet the production line cycle of more than 200m / min. In engineering, attempts have been made to first use local rigid templates for coarse positioning, and then use TPS or triangular meshes to solve the displacement field, but TPS requires global matrix inversion, and triangular meshes introduce boundary discontinuities and holes, neither of which can complete a full-image correction within 100ms.
[0004] Therefore, there is an urgent need for a deformable template matching method that has low computational cost, local continuity, and sub-pixel accuracy, in order to meet the high-speed and high-precision detection requirements of flexible printing. Summary of the Invention
[0005] Based on this, the purpose of the present invention is to provide a deformable template matching method for printing inspection, so as to solve the technical problems mentioned in the background art.
[0006] To achieve the above objectives, the present invention provides the following technical solution: a deformable template matching method for printing inspection, comprising the following steps:
[0007] Step 1: Template Training and Selection. In the template training stage, local localization templates, including grayscale templates and shape templates, are obtained from the image. The templates are selected based on stability scoring, distance thresholds between features, and the number of target features.
[0008] Step 2: Triangular mesh construction. Add mesh points on the image boundary, merge pre-selected local template points to obtain seed points covering the entire image and triangulate them to generate a triangular mesh; generate two-dimensional mesh points with an interval of n pixels for the entire image, and record the triangle ID to which the mesh point belongs;
[0009] Step 3: Local matching and offset calculation. In the matching stage, all local templates are searched, the offset vectors of the triangle vertices are calculated, and the offset vectors of the internal grid points are obtained through the transformation matrix.
[0010] Step 4: Offset interpolation and mapping generation. The offset grid is interpolated and enlarged to obtain a mapping at the original resolution.
[0011] Step 5: Optical flow optimization. Optimize the mapping map using the optical flow method.
[0012] Step 6: Image correction. The image to be tested is remapped using a mapping map to obtain a corrected image.
[0013] Preferably, the acquisition of the local positioning template in step one specifically includes: extracting the center point of the candidate region using the Good Features to Track method, and cropping a rectangular window to generate a grayscale template or edge template;
[0014] The stability of the template is verified by the score map. If the position of the maximum value is consistent with the center point and there are no easily confused points around it, it is determined to be a stable template region.
[0015] Edge templates take precedence over grayscale templates.
[0016] Preferably, the matching of grayscale templates uses a normalized cross-correlation scoring formula:
[0017]
[0018] Where T is the template image and I is the target image.
[0019] Preferably, the shape template matching uses the cosine similarity score formula of the edge gradient vector:
[0020]
[0021] Where x1 i ,y1 i The gradient vector extracted from the Sobel edge at the i-th point on the template edge, x2 i ,y2 i The gradient vector extracted for the Sobel edge at the i-th point on the template edge.
[0022] Preferably, the final shape matching score is:
[0023]
[0024] Where N is the number of edge points of the template.
[0025] Preferably, in step two, the triangulation uses the Delaunay method to generate a triangular mesh covering the entire image;
[0026] For each image boundary point, record the IDs of the k nearest points to it, ensuring that the offset vector of the boundary region is obtained through interpolation.
[0027] Preferably, in step three, for a triangle whose three vertices are successfully matched, its Affine transformation matrix is calculated, and the offset vector of the internal grid points is obtained through this matrix;
[0028] For triangles with missing vertices, extract the corresponding points from the template, re-triangulate them, and calculate the offset vectors of the grid points inside the new triangle.
[0029] Preferably, in step four, the low-resolution offset grid is enlarged to the original image resolution using an interpolation method to generate a full-size mapping map.
[0030] Preferably, in step five, the optical flow optimization specifically involves:
[0031] Using the mapping obtained in step four as the initial vector field, single-layer optical flow optimization is performed on the original image size to correct subtle deformations.
[0032] Preferably, in step six, the remapping uses bilinear interpolation or bicubic interpolation to correct the image to be tested according to the mapping map, thereby obtaining a corrected map aligned with the template.
[0033] In summary, the present invention has the following main beneficial effects:
[0034] This invention solves the problem of local deformation in the printing inspection of flexible materials through an innovative deformable template matching process, and has the advantages of high efficiency, high precision and strong robustness, providing a reliable inspection solution for high-speed printing production lines.
[0035] High efficiency: By using local template filtering and triangular mesh technology, the time-consuming problem of pixel-by-pixel calculation is avoided. Combined with optical flow optimization, fine-tuning is only needed when necessary, which significantly improves the computing speed and meets the real-time detection requirements of high-speed production lines.
[0036] High precision: The sub-pixel positioning of the edge template and the local continuous deformation capability of the triangular mesh ensure the accurate calculation of the offset vector, effectively reducing false alarms and false negatives;
[0037] Robustness: By employing stability scoring and redundant template point design, the method's adaptability in weakly textured regions is enhanced, overcoming the limitations of traditional edge matching methods;
[0038] Flexibility: Supports selective optical flow optimization, allowing for dynamic adjustment of the process based on detection accuracy requirements, balancing efficiency and effectiveness. Attached Figure Description
[0039] Figure 1 This is a flowchart of the template matching method of the present invention;
[0040] Figure 2 This is the template matching verification diagram and score diagram of the present invention;
[0041] Figure 3 This is the triangulated mesh diagram of the present invention;
[0042] Figure 4 This is a schematic diagram of the offset vector field of the sampling grid of the present invention;
[0043] Figure 5 This is a diagram showing the corrected result of the present invention;
[0044] Figure 6 This is a diagram showing the result of the correction after optimization using the optional optical flow method of the present invention. Detailed Implementation
[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.
[0046] The embodiments of the present invention will now be described.
[0047] This embodiment focuses on referring to the appendix. Figure 1 , 2 As shown, in a preferred embodiment of the present invention,
[0048] A deformable template matching method for printing inspection.
[0049] Step 1: During template training, good local localization templates are obtained from the image, including grayscale templates and shape templates. The templates are then scored and selected based on their stability performance on the training image. Uniform template selection is achieved based on the distance threshold between features and the number of target features.
[0050] First, the Good Features to Track method is used to extract the center point of the candidate region. Based on this, a small rectangular window is cropped to generate a grayscale template or edge template for local matching verification. If the position of the maximum value on the score map coincides with the center point position and there are no easily confused points with higher scores nearby, it is considered a good template region. A template region may simultaneously meet the requirements of both the edge template and the grayscale template, or it may only meet one of them. When both are met, the edge template is preferred because it provides higher positioning accuracy through sub-pixel precision localization.
[0051]
[0052] The above is the NCC scoring formula used for grayscale matching.
[0053]
[0054] Where x1 i ,y1 i The gradient vector extracted from the Sobel edge at the i-th point on the template edge, x2 i ,y2 i The gradient vector extracted for the Sobel edge at the i-th point on the template edge.
[0055]
[0056] The above is the scoring formula for shape matching, where N is the number of edge points of the template.
[0057] Appendix Figure 2 The letter C region in the above image is used as a template for matching and verification. The image below is a score chart. The highlights are the positions with higher scores. It can be seen that the letter C region is not a good template. The positions of similar-shaped letters such as the letter O and letter S around it have higher scores, which may lead to incorrect positioning.
[0058] When verifying template reliability, firstly, a pixel-by-pixel score is calculated for a local area of a single template in the template image, resulting in a score map. The distribution of high-score points on the score map determines whether a region is a stable template area. Next, during the training phase of the detection template image using multiple sample images for printing inspection, the number of successful positioning attempts and scores for the template on multiple images are counted, and a reliability score is assigned. Finally, based on the distribution of template points and the reliability score, a uniform selection is performed according to the minimum distance requirement.
[0059] Please refer to the appendix for details. Figure 1 , 3As shown, in another preferred embodiment of the present invention, in step two, grid points with a distance of m pixels are also added to the image boundary, and pre-selected local template points are merged to obtain seed points covering the entire image. Triangulation is then performed to obtain a triangular mesh. Two-dimensional grid points with a spacing of n pixels are generated for the entire image, and the IDs of the grid points falling within the triangles are determined and recorded. For image boundary points, the IDs of the k nearest points are recorded.
[0060] Based on the preferred template points and boundary addition points obtained in step one, triangulation is performed using the Delaunay method to obtain a triangular mesh. Sampling mesh points are generated according to the specified sampling mesh spacing, and the triangles to which the mesh points belong are found. All of the above processes are calculated during template creation.
[0061] Appendix Figure 3 To achieve the desired effect in step two, the colored lines represent the triangulated grid of the template points, and the red grid points represent the sampling points for calculating the offset vector.
[0062] Please refer to the appendix for details. Figure 1 , 4 As shown in Figures 5 and 6, in another preferred embodiment of the present invention, in step three, during matching, all local templates are searched. The offset vector of the image boundary point is obtained by interpolation using the offset vectors of neighboring points. Based on the triangle vertex IDs obtained through pre-triangulation, for triangles where all three points are found, the transformation matrix is calculated and the offset vector V(x,y) of the internal grid points is obtained;
[0063] This describes the online execution phase. First, global template localization narrows the search range for each local template. Then, individual local templates are located. For a grid point P0 whose triangle has all three vertices intact, the Affine transformation matrix from the coordinates of the three vertices of the triangle in the training image to the coordinates of the three vertices in the image to be detected is calculated. This transformation matrix is used to transform P0 to obtain P1, and P1 - P0 is the offset vector of point P0. For triangles with missing vertices, the vertex IDs are retained, the corresponding points in the template are extracted and triangulated individually, and the grid points contained in each new triangle are calculated. The offset vectors of these grid points are then obtained using the aforementioned method. This yields the offset vectors of the grid points across the entire image, as shown in the attached diagram. Figure 4 As shown;
[0064] Step 4: Interpolate and enlarge the offset grid to obtain a mapping map of the original resolution;
[0065] Step 5: Using the mapping obtained in Step 4 as the initial vector field, further optimize it using the optical flow method;
[0066] As attached Figure 5As shown, the overall deformation correction is good, but some minor stroke outlines are still not perfectly aligned. For scenarios where the defect size requirements are not stringent, this is sufficient, and step five can be omitted. If higher requirements are needed, steps one through four provide highly reliable vector field results compared to the optical flow method. This avoids the time-consuming process of performing optical flow layer by layer through the pyramid and the unreliable results that may occur with layer-by-layer pyramid optical flow. Furthermore, fine-tuning using optical flow at the original image size can effectively correct even the most subtle deformations, resulting in the final image shown in the attached figure. Figure 6 The results are shown below;
[0067] Step 6: Use the mapping map to remap the image to be tested to obtain the corrected image.
[0068] Furthermore, attached Figure 2 This is a partial pattern on a clothing label, captured using a line scan camera. Due to the inherent stretchability of the fabric, random deformation and stretching inevitably occur during both the printing and image acquisition stages.
[0069] Appendix Figure 3 As shown, a large number of reliable local templates are generated during template training. Furthermore, by sampling grid points, the transformation relationship for each pixel is avoided, thus achieving acceleration.
[0070] Appendix Figure 4 As shown, in the matching of the image to be tested stage, the offset vector field of the sampling grid is first generated, and then the offset vector field of the full image size is obtained by interpolation and magnification.
[0071] like Figure 5 Figure 6 As shown, after obtaining the offset vector field of the entire image size, single-layer optical flow optimization can be performed optionally, or not. Finally, the vector field is used to correct the image under test. The difference map representation in the figure is only for visual observation of the correction effect.
[0072] Although embodiments of the present invention have been shown and described, these specific embodiments are merely explanations of the invention and are not intended to limit it. The specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. After reading this specification, those skilled in the art may make modifications, substitutions, and variations to the embodiments as needed without departing from the principles and spirit of the invention, but such modifications, substitutions, and variations are protected by patent law as long as they are within the scope of the claims of the present invention.
Claims
1. A deformable template matching method for printing inspection, characterized in that, Includes the following steps: Step 1: Template Training and Selection. In the template training stage, local localization templates, including grayscale templates and shape templates, are obtained from the image. The templates are selected based on stability scoring, distance thresholds between features, and the number of target features. Step 2: Triangular mesh construction. Add mesh points on the image boundary, merge pre-selected local template points to obtain seed points covering the entire image, and triangulate them to generate a triangular mesh. Generate a two-dimensional grid of points with an interval of n pixels for the entire image, and record the triangle ID to which each grid point belongs; Step 3: Local matching and offset calculation. In the matching stage, all local templates are searched, the offset vectors of the triangle vertices are calculated, and the offset vectors of the internal grid points are obtained through the transformation matrix. Step 4: Offset interpolation and mapping generation. The offset grid is interpolated and enlarged to obtain a mapping at the original resolution. Step 5: Optical flow optimization. Optimize the mapping map using the optical flow method. Step 6: Image correction. The image to be tested is remapped using a mapping map to obtain a corrected image.
2. The deformable template matching method for printing inspection according to claim 1, characterized in that, The acquisition of the local positioning template in step one specifically includes: extracting the center point of the candidate region using the Good Features to Track method, and cropping a rectangular window to generate a grayscale template or edge template; The stability of the template is verified by the score map. If the position of the maximum value is consistent with the center point and there are no easily confused points around it, it is determined to be a stable template region. Edge templates take precedence over grayscale templates.
3. The deformable template matching method for printing inspection according to claim 2, characterized in that, The matching of grayscale templates uses a normalized cross-correlation scoring formula: Where T is the template image and I is the target image.
4. The deformable template matching method for printing inspection according to claim 2, characterized in that, Shape template matching uses a cosine similarity score based on the edge gradient vectors: Where x1 i ,y1 i The gradient vector extracted from the Sobel edge at the i-th point on the template edge, x2 i ,y2 i The gradient vector extracted for the Sobel edge at the i-th point on the template edge.
5. The deformable template matching method for printing inspection according to claim 4, characterized in that, The final shape matching score is: Where N is the number of edge points of the template.
6. The deformable template matching method for printing inspection according to claim 1, characterized in that, In step two, the Delaunay method is used for triangulation to generate a triangular mesh covering the entire image; For each image boundary point, record the IDs of the k nearest points to it, ensuring that the offset vector of the boundary region is obtained through interpolation.
7. The deformable template matching method for printing inspection according to claim 1, characterized in that, In step three, for a triangle whose three vertices are successfully matched, its Affine transformation matrix is calculated, and the offset vector of the internal grid points is obtained through this matrix. For triangles with missing vertices, extract the corresponding points from the template, re-triangulate them, and calculate the offset vectors of the grid points inside the new triangle.
8. The deformable template matching method for printing inspection according to claim 1, characterized in that, In step four, the low-resolution offset grid is enlarged to the original image resolution using an interpolation method to generate a full-size mapping map.
9. A deformable template matching method for printing inspection according to claim 1, characterized in that, In step five, the optical flow optimization specifically involves: Using the mapping obtained in step four as the initial vector field, single-layer optical flow optimization is performed on the original image size to correct subtle deformations.
10. A deformable template matching method for printing inspection according to claim 1, characterized in that, In step six, the remapping uses bilinear interpolation or bicubic interpolation to correct the image to be tested according to the mapping map, so as to obtain a corrected map that is aligned with the template.