Curved surface printing defect detection method based on rigid and flexible image matching

Abstract

The invention relates to industrial printing defect detection, in particular to a curved surface printing defect detection method based on rigid and flexible image matching, which comprises the following steps of: calibrating a curved surface material, determining a mapping relation between a curved surface image and a plane image, and performing curved surface correction on an image to be detected and a template image; performing rigid image matching on the corrected to-be-detected image and the template image so as to perform optimal rigid transformation on the to-be-detected image and realize primary coarse alignment; inputting the to-be-detected image after rigid transformation and the template image into a pre-trained curved surface printing defect detection model, performing flexible image matching, and detecting curved surface printing defects; according to the technical scheme provided by the invention, the defect of relatively low curved surface printing defect detection precision caused by nonlinear deformation generated by material surface bending and surface characteristic interference of a printing material in the prior art can be effectively overcome.

Description

Technical Field

[0001] The invention relates to industrial printing defect detection, and in particular to a curved surface printing defect detection method based on rigid and flexible image matching. Background Art

[0002] Curved surface printing technology is a widely used printing form in modern manufacturing, especially in the fields of automotive parts, consumer electronics and packaging materials, and has a large market demand. However, due to the complex surface geometry of curved surface printed materials, traditional printing defect detection methods face many challenges, especially for deformation and printing errors during the detection process.

[0003] At present, the existing printing defect detection methods are usually based on plane image matching or template image comparison. Such methods are effective in detecting plane printing defects, but when applied to curved surface printing, due to the geometric deformation caused by the curved surface, such methods are difficult to apply, resulting in a significant decrease in the accuracy and reliability of curved surface printing defect detection. Specifically, the existing printing defect detection methods have major limitations in the following aspects:

[0004] 1) Surface deformation problem: During the printing process, the pattern will produce nonlinear deformation due to the bending of the material surface. It is impossible to accurately align the image only through rigid image matching;

[0005] 2) Surface characteristic interference: Reflection and texture changes on the surface of printed materials can easily cause detection noise, affecting the accuracy of curved surface printing defect detection;

[0006] 3) Diversity of defect types: Curved surface printing may have various types of defects such as color deviation, blur, bubbles, scratches, etc., which are difficult to be fully covered by existing printing defect detection methods.

[0007] In recent years, with the continuous development of computer vision and deep learning technology, defect detection methods based on image matching have gradually attracted attention. Among them, rigid image matching methods (such as model translation, rotation and scaling) are suitable for detecting global deformation of images, while flexible image matching methods (such as optical flow and non-rigid registration) are more suitable for adjusting local deformation of images. However, using rigid or flexible image matching methods alone cannot take into account the global and local image alignment requirements, and it is difficult to meet the high-precision requirements of curved printing defect detection.

[0008] Therefore, there is an urgent need for a curved surface printing defect detection method based on rigid and flexible image matching, which can achieve precise alignment and difference analysis for curved surface printing images, effectively improve the efficiency and robustness of curved surface printing defect detection, and meet the high standards of curved surface printing quality control in industrial production. Summary of the invention

[0009] 1. Technical issues to be resolved

[0010] In view of the above-mentioned shortcomings of the prior art, the present invention provides a curved surface printing defect detection method based on rigid and flexible image matching, which can effectively overcome the defects of the prior art such as low curved surface printing defect detection accuracy caused by nonlinear deformation caused by material surface bending and interference from surface characteristics of printed materials.

[0011] (II) Technical solution

[0012] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0013] A method for detecting defects in curved surface printing based on rigid and flexible image matching comprises the following steps:

[0014] S1. Calibrate the curved surface material, determine the mapping relationship between the curved surface image and the flat surface image, and perform curved surface correction on the image to be tested and the template image;

[0015] S2, performing rigid image matching on the corrected image to be tested and the template image, so as to perform optimal rigid transformation on the image to be tested and realize a rough alignment;

[0016] S3, inputting the image to be tested and the template image after the rigid transformation into a pre-trained curved surface printing defect detection model, performing flexible image matching and detecting curved surface printing defects.

[0017] Preferably, in S1, the curved surface material is calibrated, the mapping relationship between the curved surface image and the flat surface image is determined, and the curved surface correction is performed on the image to be tested and the template image, including:

[0018] S11. Print a suitable grid paper according to the size of the curved material, and stick the grid paper on the surface of the curved material, and take a grid image;

[0019] S12, performing corner point detection on the grid image to determine the coordinates of the corner points of each grid on the curved surface;

[0020] S13, selecting a rectangular area in the grid image, and obtaining the coordinates of the corner points of each grid in the rectangular area on the curved surface;

[0021] S14, determining a mapping relationship between the curved surface image and the flat surface image according to a mapping relationship between the coordinates of the corner points of each grid in the rectangular region on the curved surface and the flat surface;

[0022] S15, performing curved surface correction on the image to be tested and the template image according to the mapping relationship between the curved surface image and the flat surface image, so as to correct the image to be tested and the template image into a flat surface image;

[0023] Among them, the coordinates of the corner points of each grid in the rectangular area on the plane are known.

[0024] Preferably, the curved surface printing defect detection model includes a feature extraction model, an alignment model and a defect detection model;

[0025] Feature extraction model, which consists of a publicly available pre-trained model and is used to extract multi-level features from the input image, including low-level texture features and high-level semantic features, to provide high-quality feature representation for subsequent alignment;

[0026] The alignment model consists of a window-based correlation layer. It realizes flexible image matching through feature matching and correlation calculation within a local window, accurately aligns the input image, eliminates the interference of geometric deformation, and provides stable input for subsequent defect detection;

[0027] The defect detection model, which consists of multiple convolutional layers, concatenates the features of the aligned input images and uses multiple convolutional layers to output the curved surface printing defect detection results.

[0028] Preferably, the feature extraction model extracts multi-level features in the input image to provide high-quality feature representation for subsequent alignment, including:

[0029] S311, input the template image and the image to be tested into the backbone network of the feature extraction model respectively, and obtain the corresponding feature maps F0 and F1 through feature extraction;

[0030] Among them, the size of feature maps F0 and F1 is H and W are the height and width of the input image respectively, and N is the downsampling multiple of the feature extraction model.

[0031] Preferably, the alignment model realizes flexible image matching through feature matching and correlation calculation in a local window, accurately aligns the input image, eliminates the interference of geometric deformation, and provides stable input for subsequent defect detection, including:

[0032] S321, input the feature map F0 of the template image and the feature map F1 of the image to be tested into the alignment model, and divide the feature map into a plurality of local windows W of size K×K ij , each local window corresponds to a center point (i, j), expressed as:

[0033]

[0034] S322, in each local window, by calculating the normalized cross-correlation of the features, the offset between the image to be tested and the template image is estimated:

[0035]

[0036] Among them, Cij (u,v) is the offset between the image to be tested and the template image in the local window, ||·|| represents the norm;

[0037] S323, by finding the position with the largest correlation in each local window, determine the optimal alignment offset of each local window:

[0038] (u * ,v * )=argmaxC ij (u,v);

[0039] Among them, (u * ,v * ) is the position with the largest correlation in the local window;

[0040] S324, construct a global alignment mapping function Φ(x,y) by integrating the optimal alignment offsets of all local windows:

[0041] Φ(x,y)=(x+u * ,y+v * );

[0042] S325, using the global alignment mapping function Φ(x,y) to transform the feature map F1 of the image to be tested, to obtain the feature map F1' of the image to be tested aligned with the feature map F0 of the template image:

[0043] F1'=Φ(F1).

[0044] Preferably, the defect detection model splices the features of the aligned input images and outputs curved surface printing defect detection results using multiple convolutional layers, including:

[0045] S331, concatenate the aligned feature map F1' of the image to be tested and the feature map F0 of the template image in the channel dimension to form a joint feature representation F C ;

[0046] S332, represent the joint feature F C Input multiple convolutional layers to obtain the final curved surface printing defect detection results.

[0047] (III) Beneficial effects

[0048] Compared with the prior art, the curved surface printing defect detection method based on rigid and flexible image matching provided by the present invention has the following beneficial effects:

[0049] 1) The present invention combines rigid and flexible image matching methods, which can simultaneously process global geometric deformation and local subtle deformation in curved surface printing, thereby achieving more accurate image alignment and significantly improving the accuracy of curved surface printing defect detection;

[0050] 2) The present invention can cope with complex material surface geometry and various curved surface printing defects, can work stably under different printing material surface characteristics, and has strong robustness and applicability;

[0051] 3) The present invention reduces manual intervention and manual errors through automated image alignment and efficient defect recognition, improves the accuracy and efficiency of curved surface printing defect detection, and can meet the high standards for curved surface printing quality control in large-scale industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 It is a schematic diagram of the process of the present invention;

[0054] Figure 2 It is a schematic diagram of the process of performing curved surface correction on the image to be tested and the template image in the present invention;

[0055] Figure 3 It is a schematic diagram of the process of performing flexible image matching between the image to be tested and the template image and detecting curved surface printing defects in the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] A curved surface printing defect detection method based on rigid and flexible image matching, such as Figure 1 As shown, S1, calibrate the curved surface material, determine the mapping relationship between the curved surface image and the flat surface image, and perform surface correction on the image to be tested and the template image, such as Figure 2 As shown, specifically including:

[0058] S11. Print a suitable grid paper according to the size of the curved material, and stick the grid paper on the surface of the curved material (try to ensure that every area of ​​the curved material surface is covered with the grid paper), and take a grid image;

[0059] S12, performing corner point detection on the grid image to determine the coordinates of the corner points of each grid on the curved surface;

[0060] S13, selecting a rectangular area in the grid image, and obtaining the coordinates of the corner points of each grid in the rectangular area on the curved surface;

[0061] S14, determining a mapping relationship between the curved surface image and the flat surface image according to a mapping relationship between the coordinates of the corner points of each grid in the rectangular region on the curved surface and the flat surface;

[0062] S15, performing curved surface correction on the image to be tested and the template image according to the mapping relationship between the curved surface image and the flat surface image, so as to correct the image to be tested and the template image into a flat surface image;

[0063] Among them, the coordinates of the corner points of each grid in the rectangular area on the plane are known.

[0064] S2. Perform rigid image matching on the corrected image to be tested and the template image to perform optimal rigid transformation on the image to be tested and achieve a rough alignment.

[0065] S3, inputting the image to be tested and the template image after the rigid transformation into a pre-trained curved surface printing defect detection model, performing flexible image matching and detecting curved surface printing defects.

[0066] In the technical solution of the present application, the curved surface printing defect detection model includes a feature extraction model, an alignment model and a defect detection model;

[0067] Feature extraction model, which consists of publicly available pre-trained models (such as ResNet, EfficientNet, and ViT deep learning models), is used to extract multi-level features from the input image, including low-level texture features and high-level semantic features, providing high-quality feature representation for subsequent alignment;

[0068] The alignment model consists of a window-based correlation layer. It realizes flexible image matching through feature matching and correlation calculation within a local window, accurately aligns the input image, eliminates the interference of geometric deformation, and provides stable input for subsequent defect detection;

[0069] The defect detection model, which consists of multiple convolutional layers, concatenates the features of the aligned input images and uses multiple convolutional layers to output the curved surface printing defect detection results.

[0070] 1) The feature extraction model extracts multi-level features from the input image and provides high-quality feature representation for subsequent alignment, such as Figure 3 As shown, including:

[0071] S311, input the template image and the image to be tested into the backbone network of the feature extraction model respectively, and obtain the corresponding feature maps F0 and F1 through feature extraction;

[0072] Among them, the size of feature maps F0 and F1 is H and W are the height and width of the input image respectively, and N is the downsampling multiple of the feature extraction model.

[0073] 2) The alignment model realizes flexible image matching through feature matching and correlation calculation within the local window, accurately aligns the input image, eliminates the interference of geometric deformation, and provides stable input for subsequent defect detection, such as Figure 3 As shown, including:

[0074] S321, input the feature map F0 of the template image and the feature map F1 of the image to be tested into the alignment model, and divide the feature map into a plurality of local windows W of size K×K ij , each local window corresponds to a center point (i, j), expressed as:

[0075]

[0076] S322, in each local window, by calculating the normalized cross-correlation of the features, the offset between the image to be tested and the template image is estimated:

[0077]

[0078] Among them, C ij (u,v) is the offset between the image to be tested and the template image in the local window, ||·|| represents the norm;

[0079] S323, by finding the position with the largest correlation in each local window, determine the optimal alignment offset of each local window:

[0080] (u * ,v * )=argmaxC ij (u,v);

[0081] Among them, (u * ,v * ) is the position with the largest correlation in the local window;

[0082] S324, construct a global alignment mapping function Φ(x,y) by integrating the optimal alignment offsets of all local windows:

[0083] Φ(x,y)=(x+u * ,y+v * );

[0084] S325, using the global alignment mapping function Φ(x,y) to transform the feature map F1 of the image to be tested, to obtain the feature map F1' of the image to be tested aligned with the feature map F0 of the template image:

[0085] F1'=Φ(F1).

[0086] 3) The defect detection model splices the features of the aligned input images and uses multiple convolutional layers to output the surface printing defect detection results, such as Figure 3 As shown, including:

[0087] S331, concatenate the aligned feature map F1' of the image to be tested and the feature map F0 of the template image in the channel dimension to form a joint feature representation F C ;

[0088] S332, represent the joint feature F C Input multiple convolutional layers to obtain the final curved surface printing defect detection results.

[0089] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting defects in curved surface printing based on matching of rigid and flexible images, characterized in that: The following steps are involved: S1. Calibrate the curved surface material, determine the mapping relationship between the curved surface image and the flat surface image, and perform curved surface correction on the image to be tested and the template image; S2, performing rigid image matching on the corrected image to be tested and the template image, so as to perform optimal rigid transformation on the image to be tested and realize a rough alignment; S3, inputting the image to be tested and the template image after the rigid transformation into a pre-trained curved surface printing defect detection model, performing flexible image matching and detecting curved surface printing defects.

2. The method for detecting defects in curved surface printing based on rigid and flexible image matching according to claim 1, characterized in that: In S1, the curved material is calibrated, the mapping relationship between the curved image and the flat image is determined, and the surface correction is performed on the image to be tested and the template image, including: S11. Print a suitable grid paper according to the size of the curved material, and stick the grid paper on the surface of the curved material, and take a grid image; S12, performing corner point detection on the grid image to determine the coordinates of the corner points of each grid on the curved surface; S13, selecting a rectangular area in the grid image, and obtaining the coordinates of the corner points of each grid in the rectangular area on the curved surface; S14, determining a mapping relationship between the curved surface image and the flat surface image according to a mapping relationship between the coordinates of the corner points of each grid in the rectangular region on the curved surface and the flat surface; S15, performing curved surface correction on the image to be tested and the template image according to the mapping relationship between the curved surface image and the flat surface image, so as to correct the image to be tested and the template image into a flat surface image; Among them, the coordinates of the corner points of each grid in the rectangular area on the plane are known.

3. The method for detecting defects in curved surface printing based on rigid and flexible image matching according to claim 1, characterized in that: The curved surface printing defect detection model includes a feature extraction model, an alignment model and a defect detection model; Feature extraction model, which consists of a publicly available pre-trained model and is used to extract multi-level features from the input image, including low-level texture features and high-level semantic features, to provide high-quality feature representation for subsequent alignment; The alignment model consists of a window-based correlation layer. It realizes flexible image matching through feature matching and correlation calculation within a local window, accurately aligns the input image, eliminates the interference of geometric deformation, and provides stable input for subsequent defect detection; The defect detection model, which consists of multiple convolutional layers, concatenates the features of the aligned input images and uses multiple convolutional layers to output the curved surface printing defect detection results.

4. The method for detecting defects in curved surface printing based on rigid and flexible image matching according to claim 3, characterized in that: The feature extraction model extracts multi-level features from the input image and provides high-quality feature representation for subsequent alignment, including: S311, input the template image and the image to be tested into the backbone network of the feature extraction model respectively, and obtain the corresponding feature maps F0 and F1 through feature extraction; Among them, the size of feature maps F0 and F1 is H and W are the height and width of the input image respectively, and N is the downsampling multiple of the feature extraction model.

5. The method for detecting defects in curved surface printing based on rigid and flexible image matching according to claim 4, characterized in that: The alignment model achieves flexible image matching through feature matching and correlation calculation within a local window, accurately aligns the input image, eliminates the interference of geometric deformation, and provides stable input for subsequent defect detection, including: S321, input the feature map F0 of the template image and the feature map F1 of the image to be tested into the alignment model, and divide the feature map into a plurality of local windows W of size K×K ij , each local window corresponds to a center point (i, j), expressed as: S322, in each local window, by calculating the normalized cross-correlation of the features, the offset between the image to be tested and the template image is estimated: Among them, C ij (u,v) is the offset between the image to be tested and the template image in the local window, ||·|| represents the norm; S323, by finding the position with the largest correlation in each local window, determine the optimal alignment offset of each local window: (u * ,v * )=arg max C ij (u,v); Among them, (u * ,v * ) is the position with the largest correlation in the local window; S324, construct a global alignment mapping function Φ(x,y) by integrating the optimal alignment offsets of all local windows: Φ(x,y)=(x+u * ,y+v * ); S325, using the global alignment mapping function Φ(x,y) to transform the feature map F1 of the image to be tested, to obtain the feature map F1' of the image to be tested aligned with the feature map F0 of the template image: F1'=Φ(F1).

6. The method for detecting defects in curved surface printing based on rigid and flexible image matching according to claim 5, characterized in that: The defect detection model splices the features of the aligned input images and outputs the curved surface printing defect detection results using multiple convolutional layers, including: S331, concatenate the aligned feature map F1' of the image to be tested and the feature map F0 of the template image in the channel dimension to form a joint feature representation F C ; S332, represent the joint feature F C Input multiple convolutional layers to obtain the final curved surface printing defect detection results.

Images