An Online Detection Method and System for Printing Defects Based on the Internet of Things

Abstract

The present invention relates to the field of intelligent detection technology, and discloses an online detection method and system for printing defects based on the Internet of Things. In the standard printing data construction stage, it includes: implementing a thermal printing process on a thermally responsive substrate, and continuously acquiring printing pattern images with a sampling time granularity of seconds; after cooling the acquired printing pattern images, if the final printing pattern image presents an expected shaping effect, terminate the acquisition and retain to form a standard image set; otherwise discard and repeat the thermal printing process on the thermally responsive substrate; for several images in the standard image set, use a multi-level superpixel segmentation technology to generate a superpixel network and the corresponding superpixel network index. In the target medium defect detection stage, it includes: implementing a thermal printing process on the target thermally responsive substrate, and continuously acquiring target printing pattern images with a sampling time granularity of seconds; generating a corresponding target superpixel network and target superpixel network index for each frame of the target printing pattern image.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent detection, and more specifically, it relates to an online detection method and system for printing defects based on the Internet of Things. Background Art

[0002] In the existing printing process detection technology, traditional machine vision systems mainly rely on static image acquisition and defect recognition algorithms based on feature matching, achieving high-precision determination of printed patterns under standardized and temperature-stable conditions. However, for thermal printing technology, some printing media will exhibit significant thermal expansion and contraction phenomena in a high-temperature heating environment, resulting in a series of complex, non-linear, and time-varying geometric deformation processes of the printed pattern during the cooling process, and finally shrinking to the expected image. Therefore, the detection of defects in printed patterns requires matching detection of the geometric deformation process, while traditional machine vision systems can only recognize the expected image after shrinkage, and it is difficult to detect defects during the geometric deformation process, resulting in the inability to discover defects in the first time during the geometric deformation process, and a large number of defective printed products are produced, reducing the yield rate. Summary of the Invention

[0003] The present invention provides an online detection method and system for printing defects based on the Internet of Things to solve the technical problems proposed in the background art.

[0004] The present invention provides an online detection method for printing defects based on the Internet of Things, including:

[0005] Step 1, the standard printing data construction stage, including:

[0006] Step 11, perform a thermal printing process on a heat-responsive substrate, and continuously obtain printed pattern images at a sampling time granularity of seconds;

[0007] Step 12, after the collected printed pattern images are cooled, if the final printed pattern image presents the expected shaping effect, terminate the collection and retain to form a standard image set; otherwise, discard and repeat the thermal printing process on the heat-responsive substrate;

[0008] Step 13, for several images in the standard image set, use a multi-level superpixel segmentation technology to generate a superpixel network and the corresponding superpixel network index;

[0009] Step 2, the target medium defect detection stage, including:

[0010] Step 21, perform a thermal printing process on the target heat-responsive substrate, and continuously obtain target printed pattern images at a sampling time granularity of seconds;

[0011] Step 22: Based on Step 13, generate a corresponding target superpixel network and target superpixel network index for each frame of the target printed pattern image;

[0012] Step 23: Use the superpixel index to match the target printed pattern image with the corresponding image in the standard image set;

[0013] Step 231: If the matching is successful, obtain the time-domain synchronization of the cooling phase; and perform frame-by-frame defect detection on the subsequent target printed pattern images dynamically according to a preset rule to determine the defect abnormal state of the thermal printing process of the target thermally responsive substrate;

[0014] Step 232: If the matching fails, determine that there is a defect abnormality in the thermal printing process of the target thermally responsive substrate.

[0015] Furthermore, the multi-level superpixel segmentation consists of a preprocessing unit, a pre-segmentation unit, a map construction unit, a map clustering unit, and a superpixel network construction unit; among them, the preprocessing unit is used to convert the printed pattern image from the RGB color representation domain to the CIELAB color representation domain to obtain the first-stage processed image, and further perform denoising on the first-stage processed image through anisotropic diffusion filtering to obtain the standard image.

[0016] Furthermore, the pre-segmentation unit includes:

[0017] Pre-segment the standard image based on the improved SLIC algorithm as follows:

[0018] Evenly distribute a number of superpixel centers to the standard image, and the distance between adjacent superpixel centers is determined by the square root of the ratio of the product of the number of horizontal pixels and the number of vertical pixels of the standard image to the number of superpixel centers;

[0019] Assign a label to each superpixel center, and the label includes: the color components of the superpixel center in the CIELAB color space, and the coordinates of the superpixel center in the standard image;

[0020] For the pixel at the x-th row and y-th column in the standard image, calculate the distance from each superpixel center respectively, and the calculation formula is as follows:

[0021] ;

[0022] Among them, represents the distance between the pixel at the x-th row and y-th column in the standard image and the c-th superpixel center, 、 and respectively represent the color components of the pixel at the x-th row and y-th column in the CIELAB color space, 、 and respectively represent the color components of the center of the c-th superpixel in the CIELAB color space, represents the first weight coefficient, represents the second weight coefficient, and , and represent the row number and column number of the center of the c-th superpixel in the standard image, represents the gradient value of the pixel at the x-th row and y-th column in the standard image; where, Based on edge detection of the standard image, the gradient value corresponding to the pixel at the x-th row and y-th column is obtained;

[0023] The pixel at the x-th row and y-th column in the standard image is assigned to the center of the superpixel with the shortest distance to pre-segment the standard image to obtain a number of superpixels.

[0024] Furthermore, the atlas construction unit constructs a weighted undirected graph based on the superpixels, including:

[0025] Obtain the updated label of each superpixel, including: using the average value of the coordinates of the pixels corresponding to the superpixel in the standard image as the coordinates of the superpixel, and using the average value of the color components of the pixels corresponding to the superpixel in the CIELAB color space as the color components of the superpixel;

[0026] Map the superpixel to a topological point in the weighted undirected graph, and construct a vector based on the label of the superpixel as the feature vector of the topological point;

[0027] For any two superpixels, if there is a common boundary or the same pixel in the standard image, establish a topological edge between the corresponding topological points;

[0028] For the topological edge established between the i-th superpixel and the j-th superpixel, assign a weight through the composite method of Euclidean metric and Gaussian kernel mapping, and map the weight to , and the weight calculation formula is as follows:

[0029] ;

[0030] ;

[0031] where, represents the weight of the topological edge established between the i-th superpixel and the j-th superpixel, represents the Gaussian smoothing coefficient, represents the difference coefficient of the superpixel, represents the feature vector of the topological point corresponding to the i-th superpixel, represents the -th feature vector of the topological point corresponding to the superpixel, denotes the average of the gradient values of the pixels corresponding to the \(i\)-th superpixel, denotes the average of the gradient values of the pixels corresponding to the -th superpixel, and denote the first difference weight and the second difference weight,

[0032] Furthermore, the spectral clustering unit includes:

[0033] Step 51: Obtain the adjacency matrix and degree matrix of the weighted undirected graph including: where

[0034] and ;

[0035] wherein, denotes the element in the \(m\)-th row and \(m\)-th column of the degree matrix , denotes the number of topological points of the weighted undirected graph, denotes index of denotes the element in the \(m\)-th row and \(n\)-th column of the adjacency matrix , denotes the element in the \(m\)-th row and \(n\)-th column of the degree matrix ;

[0036] Step 52: Based on the adjacency matrix and the degree matrix , construct a symmetric normalized Laplacian matrix including:

[0037] ;

[0038] Step 53: Perform eigenvalue decomposition on the Laplacian matrix to obtain a number of eigenvalues; and sort the number of eigenvalues from smallest to largest based on the magnitude of the eigenvalues to obtain an eigenvalue sorting; intercept the first \(K\) eigenvalues from the eigenvalue sorting from front to back, and merge the eigenvectors corresponding to the \(K\) eigenvalues by columns to obtain a spectral domain space matrix; wherein, the \(i\)-th row in the spectral domain space matrix represents the coordinates of the topological point corresponding to the \(i\)-th superpixel in the spectral domain space;

[0039] Step 54: Establish a spectral domain embedding space; wherein, the \(m\)-th row in the spectral domain space matrix represents the spectral domain coordinates of the topological point corresponding to the \(m\)-th superpixel in the spectral domain embedding space; define \(S\) clustering centers in the spectral domain embedding space, denoted as: ; wherein, Denote the clustering vector of the th clustering center in the spectral domain embedding space, ;

[0040] Step 55, define the fuzzy C-means objective function in the spectral domain embedding space , as follows:

[0041] ;

[0042] ;

[0043] , ;

[0044] Among them, denotes index of, denotes the number of clustering centers, and both denote index of, ≠ , denotes the membership degree of the topological point corresponding to the m-th superpixel to the th clustering center, denotes the fuzzy index, , denotes the Euclidean distance between the topological point corresponding to the m-th superpixel and the th clustering center in the spectral domain embedding space, denotes the m-th row of the spectral domain space matrix, denotes the clustering vector of the th clustering center in the spectral domain embedding space;

[0045] Step 56, based on the fuzzy C-means objective function of the topological point corresponding to the m-th superpixel for each clustering center, and partition the topological point corresponding to the m-th superpixel into the clustering center corresponding to the maximum fuzzy C-means objective function value, to obtain clusters; update the clustering vector of the clustering center for each cluster, and the update formula is as follows:

[0046] ;

[0047] Among them, denotes the updated clustering vector of the th clustering center, denotes the number of topological points of the th clustering center, denotes index of, denotes the membership degree of the topological point corresponding to the q-th superpixel to the The membership degree of each cluster center represents the q-th row of the spectral domain space matrix;

[0048] Step 57: Repeat Step 55 and Step 56 for a preset number of times to obtain the membership degrees of the topological points corresponding to each superpixel with respect to S cluster centers, and construct a segmentation vector of the topological points corresponding to each superpixel based on the membership degrees of the S cluster centers.

[0049] Furthermore, the superpixel network construction unit includes:

[0050] Step 61: Calculate the mutual value for the topological points with topological edges based on the segmentation vector. The calculation of the mutual value is as follows:

[0051] ;

[0052] where, represents the mutual value between the topological point corresponding to the m-th superpixel and the topological point corresponding to the n-th superpixel, represents the membership degree of the topological point corresponding to the n-th superpixel with respect to the -th cluster center, represents the segmentation vector of the topological point corresponding to the m-th superpixel, the segmentation vector of the topological point corresponding to the n-th superpixel;

[0053] Step 62: If is less than or equal to the first predetermined steady-state threshold, then merge the topological point corresponding to the m-th superpixel and the topological point corresponding to the n-th superpixel to obtain a merged superpixel, and use the average value of the segmentation vectors of the topological point corresponding to the m-th superpixel and the topological point corresponding to the n-th superpixel as the segmentation vector of the merged superpixel;

[0054] Step 63: Loop Step 62 until the mutual value between any two superpixels is greater than the first predetermined steady-state threshold, to obtain several merged superpixels; and segment the standard image according to the several merged superpixels, and obtain the corresponding segmented topological network as the superpixel network;

[0055] Step 64: Set a second predetermined steady-state threshold, and the first predetermined steady-state threshold is less than the second predetermined steady-state threshold, and loop according to Step 62 until the mutual value between any two superpixels is greater than the second predetermined steady-state threshold; and segment the standard image according to the several merged superpixels, and obtain the corresponding segmented topological network as the superpixel network index of the corresponding superpixel network.

[0056] Furthermore, matching the target printed pattern image with the corresponding image in the standard image set includes:

[0057] Compare the super-pixel network index corresponding to the target printed pattern image with the super-pixel network index of each printed pattern image in the standard image set, and calculate the similarity:

[0058] Map the super-pixel network index to a Boolean matrix, where the element 1 in the Boolean matrix represents the structure of the super-pixel network index, and the rest of the elements are 0; take the Hamming distance between the Boolean matrix corresponding to the target printed pattern image and the Boolean matrix corresponding to the printed pattern image as the similarity;

[0059] If the similarity of the super-pixel network index is greater than the first similarity threshold, further calculate the similarity of the corresponding super-pixel network; if the similarity of the super-pixel network is greater than the second similarity threshold, the target printed pattern image is successfully matched.

[0060] Furthermore, the time-domain synchronization of the cooling phase means that the cold shrinkage time corresponding to the target printed pattern image is the same as the cold shrinkage time of the printed pattern image successfully matched in the standard image set.

[0061] Furthermore, the preset rules include:

[0062] After the time-domain synchronization of the cooling phase, perform a similarity comparison between the super-pixel network index of the subsequent target printed pattern image and the super-pixel network index of the printed pattern image with the same cold shrinkage time in the standard image set based on the cold shrinkage time;

[0063] If the similarity of the super-pixel network index is greater than the first similarity threshold, perform a similarity comparison based on the super-pixel network index for the subsequent target printed pattern image;

[0064] If the similarity of the super-pixel network index is less than the first similarity threshold, perform a similarity comparison of the super-pixel network for the target printed pattern image again; if the similarity of the super-pixel network is less than the second similarity threshold, it is determined that there is a defect abnormality in the target printed pattern image; if the similarity of the super-pixel network is greater than the second similarity threshold, perform a similarity comparison based on the super-pixel network index for the subsequent target printed pattern image.

[0065] The beneficial effects of the present invention are as follows: By adopting the super-pixel segmentation technology, the printed pattern is carefully decomposed into multiple super-pixels, and matching is performed based on the constructed super-pixel network, thus greatly simplifying the traditional image recognition process and achieving real-time detection without sacrificing accuracy; at the same time, by using the time-domain synchronization mechanism, the printed images with the same cold shrinkage time are accurately compared, so that defect detection can be realized during the cooling process of the printed product, thereby accelerating the detection efficiency and significantly improving the production yield. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1It is a flowchart of an online detection method for printing defects based on the Internet of Things according to the present invention. Detailed implementation manners

[0067] Now, the subject matter described herein will be discussed with reference to exemplary embodiments. It should be understood that discussing these embodiments is only to enable those skilled in the art to better understand and thus implement the subject matter described herein. Without departing from the protection scope of the content of this specification, changes can be made to the functions and arrangements of the elements discussed. Each example can omit, substitute, or add various processes or components as needed. Additionally, the features described for some examples can also be combined in other examples.

[0068] As Figure 1 shown, an online detection method for printing defects based on the Internet of Things includes:

[0069] Step 1, the standard printing data construction stage, includes:

[0070] Step 11, perform a thermal printing process on a thermally responsive substrate, and continuously obtain printing pattern images with a sampling time granularity of seconds;

[0071] Step 12, after cooling the collected printing pattern images, if the final printing pattern image presents the expected setting effect, terminate the collection and retain to form a standard image set; otherwise discard and repeat the thermal printing process on the thermally responsive substrate;

[0072] Step 13, for several images in the standard image set, use a multi-level superpixel segmentation technique to generate a superpixel network and the corresponding superpixel network index;

[0073] Step 2, the target medium defect detection stage, includes:

[0074] Step 21, perform a thermal printing process on the target thermally responsive substrate, and continuously obtain target printing pattern images with a sampling time granularity of seconds;

[0075] Step 22, for each frame of the target printing pattern image, based on Step 13, generate the corresponding target superpixel network and target superpixel network index;

[0076] Step 23, use the superpixel index to match the target printing pattern image with the corresponding image in the standard image set;

[0077] Step 231, if the matching is successful, obtain the time-domain synchronization of the cooling phase; and perform frame-by-frame defect detection on the subsequent target printing pattern images dynamically according to a preset rule to determine the defect abnormal state of the thermal printing process of the target thermally responsive substrate;

[0078] Step 232, if the matching fails, it is determined that there is a defect abnormality in the thermal printing process of the target thermal response substrate.

[0079] In an embodiment of the present invention, the multi-level superpixel segmentation consists of a preprocessing unit, a pre-segmentation unit, a map construction unit, a map clustering unit, and a superpixel network construction unit; wherein, the preprocessing unit is used to convert the printed pattern image from the RGB color representation domain to the CIELAB color representation domain to obtain the first-stage processed image, and further denoise the first-stage processed image through anisotropic diffusion filtering to obtain a standard image.

[0080] Specifically, first, the original printed pattern image with RGB color representation is converted to the CIELAB color representation domain by the preprocessing unit. The significance of this step is to utilize the CIELAB color space, which is closer to the human eye's perception of color differences, to more accurately express the image color information. Subsequently, the anisotropic diffusion filtering algorithm is applied to the converted image for noise reduction processing to eliminate the noise and interference that may occur during the image acquisition process, thereby obtaining a clear and standardized image. This standard image provides a solid data basis for the subsequent realization of multi-level superpixel segmentation by the pre-segmentation unit, the map construction unit, the map clustering unit, and the superpixel network construction unit, enabling the entire detection system to perform image segmentation and matching detection with high precision and low error.

[0081] In an embodiment of the present invention, the pre-segmentation unit includes:

[0082] Pre-segment the standard image based on the improved SLIC algorithm, specifically as follows:

[0083] Evenly distribute a number of superpixel centers to the standard image, and the distance between adjacent superpixel centers is determined by the square root of the ratio of the product of the number of horizontal pixels and the number of vertical pixels of the standard image to the number of superpixel centers;

[0084] Assign a label to each superpixel center, and the label includes: the color components of the superpixel center in the CIELAB color space, and the coordinates of the superpixel center in the standard image;

[0085] Calculate the distance between the pixel at the x-th row and the y-th column in the standard image and each superpixel center respectively, and the calculation formula is as follows:

[0086] ;

[0087] Wherein, represents the distance between the pixel at the x-th row and the y-th column in the standard image and the c-th superpixel center, 、 and respectively represent the color components of the pixel at the x-th row and y-th column in the CIELAB color space, , and respectively represent the color components of the center of the c-th superpixel in the CIELAB color space, represents the first weight coefficient, represents the second weight coefficient, and , and represent the number of horizontal rows and vertical columns of the center of the c-th superpixel in the standard image, represents the gradient value of the pixel at the x-th row and y-th column in the standard image; wherein, Based on edge detection of the standard image, the gradient value corresponding to the pixel at the x-th row and y-th column is obtained;

[0088] The pixel at the x-th row and y-th column in the standard image is assigned to the center of the superpixel with the shortest distance to pre-segment the standard image to obtain a number of superpixels.

[0089] Specifically, in the improved SLIC algorithm, first, the pixel points in the standard image are evenly divided horizontally and vertically, and the ideal distance between adjacent superpixel centers is calculated based on the image resolution and the predetermined number of superpixels. Each superpixel center is attached with its color coordinates in the CIELAB color space and its position information in the image. Subsequently, for any pixel in the image, the distance between it and each superpixel center is calculated by a formula. In this formula, the difference between the CIELAB color components of the pixel and the corresponding color components of the superpixel center, after being modulated by the first weight coefficient, acts together with the pixel gradient value obtained based on edge detection (modulated to reflect the image edge effect) and the second weight coefficient to form a comprehensive distance index. The specific values of each weight coefficient should be set according to the actual detection accuracy requirements and image characteristics to ensure that pixels can be accurately divided into the superpixel center with the shortest distance during the pre-segmentation process.

[0090] In an embodiment of the present invention, the atlas construction unit constructs a weighted undirected graph based on superpixels, including:

[0091] Obtain the updated label of each superpixel, including: using the average value of the coordinates of the pixels corresponding to the superpixel in the standard image as the coordinates of the superpixel, and using the average value of the color components of the pixels corresponding to the superpixel in the CIELAB color space as the color components of the superpixel;

[0092] Map the superpixel to a topological point in the weighted undirected graph, and construct a vector based on the label of the superpixel as the feature vector of the topological point;

[0093] For any two superpixels, if there is a common boundary or the same pixels in the standard image, a topological edge is established between the corresponding topological points;

[0094] For the topological edge established between the $i$-th superpixel and the $j$-th superpixel, a weight is assigned through the composition of the Euclidean metric and the Gaussian kernel mapping, and the weight is mapped to , and the weight calculation formula is as follows:

[0095] ;

[0096] ;

[0097] where represents the weight of the topological edge established between the $i$-th superpixel and the $j$-th superpixel, represents the Gaussian smoothing coefficient, represents the difference coefficient of the superpixel, represents the feature vector of the topological point corresponding to the $i$-th superpixel, represents the -th feature vector of the topological point corresponding to the superpixel, represents the average value of the gradient values of the pixels corresponding to the $i$-th superpixel, represents the -th average value of the gradient values of the pixels corresponding to the superpixel, and represent the first difference weight and the second difference weight, represents the operation of calculating the Euclidean distance.

[0098] Specifically, first, "updating the label" means statistically analyzing all pixels within each superpixel. Specifically, by calculating the arithmetic mean of the coordinates of these pixels in the standard image and the arithmetic mean of the color components in the CIELAB color space, descriptive information that can better reflect the spatial position and color characteristics of the superpixel is obtained. This updated label is used as the basis for the feature vectors of each topological point when constructing the atlas subsequently. Secondly, when mapping a superpixel to a topological point in a weighted undirected graph, each superpixel is regarded as a node in the graph, and its feature vector is composed of the above-mentioned updated label, which is used to quantify the attributes of the superpixel in the image. For any two superpixels, when they have a common boundary or the same pixels in the standard image, it is considered that these two superpixels have an adjacent or overlapping relationship, so an edge is established between the corresponding nodes in the graph. Here, "having a common boundary or the same pixels" means determining whether two superpixels are directly connected through pixel neighborhood or edge detection techniques during the image segmentation process, and its determination criterion can be determined according to specific implementations. Finally, for the established topological edges, the distance between the feature vectors of the corresponding topological points of two superpixels is calculated through the Euclidean metric, and the distance is smoothed and weighted by combining the Gaussian kernel mapping, thereby assigning a weight to the edge, which reflects the similarity between the two superpixels in terms of color, position, etc. The Gaussian smoothing coefficient and the superpixel difference coefficient play a regulatory role in this process, and their specific values are set according to the actual application scenario.

[0099] In an embodiment of the present invention, the atlas clustering unit includes:

[0100] Step 51, obtaining the adjacency matrix of the weighted undirected graph and the degree matrix , including:

[0101] , and ;

[0102] Wherein, represents the element in the m-th row and m-th column of the degree matrix , represents the number of topological points of the weighted undirected graph, represents 's index, represents the element in the m-th row and n-th column of the adjacency matrix , represents the element in the m-th row and n-th column of the degree matrix ;

[0103] Step 52, based on the adjacency matrix and the degree matrix , constructing a symmetric normalized Laplacian matrix , including:

[0104] ;

[0105] Step 53, perform eigenvalue decomposition on the Laplacian matrix to obtain a number of eigenvalues; and sort the number of eigenvalues from smallest to largest based on the magnitudes of the eigenvalues to obtain an eigenvalue sorting; intercept the first K eigenvalues from the eigenvalue sorting from front to back, and respectively combine the eigenvectors corresponding to the K eigenvalues column by column to obtain a spectral domain space matrix; wherein, the i-th row in the spectral domain space matrix represents the coordinates of the topological point corresponding to the i-th superpixel in the spectral domain space;

[0106] Step 54, establish a spectral domain embedding space; wherein, the m-th row in the spectral domain space matrix represents the spectral domain coordinates of the topological point corresponding to the m-th superpixel in the spectral domain embedding space; define S clustering centers in the spectral domain embedding space, expressed as: ; wherein, represents the clustering vector of the -th clustering center in the spectral domain embedding space, ;

[0107] Step 55, define a fuzzy C-means objective function in the spectral domain embedding space, as follows:

[0108] ;

[0109] ;

[0110] , ;

[0111] wherein, represents 's index, represents the number of clustering centers, and both represent 's index, ≠ , represents the membership degree of the topological point corresponding to the m-th superpixel to the -th clustering center, represents the fuzzy index, , represents the Euclidean distance between the topological point corresponding to the m-th superpixel and the -th clustering center in the spectral domain embedding space, represents the m-th row of the spectral domain space matrix, represents the -th clustering center's clustering vector in the spectral domain embedding space;

[0112] Step 56: Based on the topological points corresponding to the m-th superpixel, calculate the fuzzy C-means objective function for each cluster center, and divide the topological points corresponding to the m-th superpixel into the cluster center corresponding to the maximum fuzzy C-means objective function value, obtaining clusters; update the cluster vectors of the cluster centers for each cluster, and the update formula is as follows:

[0113] ;

[0114] where, represents the updated cluster vector of the -th cluster center, represents the number of topological points of the -th cluster center, represents index, represents the membership degree of the topological point corresponding to the q-th superpixel to the -th cluster center, represents the q-th row of the spectral domain space matrix;

[0115] Step 57: Repeat Step 55 and Step 56 for a preset number of times to obtain the membership degrees of the topological points corresponding to each superpixel to S cluster centers respectively, and construct the segmentation vector of the topological points corresponding to each superpixel based on the membership degrees of the S cluster centers.

[0116] Specifically, in Step 51, the obtaining of the adjacency matrix and degree matrix of the weighted undirected graph means that for any two topological points in the graph, the corresponding element in the adjacency matrix is the weight of the edge between them; and the degree matrix is a diagonal matrix, and each element on the diagonal represents the sum of the weights of all the connected edges of the topological point. This expresses the structural information of the graph spectrum in matrix form and provides a mathematical basis for subsequent spectral analysis. In Step 53, the Laplacian matrix is eigen-decomposed, and the first K eigenvalues and their corresponding eigenvectors are intercepted according to the eigenvalue magnitudes (from small to large) and merged to form the spectral domain space matrix, where each row of the matrix represents the coordinates of a superpixel in the spectral domain space. The selection of the K value should be determined according to information retention and actual application requirements to ensure that the structural characteristics of the graph spectrum can be fully reflected. In Step 54, a spectral domain embedding space is established based on the spectral domain space matrix, and S cluster centers are preset in this space. Each cluster center is represented by a cluster vector, and its initial value can be determined by random selection or other reasonable methods and is used as a reference point in subsequent clustering to effectively classify the superpixels. By performing spectral clustering on the weighted undirected graph, the superpixels are effectively divided in the spectral domain space, providing an accurate local feature segmentation basis for subsequent printed pattern defect detection.

[0117] In an embodiment of the present invention, the superpixel network construction unit includes:

[0118] Step 61: Calculate the mutual value for the topological points with topological edges established based on the segmentation vector. The calculation of the mutual value is as follows:

[0119] ;

[0120] Wherein, represents the mutual value between the topological point corresponding to the m-th superpixel and the topological point corresponding to the n-th superpixel, represents the membership degree of the topological point corresponding to the n-th superpixel to the -th clustering center, represents the segmentation vector of the topological point corresponding to the m-th superpixel, the segmentation vector of the topological point corresponding to the n-th superpixel;

[0121] Step 62: If is less than or equal to the first predetermined steady-state threshold, then merge the topological point corresponding to the m-th superpixel and the topological point corresponding to the n-th superpixel to obtain a merged superpixel, and use the average value of the segmentation vectors of the topological point corresponding to the m-th superpixel and the topological point corresponding to the n-th superpixel as the segmentation vector of the merged superpixel;

[0122] Step 63: Loop step 62 until the mutual value between any two superpixels is greater than the first predetermined steady-state threshold, to obtain several merged superpixels; and segment the standard image according to the several merged superpixels, and obtain the corresponding segmented topological network as the superpixel network;

[0123] Step 64: Set the second predetermined steady-state threshold, and the first predetermined steady-state threshold is less than the second predetermined steady-state threshold. Loop according to step 62 until the mutual value between any two superpixels is greater than the second predetermined steady-state threshold; and segment the standard image according to the several merged superpixels, and obtain the corresponding segmented topological network as the superpixel network index of the corresponding superpixel network.

[0124] In an embodiment of the present invention, matching the target printed pattern image with the corresponding image in the standard image set includes:

[0125] Compare the superpixel network index corresponding to the target printed pattern image with the superpixel network indexes of each printed pattern image in the standard image set, and calculate the similarity:

[0126] Map the superpixel network index to a Boolean matrix, where the element 1 in the Boolean matrix represents the structure of the superpixel network index, and the rest of the elements are all 0; use the Hamming distance between the Boolean matrix corresponding to the target printed pattern image and the Boolean matrix corresponding to the printed pattern image as the similarity;

[0127] If the similarity of the super-pixel network index is greater than the first similarity threshold, the similarity of the corresponding super-pixel network is further calculated; if the similarity of the super-pixel network is greater than the second similarity threshold, the target printed pattern image matching is successful.

[0128] Specifically, the "segmentation vector" refers to the vector output by the spectral clustering unit, which reflects the membership degree of each super-pixel in the spectral domain embedding space to each clustering center, and its value characterizes the attribution of the super-pixel in the feature space. Based on this, the "mutual value" is the similarity index calculated between the segmentation vectors of any two super-pixels. If the mutual value is less than or equal to the first predetermined steady-state threshold, it indicates that the corresponding super-pixels are highly similar in the feature space and should be merged; the segmentation vector of the merged super-pixel takes the arithmetic mean of the original segmentation vectors as the new comprehensive description. Secondly, steps 62 and 63 require cyclic merging of all super-pixels until the mutual value between any two super-pixels is greater than the first predetermined steady-state threshold. At this time, the obtained merging result constitutes a preliminary super-pixel network, and the standard image is segmented accordingly to obtain the corresponding segmentation topology network. Subsequently, in step 64, a second predetermined steady-state threshold is set, and its value is greater than the first steady-state threshold. This is aimed at making a more rough adjustment to the preliminary merging result, and the merging operation is performed again until the mutual value between all super-pixels is greater than the second predetermined steady-state threshold, so as to finally obtain the super-pixel network index of the super-pixel network for subsequent processing.

[0129] In an embodiment of the present invention, the time-domain synchronization of the cooling phase means that the cold shrinkage time corresponding to the target printed pattern image is the same as the cold shrinkage time of the printed pattern image successfully matched in the standard image set.

[0130] For example, for a standard printed pattern, its cold shrinkage process lasts for 3 minutes, and all cold shrinkage images are collected at a high frequency in seconds during this period; while the cold shrinkage process of the target printed pattern to be detected is theoretically also 3 minutes, but due to factors such as transmission delay in industrial production, the starting time of the images received by the detection device may be deviated. To solve this problem, the detection system compares the super-pixel network indexes of the standard image and the target image, and finds that there is a continuous 58-second image that is almost the same visually and in terms of features, that is, it is considered that the time-domain synchronization has been achieved during this period. After that, the system performs timed comparison on the subsequent cold shrinkage stage images at a sampling interval of 15 seconds, so as to ensure that during the entire 3-minute cold shrinkage process, the target image and the standard image can accurately correspond at the corresponding moments. Such an operation not only realizes the time-domain synchronization of the cooling phase, but also provides a reliable time reference for the subsequent detailed comparison of the cold shrinkage state.

[0131] In an embodiment of the present invention, the preset rules include:

[0132] After the time domain synchronization of the cooling phase, the super-pixel network index of the subsequent target printed pattern image is compared with the super-pixel network index of the printed pattern image with the same cold shrinkage time in the standard image set based on the cold shrinkage time;

[0133] If the similarity of the super-pixel network index is greater than the first similarity threshold, the similarity comparison based on the super-pixel network index is performed on the subsequent target printed pattern image;

[0134] If the similarity of the super-pixel network index is less than the first similarity threshold, the similarity comparison of the super-pixel network is performed again on the target printed pattern image; if the similarity of the super-pixel network is less than the second similarity threshold, it is determined that there is a defect abnormality in the target printed pattern image; if the similarity of the super-pixel network is greater than the second similarity threshold, the similarity comparison based on the super-pixel network index is performed on the subsequent target printed pattern image.

[0135] Specifically, first, after the time domain synchronization of the cooling phase is achieved, that is, when it is confirmed that the cold shrinkage time of the target printed pattern is consistent with the cold shrinkage image time corresponding in the standard image set, the system uses this cold shrinkage time as a reference. At this time, the printed pattern corresponding to this time period is taken out from the standard image, and its super-pixel network index has been constructed in advance. Then, for the subsequently collected target printed pattern image, first extract its super-pixel network index and compare it with the super-pixel network index corresponding to the same cold shrinkage time in the standard image set. If the similarity is greater than the first similarity threshold, it means that the structure and features of the target image and the standard image are basically the same during this time period. At this time, the system continues to use the similarity comparison method based on the super-pixel network index to detect the subsequent images. However, if the similarity of the super-pixel network index of the target image is lower than the first similarity threshold in the preliminary comparison, it is considered that the preliminary result is doubtful. At this time, the system will re-perform a more detailed similarity comparison of the super-pixel network on this target image. If the similarity is still lower than the preset second similarity threshold after the comparison of the entire network, it is determined that there is a defect abnormality in the target printed pattern image; on the contrary, if the similarity of the entire network is higher than the second similarity threshold, it means that although the preliminary index comparison result is not ideal, after further verification, the target image still matches the standard image. After that, the system will resume the similarity comparison method based on the super-pixel network index. In summary, this preset rule adopts a hierarchical comparison strategy: first, use the super-pixel network index after cold shrinkage time synchronization for rapid comparison, and then call the detailed comparison of the entire network when necessary, so as to ensure the accuracy and reliability of the detection result while ensuring the detection efficiency.

[0136] The above has described the embodiments of this example, but this example is not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative rather than restrictive. Under the inspiration of this example, those of ordinary skill in the art can also make many forms, all of which fall within the protection scope of this example.

Claims

1. A method for online detection of printed product defects based on the Internet of Things, characterized in that: include: Step 1, standard printing data construction phase, including: Step 11, performing a thermal printing process on the thermally responsive substrate, and continuously acquiring printed pattern images at a sampling time granularity of seconds; Step 12, after the collected printing pattern image is cooled, if the final printing pattern image shows the expected finalization effect, then the collection is terminated and retained to form a standard image set; otherwise, it is discarded and the thermal printing process is repeated on the thermal responsive substrate; Step 13, for several images in the standard image set, a multi-level superpixel segmentation technology is used to generate a superpixel network and a corresponding superpixel network index, and the multi-level superpixel segmentation is composed of a preprocessing unit, a pre-segmentation unit, a graph construction unit, a graph clustering unit and a superpixel network construction unit; the graph construction unit constructs a weighted undirected graph based on the superpixel, including: obtaining an updated label for each superpixel, including: taking the average value of the coordinates of the pixels corresponding to the superpixel in the standard image as the coordinates of the superpixel, and taking the average value of the color components of the pixels corresponding to the superpixel in the CIELAB color space as the color components of the superpixel; mapping the superpixel to a topological point in the weighted undirected graph, and constructing a vector based on the label of the superpixel as the feature vector of the topological point; for any two superpixels, if there are common boundaries or the same pixels in the standard image, a topological edge is established between the corresponding topological points; for the topological edge established between the i-th superpixel and the j-th superpixel, a weight is assigned by a composite method of Euclidean metric and Gaussian kernel mapping, and the weight is mapped to ; Step 2, target medium defect detection phase, includes: Step 21, performing a thermal printing process on a target thermally responsive substrate, and continuously acquiring a target printed pattern image at a sampling time granularity of seconds; Step 22, generating a corresponding target super-pixel network and a target super-pixel network index for each frame of the target printed pattern image based on step 13; Step 23, using superpixel indexing, matching the target printed pattern image with the corresponding image in the standard image set; Step 231, if the match is successful, the time domain synchronization of the cooling phase is obtained; and the subsequent target printing pattern image is dynamically inspected frame by frame according to the preset rules to determine the defect abnormality state of the thermal printing process of the target thermal response substrate; Step 232 , if the matching fails, it is determined that there is an abnormal defect in the thermal printing process of the target thermally responsive substrate.

2. The method for online detection of printed product defects based on the Internet of Things according to claim 1, characterized in that: The preprocessing unit is used to convert the printed pattern image from the RGB color representation domain to the CIELAB color representation domain to obtain a first-stage processed image, and further denoise the first-stage processed image through anisotropic diffusion filtering to obtain a standard image.

3. The online detection method for printed product defects based on the Internet of Things according to claim 2 is characterized in that: Pre-divided unit, comprising: The standard image is pre-segmented based on the improved SLIC algorithm, as follows: A number of super-pixel centers are evenly distributed to the standard image, and the spacing between adjacent super-pixel centers is determined by the square root of the ratio of the product of the number of horizontal axis pixels and the number of vertical axis pixels of the standard image to the number of super-pixel centers; Each superpixel center is assigned a label, which includes: the color component of the superpixel center in the CIELAB color space, and the coordinates of the superpixel center in the standard image; For the pixel in the xth row and yth column of the standard image, the distance to the center of each super pixel is calculated respectively. The calculation formula is as follows: ； in, represents the distance between the pixel at the xth row and the yth column in the standard image and the center of the cth superpixel, , and Respectively represent the color components of the pixel at the xth row and yth column in the CIELAB color space, , and They represent the color components of the c-th superpixel center in the CIELAB color space, represents the first weight coefficient, represents the second weight coefficient, and , and represents the number of rows and columns of the center of the cth superpixel in the standard image, Represents the gradient value of the pixel at the xth row and yth column in the standard image; where, Based on edge detection of the standard image, the gradient value corresponding to the pixel in the xth row and yth column is obtained; The pixels in the xth row and yth column of the standard image are allocated to the super-pixel center of the distance to pre-segment the standard image to obtain a plurality of super-pixels.

4. The online detection method for printed product defects based on the Internet of Things according to claim 3 is characterized in that: The weight calculation formula is as follows: ； ； in, represents the weight of the topological edge established between the i-th superpixel and the j-th superpixel, represents the Gaussian smoothing coefficient, represents the superpixel diversity coefficient, Represents the eigenvector of the topological point corresponding to the i-th superpixel, Indicates The feature vector of the topological point corresponding to the superpixel, represents the average value of the gradient value of the pixel corresponding to the i-th super pixel, Indicates The average value of the gradient value of the pixel corresponding to the super pixel, and represents the first difference weight and the second difference weight, Represents an operation that computes the Euclidean distance.

5. The online detection method for printed product defects based on the Internet of Things according to claim 4 is characterized in that: Graph clustering unit, including: Step 51, obtain the adjacency matrix of the weighted undirected graph Sum degree matrix ,include: ,and ; in, Degree matrix The element at the mth row and mth column in represents the number of topological points of a weighted undirected graph, express The index of Represents the adjacency matrix The element at row m and column n in Degree matrix The element at the mth row and nth column in ; Step 52, based on the adjacency matrix Sum degree matrix , construct the symmetric normalized Laplace matrix ,include: ； Step 53, Laplacian matrix Perform eigendecomposition to obtain several eigenvalues; and sort the eigenvalues ​​from small to large based on the size of the eigenvalues ​​to obtain a feature sorting; intercept K eigenvalues ​​from the front to the back of the feature sorting, and merge the eigenvectors corresponding to the K eigenvalues ​​by column to obtain a spectral domain space matrix; wherein the i-th row in the spectral domain space matrix represents the coordinates of the topological point corresponding to the i-th superpixel in the spectral domain space; Step 54, establish a spectral domain embedding space; wherein the mth row in the spectral domain space matrix represents the spectral domain coordinates of the topological point corresponding to the mth superpixel in the spectral domain embedding space; define S cluster centers in the spectral domain embedding space, expressed as: ;in, represents the first The cluster vector of the cluster centers, ; Step 55, define the fuzzy C-means objective function in the spectral domain embedding space ,as follows: ； ； ， ； in, express The index of represents the number of cluster centers, and Both said The index of ≠ , Indicates the topological point corresponding to the mth superpixel The membership degree of the cluster centers is represents the fuzzy index, , Represents the topological point corresponding to the mth superpixel and the The Euclidean distance of cluster centers in the spectral domain embedding space, represents the mth row of the spectral domain space matrix, represents the first The cluster vector of the cluster centers; Step 56, based on the fuzzy C-means objective function of the topological point corresponding to the m-th superpixel for each cluster center, the topological point corresponding to the m-th superpixel is divided into the cluster center corresponding to the maximum fuzzy C-means objective function value, and the result is clusters; update the cluster vector of the cluster center for each cluster, and the update formula is as follows: ； in, Indicates The updated cluster vector of cluster centers, Indicates The number of topological points of cluster centers, express The index of Indicates the topological point corresponding to the qth superpixel The membership degree of the cluster centers is represents the qth row of the spectral domain space matrix; Step 57, repeating steps 55 and 56 for a preset number of times, obtaining the membership of each topological point corresponding to each superpixel to the S cluster centers, and constructing a segmentation vector of the topological point corresponding to each superpixel based on the membership of the S cluster centers.

6. The method for online detection of printed product defects based on the Internet of Things according to claim 5, characterized in that: Superpixel network building unit, including: Step 61, for the topological points with established topological edges, mutual values ​​are calculated based on the segmentation vectors. The mutual values ​​are calculated as follows: ； in, represents the mutual value of the topological point corresponding to the mth superpixel and the topological point corresponding to the nth superpixel, Indicates the topological point corresponding to the nth superpixel The membership degree of the cluster centers is Represents the segmentation vector of the topological point corresponding to the mth superpixel, The segmentation vector of the topological point corresponding to the nth superpixel; Step 62, if is less than or equal to a first predetermined steady-state threshold, the topological point corresponding to the m-th super-pixel and the topological point corresponding to the n-th super-pixel are merged to obtain a merged super-pixel, and the average value of the segmentation vectors of the topological point corresponding to the m-th super-pixel and the topological point corresponding to the n-th super-pixel is used as the segmentation vector of the merged super-pixel; Step 63, looping step 62 until the mutual value of any two super-pixels is greater than the first predetermined steady-state threshold, to obtain a plurality of merged super-pixels; and segmenting the standard image according to the plurality of merged super-pixels, and obtaining a corresponding segmentation topology network as a super-pixel network; Step 64, set a second predetermined steady-state threshold, and the first predetermined steady-state threshold is less than the second predetermined steady-state threshold, and loop according to step 62 until the mutual value of any two superpixels is greater than the second predetermined steady-state threshold; and segment the standard image according to a number of merged superpixels, and obtain the corresponding segmentation topology network as the superpixel network index of the corresponding superpixel network.

7. The online detection method for printed product defects based on the Internet of Things according to claim 6 is characterized in that: Match the target printed pattern image with the corresponding image in the standard image set, including: Compare the superpixel network index corresponding to the target printed pattern image with the superpixel network index of each printed pattern image in the standard image set, and calculate the similarity: The super-pixel network index is mapped into a Boolean matrix, in which the element 1 represents the structure of the super-pixel network index, and the other elements are all 0; the Hamming distance between the Boolean matrix corresponding to the target printed pattern image and the Boolean matrix corresponding to the printed pattern image is used as the similarity; If the similarity of the superpixel network index is greater than the first similarity threshold, the similarity of the corresponding superpixel network is further calculated; if the similarity of the superpixel network is greater than the second similarity threshold, the target printed pattern image is successfully matched.

8. The method for online detection of printed matter defects based on the Internet of Things according to claim 7, characterized in that: The time domain synchronization of the cooling phase indicates that the shrinkage time corresponding to the target printing pattern image is the same as the shrinkage time of the successfully matched printing pattern image in the standard image set.

9. The online detection method for printed product defects based on the Internet of Things according to claim 8, characterized in that: Preset rules include: After the time domain of the cooling phase is synchronized, the superpixel network index of the subsequent target printing pattern image based on the cooling time is compared with the superpixel network index of the printing pattern image with the same cooling time in the standard image set for similarity; If the similarity of the superpixel network index is greater than a first similarity threshold, performing a similarity comparison based on the superpixel network index on a subsequent target printed pattern image; If the similarity of the superpixel network index is less than the first similarity threshold, the superpixel network similarity comparison is performed again on the target printed pattern image; if the similarity of the superpixel network is less than the second similarity threshold, it is determined that there are defect anomalies in the target printed pattern image; if the similarity of the superpixel network is greater than the second similarity threshold, the similarity comparison based on the superpixel network index is performed on the subsequent target printed pattern image.

Images