Online printed matter flaw detection method and system based on Internet of Things

Abstract

The invention relates to the technical field of intelligent detection, and discloses a printed matter defect online detection method and system based on the Internet of Things, and a standard printing data construction stage, and the method comprises the steps: carrying out a thermal printing process on a thermal response substrate, and continuously obtaining a printing pattern image according to a second-level sampling time granularity; after the collected printing pattern images are cooled, if the final printing pattern image shows an expected shaping effect, collection is stopped, and a standard image set is reserved; otherwise, abandoning and repeatedly executing the thermal printing process on the thermal response base material; generating a superpixel network and a corresponding superpixel network index for a plurality of images in the standard image set by adopting a multi-level superpixel segmentation technology; the target medium flaw detection stage comprises the following steps: implementing a thermal printing process on a target thermal response substrate, and continuously acquiring target printing pattern images according to second-level sampling time granularity; and generating a corresponding target superpixel network and a target superpixel network index for each frame of target printing pattern image.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection technology, and more specifically, to an online detection method and system for printed product defects based on the Internet of Things. Background Art

[0002] In the existing printing process detection technology, the traditional machine vision system mainly relies on static image acquisition and defect recognition algorithms based on feature matching, and achieves high-precision judgment of printed patterns under standardized and temperature-stable conditions. However, for thermal printing technology, some printing media will show significant thermal expansion and contraction under high-temperature heating environments, causing the printed pattern to undergo a series of complex, nonlinear and time-varying geometric deformation processes during the cooling process, and finally shrink to the expected image. Therefore, the defect detection of the printed pattern requires matching detection of the geometric deformation process, while the traditional machine vision system can only recognize the expected image after shrinkage, but it is difficult to detect defects during the geometric deformation process, resulting in the defects not being discovered in the first place during the geometric deformation process, and a large number of defective prints are produced, reducing the yield rate. Summary of the invention

[0003] The present invention provides an online detection method and system for printed product defects based on the Internet of Things, which solve the technical problems raised in the background technology.

[0004] The present invention provides an online detection method for printed product defects based on the Internet of Things, comprising:

[0005] Step 1, standard printing data construction phase, including:

[0006] Step 11, performing a thermal printing process on the thermally responsive substrate, and continuously acquiring printed pattern images at a sampling time granularity of seconds;

[0007] Step 12, after the collected printing pattern image is cooled, if the final printing pattern image shows the expected finalization effect, then the collection is terminated and retained to form a standard image set; otherwise, it is discarded and the thermal printing process is repeated on the thermal responsive substrate;

[0008] Step 13, for a number of images in the standard image set, a multi-level super-pixel segmentation technology is used to generate a super-pixel network and a corresponding super-pixel network index;

[0009] Step 2, target medium defect detection phase, includes:

[0010] Step 21, performing a thermal printing process on a target thermally responsive substrate, and continuously acquiring a target printed pattern image at a sampling time granularity of seconds;

[0011] Step 22, generating a corresponding target super-pixel network and a target super-pixel network index for each frame of the target printed pattern image based on step 13;

[0012] Step 23, using superpixel indexing, matching the target printed pattern image with the corresponding image in the standard image set;

[0013] Step 231, if the match is successful, the time domain synchronization of the cooling phase is obtained; and the subsequent target printing pattern image is dynamically inspected frame by frame according to the preset rules to determine the defect abnormality state of the thermal printing process of the target thermal response substrate;

[0014] Step 232 , if the matching fails, it is determined that there is an abnormal defect in the thermal printing process of the target thermally responsive substrate.

[0015] Furthermore, the multi-level super-pixel segmentation consists of a pre-processing unit, a pre-segmentation unit, a graph construction unit, a graph clustering unit and a super-pixel network construction unit; wherein the pre-processing unit is used to convert the printed pattern image from the RGB color representation domain to the CIELAB color representation domain to obtain a first-stage processed image, and further denoise the first-stage processed image through anisotropic diffusion filtering to obtain a standard image.

[0016] Furthermore, the pre-segmentation unit comprises:

[0017] The standard image is pre-segmented based on the improved SLIC algorithm, as follows:

[0018] A number of super-pixel centers are evenly distributed to the standard image, and the spacing between adjacent super-pixel centers is determined by the square root of the ratio of the product of the number of horizontal axis pixels and the number of vertical axis pixels of the standard image to the number of super-pixel centers;

[0019] Each superpixel center is assigned a label, which includes: the color component of the superpixel center in the CIELAB color space, and the coordinates of the superpixel center in the standard image;

[0020] For the pixel in the xth row and yth column of the standard image, the distance to the center of each super pixel is calculated respectively. The calculation formula is as follows: ;

[0021] in, represents the distance between the pixel at the xth row and the yth column in the standard image and the center of the cth superpixel, , and Respectively represent the color components of the pixel at the xth row and yth column in the CIELAB color space, , and They represent the color components of the c-th superpixel center in the CIELAB color space, represents the first weight coefficient, represents the second weight coefficient, and , and represents the number of rows and columns of the center of the cth superpixel in the standard image, Represents the gradient value of the pixel in the xth row and yth column in the standard image; where, Based on edge detection of the standard image, the gradient value corresponding to the pixel in the xth row and yth column is obtained;

[0022] The pixels in the xth row and yth column of the standard image are allocated to the super-pixel center of the distance to pre-segment the standard image to obtain a plurality of super-pixels.

[0023] Furthermore, the graph construction unit constructs a weighted undirected graph based on superpixels, including:

[0024] Obtaining an updated label for each superpixel, including: taking the average value of the coordinates of the pixels corresponding to the superpixel in the standard image as the coordinates of the superpixel, and taking the average value of the color components of the pixels corresponding to the superpixel in the CIELAB color space as the color components of the superpixel;

[0025] Map the superpixel to a topological point in a weighted undirected graph, and construct a vector based on the label of the superpixel as the feature vector of the topological point;

[0026] For any two superpixels, if there is a common boundary or the same pixel in the standard image, a topological edge is established between the corresponding topological points;

[0027] For the topological edge established between the i-th superpixel and the j-th superpixel, a weight is assigned through a composite method of Euclidean metric and Gaussian kernel mapping, and the weight is mapped to , the weight calculation formula is as follows: ; ;

[0028] in, represents the weight of the topological edge established between the i-th superpixel and the j-th superpixel, represents the Gaussian smoothing coefficient, represents the superpixel diversity coefficient, Represents the eigenvector of the topological point corresponding to the i-th superpixel, Indicates The feature vector of the topological point corresponding to the superpixel, represents the average value of the gradient value of the pixel corresponding to the i-th super pixel, Indicates The average value of the gradient value of the pixel corresponding to the super pixel, and represents the first difference weight and the second difference weight, Represents an operation that computes the Euclidean distance.

[0029] Furthermore, the graph clustering unit includes:

[0030] Step 51, obtain the adjacency matrix of the weighted undirected graph Sum degree matrix ,include: ,and ;

[0031] in, Degree matrix The element at the mth row and mth column in represents the number of topological points of a weighted undirected graph, express The index of Represents the adjacency matrix The element at row m and column n in Degree matrix The element at the mth row and nth column in ;

[0032] Step 52, based on the adjacency matrix Sum degree matrix , construct the symmetric normalized Laplace matrix ,include: ;

[0033] Step 53, Laplacian matrix Perform eigendecomposition to obtain several eigenvalues; and sort the eigenvalues ​​from small to large based on the size of the eigenvalues ​​to obtain a feature sorting; intercept K eigenvalues ​​from the front to the back of the feature sorting, and merge the eigenvectors corresponding to the K eigenvalues ​​by column to obtain a spectral domain space matrix; wherein the i-th row in the spectral domain space matrix represents the coordinates of the topological point corresponding to the i-th superpixel in the spectral domain space;

[0034] Step 54, establish a spectral domain embedding space; wherein the mth row in the spectral domain space matrix represents the spectral domain coordinates of the topological point corresponding to the mth superpixel in the spectral domain embedding space; define S cluster centers in the spectral domain embedding space, expressed as: ;in, represents the first The cluster vector of the cluster centers, ;

[0035] Step 55, define the fuzzy C-means objective function in the spectral domain embedding space ,as follows: ; ; , ;

[0036] in, express The index of represents the number of cluster centers, and Both said The index of ≠ , Indicates the topological point corresponding to the mth superpixel The membership degree of the cluster centers is represents the fuzzy index, , Represents the topological point corresponding to the mth superpixel and the The Euclidean distance of cluster centers in the spectral domain embedding space, represents the mth row of the spectral domain space matrix, represents the first The cluster vector of the cluster centers;

[0037] Step 56, based on the fuzzy C-means objective function of the topological point corresponding to the m-th superpixel for each cluster center, the topological point corresponding to the m-th superpixel is divided into the cluster center corresponding to the maximum fuzzy C-means objective function value, and the result is clusters; update the cluster vector of the cluster center for each cluster, and the update formula is as follows: ;

[0038] in, Indicates The updated cluster vector of cluster centers, Indicates The number of topological points of cluster centers, express The index of Indicates the topological point corresponding to the qth superpixel The membership degree of the cluster centers is represents the qth row of the spectral domain space matrix;

[0039] Step 57, repeating steps 55 and 56 for a preset number of times, obtaining the membership of each topological point corresponding to each superpixel to the S cluster centers, and constructing a segmentation vector of the topological point corresponding to each superpixel based on the membership of the S cluster centers.

[0040] Furthermore, the super pixel network construction unit includes:

[0041] Step 61, for the topological points with established topological edges, mutual values ​​are calculated based on the segmentation vectors. The mutual values ​​are calculated as follows: ;

[0042] in, represents the mutual value of the topological point corresponding to the mth superpixel and the topological point corresponding to the nth superpixel, Indicates the topological point corresponding to the nth superpixel The membership degree of the cluster centers is Represents the segmentation vector of the topological point corresponding to the mth superpixel, The segmentation vector of the topological point corresponding to the nth superpixel;

[0043] Step 62, if is less than or equal to a first predetermined steady-state threshold, the topological point corresponding to the m-th super-pixel and the topological point corresponding to the n-th super-pixel are merged to obtain a merged super-pixel, and the average value of the segmentation vectors of the topological point corresponding to the m-th super-pixel and the topological point corresponding to the n-th super-pixel is used as the segmentation vector of the merged super-pixel;

[0044] Step 63, looping step 62 until the mutual value of any two super-pixels is greater than the first predetermined steady-state threshold, to obtain a plurality of merged super-pixels; and segmenting the standard image according to the plurality of merged super-pixels, and obtaining a corresponding segmentation topology network as a super-pixel network;

[0045] Step 64, set a second predetermined steady-state threshold, and the first predetermined steady-state threshold is less than the second predetermined steady-state threshold, and loop according to step 62 until the mutual value of any two superpixels is greater than the second predetermined steady-state threshold; and segment the standard image according to a number of merged superpixels, and obtain the corresponding segmentation topology network as the superpixel network index of the corresponding superpixel network.

[0046] Furthermore, matching the target printed pattern image with the corresponding image in the standard image set includes:

[0047] Compare the superpixel network index corresponding to the target printed pattern image with the superpixel network index of each printed pattern image in the standard image set, and calculate the similarity:

[0048] The super-pixel network index is mapped into a Boolean matrix, in which the element 1 represents the structure of the super-pixel network index, and the other elements are all 0; the Hamming distance between the Boolean matrix corresponding to the target printed pattern image and the Boolean matrix corresponding to the printed pattern image is used as the similarity;

[0049] If the similarity of the superpixel network index is greater than the first similarity threshold, the similarity of the corresponding superpixel network is further calculated; if the similarity of the superpixel network is greater than the second similarity threshold, the target printed pattern image is successfully matched.

[0050] Furthermore, the time domain synchronization of the cooling phase indicates that the shrinkage time corresponding to the target printing pattern image is the same as the shrinkage time of the successfully matched printing pattern image in the standard image set.

[0051] Furthermore, preset rules include:

[0052] After the time domain of the cooling phase is synchronized, the superpixel network index of the subsequent target printing pattern image based on the cooling time is compared with the superpixel network index of the printing pattern image with the same cooling time in the standard image set for similarity;

[0053] If the similarity of the superpixel network index is greater than a first similarity threshold, performing a similarity comparison based on the superpixel network index on a subsequent target printed pattern image;

[0054] If the similarity of the superpixel network index is less than the first similarity threshold, the superpixel network similarity comparison is performed again on the target printed pattern image; if the similarity of the superpixel network is less than the second similarity threshold, it is determined that there are defect anomalies in the target printed pattern image; if the similarity of the superpixel network is greater than the second similarity threshold, the similarity comparison based on the superpixel network index is performed on the subsequent target printed pattern image.

[0055] The beneficial effects of the present invention are as follows: by adopting the superpixel segmentation technology, the printed pattern is carefully decomposed into multiple superpixels, and matching is performed based on the constructed superpixel network, thereby greatly simplifying the traditional image recognition process and realizing real-time detection without sacrificing accuracy; at the same time, by utilizing the time domain synchronization mechanism, the printed images with the same shrinkage time are accurately compared, so that defect detection can be achieved during the cooling process of the printed product, thereby accelerating the detection efficiency and significantly improving the production yield. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The present invention is a flowchart of a method for online detection of printed product defects based on the Internet of Things. DETAILED DESCRIPTION

[0057] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0058] like Figure 1 As shown, a method for online detection of printed product defects based on the Internet of Things includes:

[0059] Step 1, standard printing data construction phase, including:

[0060] Step 11, performing a thermal printing process on the thermally responsive substrate, and continuously acquiring printed pattern images at a sampling time granularity of seconds;

[0061] Step 12, after the collected printing pattern image is cooled, if the final printing pattern image shows the expected finalization effect, then the collection is terminated and retained to form a standard image set; otherwise, it is discarded and the thermal printing process is repeated on the thermal responsive substrate;

[0062] Step 13, for a number of images in the standard image set, a multi-level super-pixel segmentation technology is used to generate a super-pixel network and a corresponding super-pixel network index;

[0063] Step 2, target medium defect detection phase, includes:

[0064] Step 21, performing a thermal printing process on a target thermally responsive substrate, and continuously acquiring a target printed pattern image at a sampling time granularity of seconds;

[0065] Step 22, generating a corresponding target super-pixel network and a target super-pixel network index for each frame of the target printed pattern image based on step 13;

[0066] Step 23, using superpixel indexing, matching the target printed pattern image with the corresponding image in the standard image set;

[0067] Step 231, if the match is successful, the time domain synchronization of the cooling phase is obtained; and the subsequent target printing pattern image is dynamically inspected frame by frame according to the preset rules to determine the defect abnormality state of the thermal printing process of the target thermal response substrate;

[0068] Step 232 , if the matching fails, it is determined that there is an abnormal defect in the thermal printing process of the target thermally responsive substrate.

[0069] In one embodiment of the present invention, the multi-level super-pixel segmentation is composed of a pre-processing unit, a pre-segmentation unit, a graph construction unit, a graph clustering unit and a super-pixel network construction unit; wherein the pre-processing unit is used to convert the printed pattern image from the RGB color representation domain to the CIELAB color representation domain to obtain a first-stage processed image, and further denoise the first-stage processed image through anisotropic diffusion filtering to obtain a standard image.

[0070] Specifically, the original RGB color representation of the printed pattern image is first converted to the CIELAB color representation domain through the preprocessing unit. The significance of this step is to use the CIELAB color space to be closer to the human eye's perception of color differences, thereby more accurately expressing the image color information. Subsequently, the anisotropic diffusion filter algorithm is applied to the converted image for noise reduction processing to eliminate the noise and interference that may occur during the image acquisition process, thereby obtaining a clear and standardized image. This standard image provides a solid data foundation for the subsequent use of the pre-segmentation unit, the atlas construction unit, the atlas clustering unit, and the super-pixel network construction unit to achieve multi-level super-pixel segmentation, so that the entire detection system can perform image segmentation and matching detection under the premise of high precision and low error.

[0071] In one embodiment of the present invention, the pre-segmentation unit comprises:

[0072] The standard image is pre-segmented based on the improved SLIC algorithm, as follows:

[0073] A number of super-pixel centers are evenly distributed to the standard image, and the spacing between adjacent super-pixel centers is determined by the square root of the ratio of the product of the number of horizontal axis pixels and the number of vertical axis pixels of the standard image to the number of super-pixel centers;

[0074] Each superpixel center is assigned a label, which includes: the color component of the superpixel center in the CIELAB color space, and the coordinates of the superpixel center in the standard image;

[0075] For the pixel in the xth row and yth column of the standard image, the distance to the center of each super pixel is calculated respectively. The calculation formula is as follows: ;

[0076] in, represents the distance between the pixel at the xth row and the yth column in the standard image and the center of the cth superpixel, , and Respectively represent the color components of the pixel at the xth row and yth column in the CIELAB color space, , and They represent the color components of the c-th superpixel center in the CIELAB color space, represents the first weight coefficient, represents the second weight coefficient, and , and represents the number of rows and columns of the center of the cth superpixel in the standard image, Represents the gradient value of the pixel in the xth row and yth column in the standard image; where, Based on edge detection of the standard image, the gradient value corresponding to the pixel in the xth row and yth column is obtained;

[0077] The pixels in the xth row and yth column of the standard image are allocated to the super-pixel center of the distance to pre-segment the standard image to obtain a plurality of super-pixels.

[0078] Specifically, in the improved SLIC algorithm, the pixels in the standard image are first evenly divided horizontally and vertically, and the ideal spacing between adjacent super-pixel centers is calculated based on the image resolution and the predetermined number of super-pixels. Each super-pixel center is attached with color coordinates in the CIELAB color space and its position information in the image. Subsequently, for any pixel in the image, the distance between it and each super-pixel center is calculated by a formula, in which the difference between the CIELAB color component of the pixel and the corresponding color component of the super-pixel center is modulated by the first weight coefficient, and then acts together with the pixel gradient value obtained based on edge detection (modulated to reflect the image edge effect) and the second weight coefficient to form a comprehensive distance index. The specific value of each weight coefficient should be set according to the actual detection accuracy requirements and image characteristics, so as to ensure that the pixel can be accurately divided to the nearest super-pixel center during the pre-segmentation process.

[0079] In one embodiment of the present invention, the graph construction unit constructs a weighted undirected graph based on superpixels, including:

[0080] Obtaining an updated label for each superpixel, including: taking the average value of the coordinates of the pixels corresponding to the superpixel in the standard image as the coordinates of the superpixel, and taking the average value of the color components of the pixels corresponding to the superpixel in the CIELAB color space as the color components of the superpixel;

[0081] Map the superpixel to a topological point in a weighted undirected graph, and construct a vector based on the label of the superpixel as the feature vector of the topological point;

[0082] For any two superpixels, if there is a common boundary or the same pixel in the standard image, a topological edge is established between the corresponding topological points;

[0083] For the topological edge established between the i-th superpixel and the j-th superpixel, a weight is assigned through a composite method of Euclidean metric and Gaussian kernel mapping, and the weight is mapped to , the weight calculation formula is as follows: ; ;

[0084] in, represents the weight of the topological edge established between the i-th superpixel and the j-th superpixel, represents the Gaussian smoothing coefficient, represents the superpixel diversity coefficient, Represents the eigenvector of the topological point corresponding to the i-th superpixel, Indicates The feature vector of the topological point corresponding to the superpixel, represents the average value of the gradient value of the pixel corresponding to the i-th super pixel, Indicates The average value of the gradient value of the pixel corresponding to the super pixel, and represents the first difference weight and the second difference weight, Represents an operation that computes the Euclidean distance.

[0085] Specifically, first, "updated labels" refers to statistics of all pixels in each superpixel. Specifically, by calculating the arithmetic mean of the coordinates of these pixels in the standard image and the arithmetic mean of the color components in the CIELAB color space, descriptive information that can better reflect the spatial position and color characteristics of the superpixel is obtained. The updated labels are used as the basis for the feature vectors of each topological point in the subsequent construction of the atlas. Secondly, when the superpixel is mapped to a topological point in a weighted undirected graph, each superpixel is regarded as a node in the graph, and its feature vector is composed of the above-mentioned updated labels to quantify the properties of the superpixel in the image. For any two superpixels, when they have a common boundary or the same pixels in the standard image, it is considered that the two superpixels have an adjacent or overlapping relationship, and thus an edge is established between the nodes in the corresponding graph. Here, "the existence of a common boundary or the same pixel" means that in the image segmentation process, the pixel neighborhood or edge detection technology is used to determine whether the two superpixels are directly connected, and the judgment criteria can be determined according to the specific implementation. Finally, for the established topological edge, the distance between the eigenvectors of the corresponding topological points of the two superpixels is calculated by Euclidean metric, and the distance is smoothed and weighted in combination with Gaussian kernel mapping, thereby giving the edge a weight that reflects the similarity of the two superpixels in terms of color, position, etc. The Gaussian smoothing coefficient and superpixel difference coefficient play a regulatory role in this process, and their specific values ​​are set according to the actual application scenario.

[0086] In one embodiment of the present invention, the graph clustering unit includes:

[0087] Step 51, obtain the adjacency matrix of the weighted undirected graph Sum degree matrix ,include: ,and ;

[0088] in, Degree matrix The element at the mth row and mth column in represents the number of topological points of a weighted undirected graph, express The index of Represents the adjacency matrix The element at row m and column n in Degree matrix The element at the mth row and nth column in ;

[0089] Step 52, based on the adjacency matrix Sum degree matrix , construct the symmetric normalized Laplace matrix ,include: ;

[0090] Step 53, Laplacian matrix Perform eigendecomposition to obtain several eigenvalues; and sort the eigenvalues ​​from small to large based on the size of the eigenvalues ​​to obtain a feature sorting; intercept K eigenvalues ​​from the front to the back of the feature sorting, and merge the eigenvectors corresponding to the K eigenvalues ​​by column to obtain a spectral domain space matrix; wherein the i-th row in the spectral domain space matrix represents the coordinates of the topological point corresponding to the i-th superpixel in the spectral domain space;

[0091] Step 54, establish a spectral domain embedding space; wherein the mth row in the spectral domain space matrix represents the spectral domain coordinates of the topological point corresponding to the mth superpixel in the spectral domain embedding space; define S cluster centers in the spectral domain embedding space, expressed as: ;in, represents the first The cluster vector of the cluster centers, ;

[0092] Step 55, define the fuzzy C-means objective function in the spectral domain embedding space ,as follows: ; ; , ;

[0093] in, express The index of represents the number of cluster centers, and Both said The index of ≠ , Indicates the topological point corresponding to the mth superpixel The membership degree of the cluster centers is represents the fuzzy index, , Represents the topological point corresponding to the mth superpixel and the The Euclidean distance of cluster centers in the spectral domain embedding space, represents the mth row of the spectral domain space matrix, represents the first The cluster vector of the cluster centers;

[0094] Step 56, based on the fuzzy C-means objective function of the topological point corresponding to the m-th superpixel for each cluster center, the topological point corresponding to the m-th superpixel is divided into the cluster center corresponding to the maximum fuzzy C-means objective function value, and the result is clusters; update the cluster vector of the cluster center for each cluster, and the update formula is as follows: ;

[0095] in, Indicates The updated cluster vector of cluster centers, Indicates The number of topological points of cluster centers, express The index of Indicates the topological point corresponding to the qth superpixel The membership degree of the cluster centers is represents the qth row of the spectral domain space matrix;

[0096] Step 57, repeating steps 55 and 56 for a preset number of times, obtaining the membership of each topological point corresponding to each superpixel to the S cluster centers, and constructing a segmentation vector of the topological point corresponding to each superpixel based on the membership of the S cluster centers.

[0097] Specifically, in step 51, the acquisition of the adjacency matrix and degree matrix of the weighted undirected graph means that for any two topological points in the graph, the corresponding elements in the adjacency matrix are the weights of the edges between them; and the degree matrix is ​​a diagonal matrix, and each element on its diagonal represents the sum of the weights of all connected edges of the topological point. This expresses the structural information of the spectrum in matrix form, providing a mathematical basis for subsequent spectral analysis. In step 53, the Laplace matrix is ​​eigendecomposed, and the first K eigenvalues ​​and their corresponding eigenvectors are intercepted according to the size of the eigenvalue (from small to large), and the spectral domain space matrix is ​​formed after merging, wherein each row of the matrix represents the coordinates of a super pixel in the spectral domain space. The selection of the K value here should be determined according to information retention and actual application requirements to ensure that the structural characteristics of the spectrum can be fully reflected. In step 54, a spectral domain embedding space is established based on the spectral domain space matrix, and S clustering centers are preset in the space. Each cluster center is represented by a cluster vector, and its initial value can be determined by random selection or other reasonable methods, and used as a reference point in subsequent clustering to achieve effective classification of super pixels. By performing spectral clustering on weighted undirected graphs, super pixels are effectively divided in the spectral domain space, providing an accurate local feature segmentation basis for subsequent printed pattern defect detection.

[0098] In one embodiment of the present invention, the super pixel network construction unit includes:

[0099] Step 61, for the topological points with established topological edges, mutual values ​​are calculated based on the segmentation vectors. The mutual values ​​are calculated as follows: ;

[0100] in, represents the mutual value of the topological point corresponding to the mth superpixel and the topological point corresponding to the nth superpixel, Indicates the topological point corresponding to the nth superpixel The membership degree of the cluster centers is Represents the segmentation vector of the topological point corresponding to the mth superpixel, The segmentation vector of the topological point corresponding to the nth superpixel;

[0101] Step 62, if is less than or equal to a first predetermined steady-state threshold, the topological point corresponding to the m-th super-pixel and the topological point corresponding to the n-th super-pixel are merged to obtain a merged super-pixel, and the average value of the segmentation vectors of the topological point corresponding to the m-th super-pixel and the topological point corresponding to the n-th super-pixel is used as the segmentation vector of the merged super-pixel;

[0102] Step 63, looping step 62 until the mutual value of any two super-pixels is greater than the first predetermined steady-state threshold, to obtain a plurality of merged super-pixels; and segmenting the standard image according to the plurality of merged super-pixels, and obtaining a corresponding segmentation topology network as a super-pixel network;

[0103] Step 64, set a second predetermined steady-state threshold, and the first predetermined steady-state threshold is less than the second predetermined steady-state threshold, and loop according to step 62 until the mutual value of any two superpixels is greater than the second predetermined steady-state threshold; and segment the standard image according to a number of merged superpixels, and obtain the corresponding segmentation topology network as the superpixel network index of the corresponding superpixel network.

[0104] In one embodiment of the present invention, matching the target printed pattern image with the corresponding image in the standard image set includes:

[0105] Compare the superpixel network index corresponding to the target printed pattern image with the superpixel network index of each printed pattern image in the standard image set, and calculate the similarity:

[0106] The super-pixel network index is mapped into a Boolean matrix, in which the element 1 represents the structure of the super-pixel network index, and the other elements are all 0; the Hamming distance between the Boolean matrix corresponding to the target printed pattern image and the Boolean matrix corresponding to the printed pattern image is used as the similarity;

[0107] If the similarity of the superpixel network index is greater than the first similarity threshold, the similarity of the corresponding superpixel network is further calculated; if the similarity of the superpixel network is greater than the second similarity threshold, the target printed pattern image is successfully matched.

[0108] Specifically, the "segmentation vector" refers to a vector output by the atlas clustering unit that reflects the degree of membership of each super-pixel to each cluster center in the spectral domain embedding space, and its value characterizes the attribution of the super-pixel in the feature space. Based on this, the "mutual value" is calculated by calculating the similarity index between any two super-pixel segmentation vectors. If the mutual value is less than or equal to the first predetermined steady-state threshold, it indicates that the corresponding super-pixels are highly similar in performance in the feature space and should be merged; the segmentation vector of the merged super-pixel takes the arithmetic mean of the original segmentation vector as a new comprehensive description. Secondly, steps 62 and 63 require that all super-pixels be cyclically merged until the mutual value between any two super-pixels is greater than the first predetermined steady-state threshold. At this time, the merged result obtained constitutes a preliminary super-pixel network, and the standard image is segmented based on this to obtain the corresponding segmentation topology network. Subsequently, in step 64, a second predetermined steady-state threshold is set, and its value is greater than the first steady-state threshold. This is intended to make a more coarse adjustment to the preliminary merging result, and to perform the merging operation again until the mutual values ​​between all superpixels are greater than the second predetermined steady-state threshold, thereby finally obtaining the superpixel network index of the superpixel network for subsequent processing.

[0109] In one embodiment of the present invention, the time domain synchronization of the cooling phase indicates that the shrinkage time corresponding to the target printing pattern image is the same as the shrinkage time of the successfully matched printing pattern image in the standard image set.

[0110] For example, the shrinkage process of a standard printed pattern lasts for 3 minutes, during which all shrinkage images are collected at a high frequency of seconds; the shrinkage process of the target printed pattern to be detected is theoretically also 3 minutes, but due to factors such as transmission delays in industrial production, the starting time of the image received by the detection device may be deviated. To solve this problem, the detection system compares the superpixel network index of the standard image and the target image, and finds that there is a continuous 58-second image that is almost identical in vision and features, which means that the two have achieved time domain synchronization during this period. After that, the system will perform regular comparisons of subsequent shrinkage stage images at a sampling interval of 15 seconds, thereby ensuring that the target image and the standard image can accurately correspond to each other at the corresponding time during the entire 3-minute shrinkage process. This operation not only achieves time domain synchronization of the cooling phase, but also provides a reliable time reference for subsequent detailed shrinkage state comparisons.

[0111] In one embodiment of the present invention, the preset rules include:

[0112] After the time domain of the cooling phase is synchronized, the superpixel network index of the subsequent target printing pattern image based on the cooling time is compared with the superpixel network index of the printing pattern image with the same cooling time in the standard image set for similarity;

[0113] If the similarity of the superpixel network index is greater than a first similarity threshold, performing a similarity comparison based on the superpixel network index on a subsequent target printed pattern image;

[0114] If the similarity of the superpixel network index is less than the first similarity threshold, the superpixel network similarity comparison is performed again on the target printed pattern image; if the similarity of the superpixel network is less than the second similarity threshold, it is determined that there are defect anomalies in the target printed pattern image; if the similarity of the superpixel network is greater than the second similarity threshold, the similarity comparison based on the superpixel network index is performed on the subsequent target printed pattern image.

[0115] Specifically, first, when the time domain synchronization of the cooling phase is achieved, that is, it is confirmed that the shrinkage time of the target printed pattern is consistent with the shrinkage image time corresponding to the standard image set, the system uses the shrinkage time as a reference. At this time, the printed pattern corresponding to the time period is taken out from the standard image, and its superpixel network index has been constructed in advance. Then, for the target printed pattern image collected subsequently, its superpixel network index is first extracted, and the similarity is compared with the superpixel network index corresponding to the same shrinkage time in the standard image set. If the similarity is greater than the first similarity threshold, it means that the structure and features of the target image and the standard image are basically the same in this time period. At this time, the system continues to use the similarity comparison method based on the superpixel network index to detect subsequent images. However, if the superpixel network index similarity of the target image in the preliminary comparison is lower than the first similarity threshold, it is considered that there is doubt about the preliminary result. At this time, the system will re-perform a more detailed superpixel network similarity comparison on the target image. If the similarity is still lower than the preset second similarity threshold after the comparison of the entire network, it is determined that the target printed pattern image has a defect abnormality; conversely, if the similarity of the entire network is higher than the second similarity threshold, it means that although the initial index comparison result is not ideal, after further verification, the target image and the standard image are still consistent, and the system will then restore the similarity comparison method based on the superpixel network index. In summary, the preset rule uses a hierarchical comparison strategy: first, a quick comparison is performed using the superpixel network index after the cold shrink time synchronization, and then a detailed comparison of the entire network is called when necessary, thereby ensuring the accuracy and reliability of the detection results while ensuring the detection efficiency.

[0116] The above describes an embodiment of the present embodiment, but the present embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present embodiment, ordinary technicians in this field can also make many forms, all of which are within the protection of the present embodiment.

Claims

1. A method for online detection of printed product defects based on the Internet of Things, characterized in that: include: Step 1, standard printing data construction phase, including: Step 11, performing a thermal printing process on the thermally responsive substrate, and continuously acquiring printed pattern images at a sampling time granularity of seconds; Step 12, after the collected printing pattern image is cooled, if the final printing pattern image shows the expected finalization effect, then the collection is terminated and retained to form a standard image set; otherwise, it is discarded and the thermal printing process is repeated on the thermal responsive substrate; Step 13, for a number of images in the standard image set, a multi-level super-pixel segmentation technology is used to generate a super-pixel network and a corresponding super-pixel network index; Step 2, target medium defect detection phase, includes: Step 21, performing a thermal printing process on a target thermally responsive substrate, and continuously acquiring a target printed pattern image at a sampling time granularity of seconds; Step 22, generating a corresponding target super-pixel network and a target super-pixel network index for each frame of the target printed pattern image based on step 13; Step 23, using superpixel indexing, matching the target printed pattern image with the corresponding image in the standard image set; Step 231, if the match is successful, the time domain synchronization of the cooling phase is obtained; and the subsequent target printing pattern image is dynamically inspected frame by frame according to the preset rules to determine the defect abnormality state of the thermal printing process of the target thermal response substrate; Step 232 , if the matching fails, it is determined that there is an abnormal defect in the thermal printing process of the target thermally responsive substrate.

2. The method for online detection of printed product defects based on the Internet of Things according to claim 1, characterized in that: The multi-level superpixel segmentation consists of a preprocessing unit, a pre-segmentation unit, a graph construction unit, a graph clustering unit and a superpixel network construction unit; wherein the preprocessing unit is used to convert the printed pattern image from the RGB color representation domain to the CIELAB color representation domain to obtain the first-stage processed image, and further denoise the first-stage processed image through anisotropic diffusion filtering to obtain a standard image.

3. The online detection method for printed product defects based on the Internet of Things according to claim 2 is characterized in that: Pre-divided unit, comprising: The standard image is pre-segmented based on the improved SLIC algorithm, as follows: A number of super-pixel centers are evenly distributed to the standard image, and the spacing between adjacent super-pixel centers is determined by the square root of the ratio of the product of the number of horizontal axis pixels and the number of vertical axis pixels of the standard image to the number of super-pixel centers; Each superpixel center is assigned a label, which includes: the color component of the superpixel center in the CIELAB color space, and the coordinates of the superpixel center in the standard image; For the pixel in the xth row and yth column of the standard image, the distance to the center of each super pixel is calculated respectively. The calculation formula is as follows: ； in, represents the distance between the pixel at the xth row and the yth column in the standard image and the center of the cth superpixel, , and Respectively represent the color components of the pixel at the xth row and yth column in the CIELAB color space, , and They represent the color components of the c-th superpixel center in the CIELAB color space, represents the first weight coefficient, represents the second weight coefficient, and , and represents the number of rows and columns of the center of the cth superpixel in the standard image, Represents the gradient value of the pixel at the xth row and yth column in the standard image; where, Based on edge detection of the standard image, the gradient value corresponding to the pixel in the xth row and yth column is obtained; The pixels in the xth row and yth column of the standard image are allocated to the super-pixel center of the distance to pre-segment the standard image to obtain a plurality of super-pixels.

4. The online detection method for printed product defects based on the Internet of Things according to claim 3 is characterized in that: The graph construction unit constructs a weighted undirected graph based on superpixels, including: Obtaining an updated label for each superpixel, including: taking the average value of the coordinates of the pixels corresponding to the superpixel in the standard image as the coordinates of the superpixel, and taking the average value of the color components of the pixels corresponding to the superpixel in the CIELAB color space as the color components of the superpixel; Map the superpixel to a topological point in a weighted undirected graph, and construct a vector based on the label of the superpixel as the feature vector of the topological point; For any two superpixels, if there is a common boundary or the same pixel in the standard image, a topological edge is established between the corresponding topological points; For the topological edge established between the i-th superpixel and the j-th superpixel, a weight is assigned through the composite method of Euclidean metric and Gaussian kernel mapping, and the weight is mapped to , the weight calculation formula is as follows: ； ； in, represents the weight of the topological edge established between the i-th superpixel and the j-th superpixel, represents the Gaussian smoothing coefficient, represents the superpixel diversity coefficient, Represents the eigenvector of the topological point corresponding to the i-th superpixel, Indicates The feature vector of the topological point corresponding to the superpixel, represents the average value of the gradient value of the pixel corresponding to the i-th super pixel, Indicates The average value of the gradient value of the pixel corresponding to the super pixel, and represents the first difference weight and the second difference weight, Represents an operation that computes the Euclidean distance.

5. The method for online detection of printed product defects based on the Internet of Things according to claim 4, characterized in that: Graph clustering unit, including: Step 51, obtain the adjacency matrix of the weighted undirected graph Sum degree matrix ,include: ,and ; in, Degree matrix The element at the mth row and mth column in represents the number of topological points of a weighted undirected graph, express The index of Represents the adjacency matrix The element at row m and column n in Degree matrix The element at the mth row and nth column in ; Step 52, based on the adjacency matrix Sum degree matrix , construct the symmetric normalized Laplace matrix ,include: ； Step 53, Laplacian matrix Perform eigendecomposition to obtain several eigenvalues; and sort the eigenvalues ​​from small to large based on the size of the eigenvalues ​​to obtain a feature sorting; intercept K eigenvalues ​​from the front to the back of the feature sorting, and merge the eigenvectors corresponding to the K eigenvalues ​​by column to obtain a spectral domain space matrix; wherein the i-th row in the spectral domain space matrix represents the coordinates of the topological point corresponding to the i-th superpixel in the spectral domain space; Step 54, establish a spectral domain embedding space; wherein the mth row in the spectral domain space matrix represents the spectral domain coordinates of the topological point corresponding to the mth superpixel in the spectral domain embedding space; define S cluster centers in the spectral domain embedding space, expressed as: ;in, represents the first The cluster vector of the cluster centers, ; Step 55, define the fuzzy C-means objective function in the spectral domain embedding space ,as follows: ； ； ， ； in, express The index of represents the number of cluster centers, and Both said The index of ≠ , Indicates the topological point corresponding to the mth superpixel The membership degree of the cluster centers is represents the fuzzy index, , Represents the topological point corresponding to the mth superpixel and the The Euclidean distance of cluster centers in the spectral domain embedding space, represents the mth row of the spectral domain space matrix, represents the first The cluster vector of the cluster centers; Step 56, based on the fuzzy C-means objective function of the topological point corresponding to the m-th superpixel for each cluster center, the topological point corresponding to the m-th superpixel is divided into the cluster center corresponding to the maximum fuzzy C-means objective function value, and the result is clusters; update the cluster vector of the cluster center for each cluster, and the update formula is as follows: ； in, Indicates The updated cluster vector of cluster centers, Indicates The number of topological points of cluster centers, express The index of Indicates the topological point corresponding to the qth superpixel The membership degree of the cluster centers is represents the qth row of the spectral domain space matrix; Step 57, repeating steps 55 and 56 for a preset number of times, obtaining the membership of each topological point corresponding to each superpixel to the S cluster centers, and constructing a segmentation vector of the topological point corresponding to each superpixel based on the membership of the S cluster centers.

6. The online detection method for printed product defects based on the Internet of Things according to claim 5 is characterized in that: Superpixel network building unit, including: Step 61, for the topological points with established topological edges, mutual values ​​are calculated based on the segmentation vectors. The mutual values ​​are calculated as follows: ； in, represents the mutual value of the topological point corresponding to the mth superpixel and the topological point corresponding to the nth superpixel, Indicates the topological point corresponding to the nth superpixel The membership degree of the cluster centers is Represents the segmentation vector of the topological point corresponding to the mth superpixel, The segmentation vector of the topological point corresponding to the nth superpixel; Step 62, if is less than or equal to a first predetermined steady-state threshold, the topological point corresponding to the m-th super-pixel and the topological point corresponding to the n-th super-pixel are merged to obtain a merged super-pixel, and the average value of the segmentation vectors of the topological point corresponding to the m-th super-pixel and the topological point corresponding to the n-th super-pixel is used as the segmentation vector of the merged super-pixel; Step 63, looping step 62 until the mutual value of any two super-pixels is greater than the first predetermined steady-state threshold, to obtain a plurality of merged super-pixels; and segmenting the standard image according to the plurality of merged super-pixels, and obtaining a corresponding segmentation topology network as a super-pixel network; Step 64, set a second predetermined steady-state threshold, and the first predetermined steady-state threshold is less than the second predetermined steady-state threshold, and loop according to step 62 until the mutual value of any two superpixels is greater than the second predetermined steady-state threshold; and segment the standard image according to a number of merged superpixels, and obtain the corresponding segmentation topology network as the superpixel network index of the corresponding superpixel network.

7. The online detection method for printed product defects based on the Internet of Things according to claim 6 is characterized in that: Match the target printed pattern image with the corresponding image in the standard image set, including: Compare the superpixel network index corresponding to the target printed pattern image with the superpixel network index of each printed pattern image in the standard image set, and calculate the similarity: The super-pixel network index is mapped into a Boolean matrix, in which the element 1 represents the structure of the super-pixel network index, and the other elements are all 0; the Hamming distance between the Boolean matrix corresponding to the target printed pattern image and the Boolean matrix corresponding to the printed pattern image is used as the similarity; If the similarity of the superpixel network index is greater than the first similarity threshold, the similarity of the corresponding superpixel network is further calculated; if the similarity of the superpixel network is greater than the second similarity threshold, the target printed pattern image is successfully matched.

8. The online detection method for printed product defects based on the Internet of Things according to claim 7, characterized in that: The time domain synchronization of the cooling phase indicates that the shrinkage time corresponding to the target printing pattern image is the same as the shrinkage time of the successfully matched printing pattern image in the standard image set.

9. The online detection method for printed product defects based on the Internet of Things according to claim 8, characterized in that: Preset rules include: After the time domain of the cooling phase is synchronized, the superpixel network index of the subsequent target printing pattern image based on the cooling time is compared with the superpixel network index of the printing pattern image with the same cooling time in the standard image set for similarity; If the similarity of the superpixel network index is greater than a first similarity threshold, performing a similarity comparison based on the superpixel network index on a subsequent target printed pattern image; If the similarity of the superpixel network index is less than the first similarity threshold, the superpixel network similarity comparison is performed again on the target printed pattern image; if the similarity of the superpixel network is less than the second similarity threshold, it is determined that there are defect anomalies in the target printed pattern image; if the similarity of the superpixel network is greater than the second similarity threshold, the similarity comparison based on the superpixel network index is performed on the subsequent target printed pattern image.

Images