An online printing quality detection method based on artificial intelligence

Abstract

The present invention belongs to the technical field of quality inspection, and discloses an online printing quality inspection method based on artificial intelligence. The method includes: collecting a printed matter image; preprocessing the printed matter image to obtain a preprocessed image; obtaining a reference image corresponding to the printed matter image; aligning the preprocessed image with the reference image, calculating an overall offset amount, and determining whether to generate an overall offset instruction; if no overall offset instruction is generated, performing image segmentation on the preprocessed image to obtain a segmented image; performing key point matching on the segmented image, calculating a local offset amount, and determining whether to generate a local offset instruction. The present invention can automatically identify pattern offsets during the printing process, reduce manual intervention and errors, and reduce the rejection rate, thereby improving the production efficiency and product quality of printed matter.

Description

Technical Field

[0001] The present invention relates to the technical field of quality inspection, and more specifically, to an online printing quality inspection method based on artificial intelligence. Background Art

[0002] In the modern printing industry, with the continuous improvement of the requirements for the quality of printed products, the online inspection of printing quality has become a key link to ensure product quality, reduce the scrap rate, and improve production efficiency. Most traditional printing quality inspection methods rely on manual sampling inspection or simple visual inspection equipment, which have problems such as low inspection efficiency, insufficient accuracy, and being greatly affected by human factors, and are difficult to meet the production requirements of high speed and high precision. There are obvious limitations especially in identifying tiny defects or achieving multi-dimensional inspection.

[0003] With the rapid development of artificial intelligence technology, especially deep learning and computer vision, remarkable progress has been made in the fields of image recognition and pattern analysis. Using deep learning algorithms, especially convolutional neural networks, feature extraction and defect detection can be performed on printed product images to achieve high-precision and real-time quality monitoring. For example, the patent with the publication number CN118247259B discloses a high-speed online printing quality inspection method, which adaptively divides the image area occupied by several pixel points into new image blocks for image enhancement processing according to the possibility of ink spot defects, improving the accuracy of identifying ink spot defect areas. Another example is the patent with the publication number CN106228562A, which discloses an online printing color quality evaluation method based on a probabilistic neural network algorithm. By combining the color space and the probability model, the reducibility and consistency of printing colors are ensured.

[0004] The above-mentioned technologies respectively achieve printing quality inspection by detecting ink spot defects and color deviations. However, during the printing process, in addition to ink spot defects and color deviations, there are also cases of pattern misalignment. Pattern misalignment refers to the situation where text, graphics, or logos fail to accurately align with the preset positions during the printing process and are displaced. Pattern misalignment not only affects the aesthetics and consistency of printed products, but also directly reduces the product quality and usability of printed products. Especially in the fields of packaging, labels, and advertising printing that require high precision, the misalignment problem will bring more serious negative impacts, such as affecting the brand image, barcode recognition errors, and increasing the scrap rate.

[0005] In view of this, the present invention proposes an online printing quality inspection method based on artificial intelligence to solve the above problems. Summary of the Invention

[0006] In order to overcome the above-mentioned defects of the prior art and achieve the above object, the present invention provides the following technical solution: An online printing quality inspection method based on artificial intelligence, including:

[0007] S1: Collect the printed matter image;

[0008] S2: Preprocess the printed matter image to obtain a preprocessed image;

[0009] S3: Obtain the reference image corresponding to the printed matter image;

[0010] S4: Align the preprocessed image with the reference image, calculate the overall offset, and determine whether to generate an overall offset instruction;

[0011] S5: If no overall offset instruction is generated, perform image segmentation on the preprocessed image to obtain segmented images;

[0012] S6: Perform key point matching on the segmented images, calculate the local offset, and determine whether to generate a local offset instruction.

[0013] Further, the method for collecting the printed matter image is as follows: When a printed matter is detected, obtain the detection time t1; obtain the conveyor belt speed v1 and the sensor distance d1; by dividing the sensor distance d1 by the conveyor belt speed v1 and then adding the detection time t1, obtain the collection time t2. The expression for the collection time t2 is: When the collection time t2 is reached, collect the printed matter image;

[0014] The method for obtaining the preprocessed image includes:

[0015] Extract the RGB values of each pixel point in the printed matter image. The RGB values include the red channel value R, the green channel value G, and the blue channel value B; for the RGB values of each pixel point, use the weighted average method to calculate the corresponding gray value. The expression for the gray value is: Gray = 0.299R + 0.587G + 0.114B; where Gray is the gray value; based on the gray values of all pixel points, obtain the preprocessed image.

[0016] Further, construct a reference set. The reference set includes different image labels and corresponding reference data. The reference data includes a reference image and reference information; the image label is the digital label corresponding to the printed matter image, and different printed matter images have different digital labels; obtain the corresponding image label according to the printed matter image, and then according to the reference set, obtain the reference data corresponding to the image label; the reference image is a printed matter image that meets the quality standard; the reference information includes reference vertex coordinates and reference key point coordinates;

[0017] The method for obtaining the reference vertex coordinates is:

[0018] Using the trained pattern detection model, perform rectangular box annotation on the printed pattern in the reference image to obtain the first rectangular box; use the vertex coordinates of the first rectangular box as the reference vertex coordinates; the printed pattern is the pattern printed on the printed matter; the training method of the pattern detection model includes:

[0019] Pre-collect a different reference images, where a is an integer greater than 1. Perform rectangular box annotation on the printed pattern in each reference image respectively; divide the annotated reference images into a training set and a test set, use the training set to train the pattern detection model, and use the test set to test the pattern detection model; preset a coordinate error threshold. When the average value of the predicted coordinate errors of all reference images in the test set is less than the coordinate error threshold, the pattern detection model training is completed, and the pattern detection model is output; the pattern detection model is a convolutional neural network model.

[0020] Further, the method for obtaining the reference key point coordinates is:

[0021] Use the Sobel operator to calculate the horizontal gradient and vertical gradient of each pixel point in the reference image; the horizontal gradient is the change rate of the gray value of the pixel point in the horizontal direction, and the vertical gradient is the change rate of the gray value of the pixel point in the vertical direction; construct a window and calculate the structure tensor of each pixel point in the reference image. The window is centered on the pixel point to be calculated, and the pixel point to be calculated is the pixel point for which the structure tensor is being calculated; the expression of the structure tensor is:

[0022]

[0023] In the formula, M is the structure tensor, I fx is the horizontal gradient of the f-th pixel point in the window, I fy is the vertical gradient of the f-th pixel point in the window, f ∈ [1, g], and g is the number of pixel points in the window;

[0024] According to the structure tensor, calculate the response value of each pixel point in the reference image; the expression of the response value is: h = A - kA′, In the formula, h is the response value, A is the determinant, A′ is the trace, and k is an empirical parameter, k ∈ [0.04, 0.06];

[0025] Divide the reference image into regions according to the size of the window to obtain p regions, where p is an integer greater than 1; sort the response values corresponding to all pixel points in each region from largest to smallest to generate a sorting table, and the sorting table corresponds to the region one by one; retain the pixel points corresponding to the response values ranked at the top in each sorting table and mark them as extreme points; preset a response threshold, compare the response values of each extreme point with the response threshold respectively, mark the extreme points with response values greater than the response threshold as reference key points, obtain the coordinates of the reference key points, and the extreme points with response values less than or equal to the response threshold are not marked.

[0026] Further, the method for aligning the preprocessed image with the reference image includes:

[0027] Use the trained product detection model to respectively perform rectangular box annotation on the printed matter in the preprocessed image and the reference image to obtain a second rectangular box and a third rectangular box; the second rectangular box is the rectangular box corresponding to the printed matter in the preprocessed image, and the third rectangular box is the rectangular box corresponding to the printed matter in the reference image; take the vertex coordinates of the second rectangular box as the real-time product coordinates; take the vertex coordinates of the third rectangular box as the reference product coordinates; the training method of the product detection model is the same as that of the pattern detection model, and both are convolutional neural network models;

[0028] Define the alignment matrix D, where t1, t3 control scaling and rotation, t2, t4 control tilt, and t5, t6 are translation amounts; correspond each coordinate in the real-time product coordinates with each coordinate in the reference product coordinates one by one, and sequentially take the corresponding two coordinates as a set of first coordinate sets; calculate the alignment matrix according to the four sets of first coordinate sets, and the calculation expression is: D = EF -1 ; in the formula, F -1 is the inverse matrix of F, x i is the abscissa corresponding to the reference product coordinates in the i-th set of first coordinate sets, y i is the ordinate corresponding to the reference product coordinates in the i-th set of first coordinate sets, x i ′ is the abscissa corresponding to the real-time product coordinates in the i-th set of first coordinate sets, y i ′ is the ordinate corresponding to the real-time product coordinates in the i-th set of first coordinate sets, i ∈ [1, 4]; substitute the four sets of first coordinate sets into the calculation expression of the alignment matrix in sequence and perform simultaneous equations to calculate the values of t1, t2, t3, t4, t5, t6 corresponding to the alignment matrix D;

[0029] Using the trained pattern detection model, perform rectangular box annotation on the printed pattern in the preprocessed image to obtain the fourth rectangular box; use the vertex coordinates of the fourth rectangular box as the real-time vertex coordinates; multiply each vertex coordinate in the real-time vertex coordinates by the alignment matrix to obtain the aligned vertex coordinates.

[0030] Further, the method for calculating the overall offset includes:

[0031] Correspond each coordinate in the real-time vertex coordinates with each coordinate in the reference vertex coordinates one by one, and sequentially use the corresponding two coordinates as a set of second coordinate sets; replace the real-time vertex coordinates in each set of second coordinate sets with the corresponding aligned vertex coordinates; sequentially calculate the Euclidean distance between the two coordinates in each set of second coordinate sets and use it as the offset; sequentially add each offset and then divide by 4 to obtain the overall offset;

[0032] The method for determining whether to generate an overall offset instruction includes:

[0033] Dynamically calculate the offset threshold and compare the overall offset with the offset threshold; if the overall offset is less than the offset threshold, do not generate an overall offset instruction; if the overall offset is greater than or equal to the offset threshold, generate an overall offset instruction.

[0034] Further, the steps for dynamically calculating the offset threshold include:

[0035] Step S401: Set the search interval [m, n] and the precision ε;

[0036] Step S402: Determine the search sequence F(j);

[0037] Step S403: Determine the partition coefficient F(z) from the search sequence;

[0038] Step S404: Use the partition coefficient to partition the search interval to obtain two points ψ1 and ψ2;

[0039] Step S405: Calculate the evaluation values corresponding to ψ1 and ψ2 respectively;

[0040] Step S406: Update the search interval;

[0041] Step S407: Calculate the width υ of the search interval;

[0042] Step S408: If the width υ is less than or equal to the precision ε, divide the width υ by 2 as the offset threshold; if the width υ is greater than the precision ε, return to step S403.

[0043] Further, in step S402, the search sequence

[0044] In the step S403, the method for determining the partitioning coefficient F(z) is as follows: Calculate the initial partitioning coefficient δ′. Subtract the initial partitioning coefficient from each value in the search sequence to obtain the value difference; sort each value difference from smallest to largest, and obtain the value corresponding to the value difference ranked at the front, which is used as the partitioning coefficient F(z).

[0045] In the step S404, In the formula, F(z - 2) is the value ranked two positions before the partitioning coefficient F(z) in the search sequence, and F(z - 1) is the value ranked one position before the partitioning coefficient F(z) in the search sequence.

[0046] In the step S405, the method for obtaining the evaluation value is as follows: Use the value corresponding to the image label and the point as the analysis data, input the analysis data into the trained threshold evaluation model, and predict the corresponding evaluation value. The threshold evaluation model is a deep neural network model.

[0047] In the step S406, the method for updating the search interval is as follows: Mark the evaluation value corresponding to ψ1 as the first evaluation value ω1, and mark the evaluation value corresponding to ψ2 as the second evaluation value ω2; if ω1 < ω2, update the search interval to [ω1, n], that is, m = ω1; if ω1 ≥ ω2, update the search interval to [m, ω2], that is, n = ω2.

[0048] In the step S407, subtract the minimum value from the maximum value of the search interval to obtain the width υ.

[0049] Furthermore, the method for obtaining the segmented image includes:

[0050] Preset a boundary threshold; compare the gray value of each pixel point in the preprocessed image with the boundary threshold respectively; mark the pixel points with gray value greater than or equal to the boundary threshold as pattern points, and do not mark the pixel points with gray value less than the boundary threshold; count the number of adjacent pixel points of each pixel point in the preprocessed image and mark it as the adjacent number; sort each adjacent number from largest to smallest, and obtain the adjacent number ranked at the front and mark it as the maximum number; mark the pixel points with adjacent number not being the maximum number as boundary points.

[0051] Analyze the pixel points adjacent to each pattern point; if not all the pixel points adjacent to the pattern point are pattern points, mark the corresponding pattern point as an edge point; if the pattern point is marked as a boundary point and all the adjacent pixel points are pattern points, mark the corresponding pattern point as an edge point; if the pattern point is not marked as a boundary point and all the adjacent pixel points are pattern points, do not mark the corresponding pattern point; perform image segmentation on the preprocessed image according to all the edge points corresponding to the preprocessed image; mark the image including all the pattern points in the segmented image as the segmented image.

[0052] Further, the method for calculating the local offset includes:

[0053] Obtain the real-time key point coordinates in the segmented image. The method for obtaining the real-time key point coordinates is the same as that for obtaining the reference key point coordinates; multiply each coordinate in the real-time key point coordinates by the alignment matrix to obtain the aligned key point coordinates; make each coordinate in the aligned key point coordinates correspond one by one with each coordinate in the reference key point coordinates, and sequentially use the corresponding two coordinates as a set of third coordinate sets; replace the real-time key point coordinates in each set of third coordinate sets with the corresponding aligned key point coordinates; sequentially calculate the Euclidean distance between the two coordinates in each set of third coordinate sets and use it as the local offset.

[0054] The method for determining whether to generate a local offset instruction includes:

[0055] Compare each local offset with the offset threshold respectively. If the local offset is less than the offset threshold, no local offset instruction is generated; if the local offset is greater than or equal to the offset threshold, a local offset instruction is generated, and the corresponding real-time key point coordinates are marked as offset coordinates.

[0056] The technical effects and advantages of an online printing quality detection method based on artificial intelligence according to the present invention:

[0057] Through deep learning technology, image alignment and key point matching between the printed image and the reference image are realized; and by calculating the overall offset and local offset, it is quantitatively detected whether the printed pattern in the printed matter has an overall or local offset, so as to realize the real-time detection of the printed matter quality; it can automatically identify the pattern offset in the printing process, reduce manual intervention and errors, reduce the scrap rate, thereby improving the production efficiency and product quality of the printed matter, and meeting the requirements of the modern printing industry for high precision and high efficiency. Brief Description of the Drawings

[0058] Figure 1 It is a flowchart of an online printing quality detection method based on artificial intelligence according to Embodiment 1 of the present invention;

[0059] Figure 2 It is a flowchart of a method for dynamically calculating the offset threshold according to Embodiment 1 of the present invention. Detailed Embodiments

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0061] Embodiment 1

[0062] Please refer to Figure 1 As shown, a method for online detection of printing quality based on artificial intelligence in this embodiment includes:

[0063] S1: Collect the printing image.

[0064] The printing image is obtained by a high-resolution camera installed on the printing production line taking pictures of the printed matter; the method for collecting the printing image is: when the printed matter is detected, obtain the detection time t1; obtain the conveyor belt speed v1 and the sensor distance d1; by the ratio of the sensor distance d1 to the conveyor belt speed v1, plus the detection time t1, obtain the collection time t2, and the expression of the collection time t2 is: When the collection time t2 is reached, collect the printing image; it should be understood that the printed matter is detected by a photoelectric sensor, and the photoelectric sensor is installed at a position with a distance of d1 from the high-resolution camera according to the reverse direction of the conveying direction of the conveyor belt in the printing production line; the conveyor belt speed v1 and the sensor distance d1 are obtained by those skilled in the art through measurement.

[0065] S2: Preprocess the printing image to obtain a preprocessed image.

[0066] The method for obtaining the preprocessed image includes:

[0067] Extract the RGB values of each pixel point in the printing image, where the RGB values include the red channel value R, the green channel value G, and the blue channel value B; use the weighted average method for the RGB values of each pixel point to calculate the corresponding gray value; the expression of the gray value is: Gray = 0.299R + 0.587G + 0.114B; in the formula, Gray is the gray value; obtain the preprocessed image according to the gray values of all pixel points.

[0068] S3: Obtain the reference image corresponding to the printing image.

[0069] Construct a reference set, which includes different image tags and corresponding reference data. The reference data includes reference images and reference information. The image tags are digital tags corresponding to printed images, and different printed images have different digital tags. Obtain the corresponding image tag according to the printed image, and then obtain the reference data corresponding to the image tag according to the reference set. The reference image is a printed image that meets the quality standard, and the reference image is determined by those skilled in the art after performing quality inspection on the printed image. The reference information includes reference vertex coordinates and reference key point coordinates.

[0070] The method for obtaining the reference vertex coordinates is as follows:

[0071] Use the trained pattern detection model to perform rectangular box annotation on the printed pattern in the reference image to obtain the first rectangular box. Take the vertex coordinates of the first rectangular box as the reference vertex coordinates. The printed pattern is the pattern printed on the printed matter. The training method of the pattern detection model includes:

[0072] Pre-collect a different reference images, where a is an integer greater than 1. Perform rectangular box annotation on the printed pattern in each reference image respectively. Divide the annotated reference images into a training set and a test set. Take 70% of the reference images as the training set and 30% of the reference images as the test set. Use the training set to train the pattern detection model and use the test set to test the pattern detection model. Preset a coordinate error threshold. When the average value of the predicted coordinate errors of all reference images in the test set is less than the coordinate error threshold, the training of the pattern detection model is completed and the pattern detection model is output. Among them, the calculation formula for the average value of the predicted coordinate errors is:

[0073]

[0074] Where ZB b is the predicted coordinate error, b is the number of the reference image, C is the number of reference images in the test set, x be is the predicted abscissa of a vertex, x′ be is the actual abscissa of a vertex, y be is the predicted ordinate of a vertex, y′ be is the actual ordinate of a vertex, e is the number of the vertex of the rectangular box, and the rectangular box has 4 vertices. The coordinate error threshold is preset according to the required accuracy of the pattern detection model. The pattern detection model is specifically a convolutional neural network model.

[0075] The method for obtaining the reference key point coordinates is as follows:

[0076] The horizontal gradient and vertical gradient of each pixel point in the reference image are calculated using the Sobel operator. The Sobel operator is a prior art and will not be elaborated here. The horizontal gradient is the change rate of the gray value of the pixel point in the horizontal direction, and the vertical gradient is the change rate of the gray value of the pixel point in the vertical direction. A window is constructed to calculate the structure tensor of each pixel point in the reference image. The window is centered on the pixel point to be calculated, and the pixel point to be calculated is the pixel point for which the structure tensor is being calculated. In this embodiment, the window is preferably 3×3. The expression of the structure tensor is:

[0077]

[0078] In the formula, M is the structure tensor, and I fx is the horizontal gradient of the f-th pixel point in the window, and I fy is the vertical gradient of the f-th pixel point in the window, f ∈ [1, g], and g is the number of pixel points in the window;

[0079] According to the structure tensor, the response value of each pixel point in the reference image is calculated. The expression of the response value is: h = A - kA′, In the formula, h is the response value, A is the determinant, A′ is the trace, and k is an empirical parameter, k ∈ [0.04, 0.06];

[0080] The reference image is divided into p regions according to the size of the window, where p is an integer greater than 1. All the pixel points corresponding to the response values within each region are sorted from large to small to generate a sorting table, and the sorting table corresponds to the region one by one. The pixel points corresponding to the response values ranked at the top of each sorting table are retained and marked as extreme points. A response threshold is preset, and the response threshold is preset by those skilled in the art according to the actual situation. The response values of each extreme point are respectively compared with the response threshold, and the extreme points with response values greater than the response threshold are marked as reference key points to obtain the coordinates of the reference key points, while the extreme points with response values less than or equal to the response threshold are not marked.

[0081] S4: Align the preprocessed image with the reference image, calculate the overall offset, and determine whether to generate an overall offset instruction.

[0082] The method for aligning the preprocessed image with the reference image includes:

[0083] Using the trained product detection model, the printed matter in the preprocessed image and the reference image are respectively annotated with rectangular frames to obtain the second rectangular frame and the third rectangular frame; the second rectangular frame is the rectangular frame corresponding to the printed matter in the preprocessed image, and the third rectangular frame is the rectangular frame corresponding to the printed matter in the reference image; the vertex coordinates of the second rectangular frame are used as the real-time product coordinates; the vertex coordinates of the third rectangular frame are used as the reference product coordinates; the training method of the product detection model is consistent with the training method of the pattern detection model, and both are convolutional neural network models;

[0084] Define the alignment matrix D, Among them, t1 and t3 control scaling and rotation, t2 and t4 control tilt, and t5 and t6 are translations; each coordinate in the real-time product coordinates is matched one by one with each coordinate in the reference product coordinates, and the corresponding two coordinates are used as a set of first coordinate sets in turn; the alignment matrix is ​​calculated based on the four sets of first coordinate sets, and the calculation expression is: D = EF -1 Where, F -1 is the inverse matrix of F, x i is the horizontal coordinate corresponding to the coordinate of the reference product in the first coordinate set of the i-th group, y i is the ordinate corresponding to the coordinate of the reference product in the first coordinate set of the i-th group, x i ′ is the horizontal coordinate corresponding to the real-time product coordinate in the first coordinate set of the i-th group, y i ′ is the ordinate corresponding to the real-time product coordinate in the i-th group of first coordinate sets, i∈[1,4]; the four groups of first coordinate sets are sequentially substituted into the calculation expression of the alignment matrix and combined to calculate the values ​​corresponding to t1, t2, t3, t4, t5, and t6 in the alignment matrix D;

[0085] The trained pattern detection model is used to mark the printed pattern in the preprocessed image with a rectangular frame to obtain a fourth rectangular frame; the vertex coordinates of the fourth rectangular frame are used as real-time vertex coordinates; each vertex coordinate in the real-time vertex coordinates is multiplied by the alignment matrix to obtain aligned vertex coordinates.

[0086] Methods for calculating the overall offset include:

[0087] Make a one-to-one correspondence between each coordinate in the real-time vertex coordinates and each coordinate in the reference vertex coordinates, and use the corresponding two coordinates as a set of second coordinate sets in turn; replace the real-time vertex coordinates in each set of second coordinates with the corresponding aligned vertex coordinates; calculate the Euclidean distance between the two coordinates in each set of second coordinates in turn, and use it as the offset; add each offset in turn, and then divide it by 4 to obtain the overall offset.

[0088] Methods for determining whether to generate an overall offset instruction include:

[0089] Dynamically calculate the offset threshold, and compare the overall offset with the offset threshold; if the overall offset is less than the offset threshold, no overall offset instruction is generated; if the overall offset is greater than or equal to the offset threshold, an overall offset instruction is generated.

[0090] As Figure 2 shown, the steps of dynamically calculating the offset threshold include:

[0091] Step S401: Set the search interval [m, n] and the precision ε;

[0092] Step S402: Determine the search sequence F(j);

[0093] Step S403: Determine the partitioning coefficient F(z) from the search sequence;

[0094] Step S404: Use the partitioning coefficient to partition the search interval to obtain two points ψ1 and ψ2;

[0095] Step S405: Calculate the evaluation values corresponding to ψ1 and ψ2 respectively;

[0096] Step S406: Update the search interval;

[0097] Step S407: Calculate the width υ of the search interval;

[0098] Step S408: If the width υ is less than or equal to the precision ε, divide the width υ by 2 as the offset threshold; if the width υ is greater than the precision ε, return to Step S403.

[0099] In the above Step S401, the search interval [m, n] and the precision ε are preset by those skilled in the art according to the actual situation.

[0100] In the above Step S402, the search sequence

[0101] In the above Step S403, the method for determining the partitioning coefficient F(z) is: calculate the initial partitioning coefficient δ′, Subtract the initial partitioning coefficient from each value in the search sequence to obtain the value difference; sort each value difference from small to large, and obtain the value corresponding to the value difference ranked at the forefront as the partitioning coefficient F(z).

[0102] In the above Step S404, wherein, F(z - 2) is the value ranked two positions before the partitioning coefficient F(z) in the search sequence, and F(z - 1) is the value ranked one position before the partitioning coefficient F(z) in the search sequence.

[0103] In the above step S405, the method for obtaining the evaluation value is as follows: taking the value corresponding to the image label and the point as the analysis data, inputting the analysis data into the trained threshold evaluation model, and predicting the corresponding evaluation value; the training process of the threshold evaluation model includes.

[0104] Pre-collect r sets of analysis data, set corresponding evaluation values for the r sets of analysis data, where r is an integer greater than 1, and convert the analysis data and the corresponding evaluation values into a corresponding set of feature vectors; the evaluation value corresponding to the analysis data is determined by those skilled in the art during the historical pattern offset analysis process. r sets of analysis data are collected, and each set of analysis data is analyzed in turn in combination with practical experience to evaluate the adaptation degree of the image label and the point in each set of analysis data, and used as the corresponding evaluation value, and corresponding evaluation values are set for the a sets of analysis data in turn;

[0105] Taking each set of feature vectors as the input of the threshold evaluation model, the threshold evaluation model outputs a set of predicted evaluation values corresponding to each set of analysis data, takes the actual evaluation value corresponding to each set of analysis data as the prediction target, and the actual evaluation value is the pre-set evaluation value corresponding to the analysis data; taking the minimization of the sum of the prediction errors of all analysis data as the training target; where the calculation formula of the prediction error is η K =(β K -α K ) 2 , where η K is the prediction error, K is the group number of the feature vector corresponding to the analysis data, β K is the predicted evaluation value corresponding to the Kth set of analysis data, and α K is the actual evaluation value corresponding to the Kth set of analysis data; training the threshold evaluation model until the sum of the prediction errors reaches convergence and then stopping the training.

[0106] The above threshold evaluation model is specifically a deep neural network model; which includes an input layer, a hidden layer and an output layer; each hidden layer includes multiple neurons, and there are connections between each neuron and the neurons in the next layer, and the connections contain weights, which determine the importance and influence of data transmission in the neural network; an activation function is applied to each neuron between the hidden layer and the output layer, and the activation function introduces non-linearity, allowing the network to learn more complex patterns and features.

[0107] In the above step S406, the method for updating the search interval is as follows: marking the evaluation value corresponding to ψ1 as the first evaluation value ω1, and marking the evaluation value corresponding to ψ2 as the second evaluation value ω2; if ω1 < ω2, the search interval is updated to [ω1, n], that is, m = ω1; if ω1 ≥ ω2, the search interval is updated to [m, ω2], that is, n = ω2.

[0108] In the above step S407, subtract the minimum value from the maximum value of the search interval to obtain the width υ.

[0109] S5: If no overall offset instruction is generated, perform image segmentation on the preprocessed image to obtain a segmented image; this is to eliminate the interference of background noise and improve the extraction accuracy of subsequent real-time key point coordinates.

[0110] The method for obtaining the segmented image includes:

[0111] Preset a boundary threshold; the boundary threshold is obtained by those skilled in the art analyzing multiple historical images to obtain the edge points corresponding to each historical image; the historical images are preprocessed images obtained at historical moments, and the edge points are the pixel points corresponding to the edges of the printed patterns; sort the pixel values of the edge points corresponding to each historical image from small to large, and obtain the smallest gray value as the minimum gray value corresponding to the historical image; take the average value of the minimum gray values corresponding to all historical images as the boundary threshold;

[0112] Compare the gray value of each pixel point in the preprocessed image with the boundary threshold respectively; mark the pixel points with gray values greater than or equal to the boundary threshold as pattern points, and do not mark the pixel points with gray values less than the boundary threshold; count the number of adjacent pixel points of each pixel point in the preprocessed image and mark it as the adjacent number; sort each adjacent number from large to small, obtain the adjacent numbers ranked in the front, and mark them as the maximum numbers; mark the pixel points with adjacent numbers not being the maximum numbers as boundary points;

[0113] Analyze the pixel points adjacent to each pattern point; if the pixel points adjacent to the pattern point are not all pattern points, mark the corresponding pattern point as an edge point; if the pattern point is marked as a boundary point and all adjacent pixel points are pattern points, mark the corresponding pattern point as an edge point; if the pattern point is not marked as a boundary point and all adjacent pixel points are pattern points, do not mark the corresponding pattern point; perform image segmentation on the preprocessed image according to all the edge points corresponding to the preprocessed image; mark the image including all pattern points in the segmented image as the segmented image.

[0114] S6: Perform key point matching on the segmented image, calculate the local offset, and determine whether to generate a local offset instruction.

[0115] The method for calculating the local offset includes:

[0116] Obtain the real-time key point coordinates in the segmented image. The method for obtaining the real-time key point coordinates is the same as the method for obtaining the reference key point coordinates; multiply each coordinate in the real-time key point coordinates by the alignment matrix respectively to obtain the aligned key point coordinates; make each coordinate in the aligned key point coordinates correspond one by one with each coordinate in the reference key point coordinates, and sequentially take the corresponding two coordinates as a set of third coordinate sets; replace the real-time key point coordinates in each set of third coordinate sets with the corresponding aligned key point coordinates; calculate the Euclidean distance between the two coordinates in each set of third coordinate sets in turn, and use it as the local offset.

[0117] The method for determining whether to generate a local offset instruction includes:

[0118] Compare each local offset with the offset threshold respectively. If the local offset is less than the offset threshold, no local offset instruction is generated; if the local offset is greater than or equal to the offset threshold, a local offset instruction is generated, and the corresponding real-time key point coordinates are marked as offset coordinates to quickly identify the specific offset position of the printed pattern in the printed matter image.

[0119] In this embodiment, through deep learning technology, image alignment and key point matching between the printed matter image and the reference image are realized; and by calculating the overall offset and local offset, it is quantitatively detected whether the printed pattern in the printed matter has an overall or local offset, so as to realize the real-time detection of the printed matter quality; it can automatically identify the pattern offset in the printing process, reduce manual intervention and errors, reduce the scrap rate, thereby improving the production efficiency and product quality of the printed matter, and meeting the requirements of the modern printing industry for high precision and high efficiency.

[0120] Embodiment 2

[0121] This application also provides an electronic device. The electronic device may include one or more processors and one or more memories. Among them, computer-readable code is stored in the memory, and when the computer-readable code is run by one or more processors, it can execute an online printing quality detection method based on artificial intelligence as described above.

[0122] The method or system according to the embodiment of the present application can also be implemented by means of the architecture of the electronic device shown in the present application. The electronic device may include a bus, one or more CPUs, ROM, RAM, a communication port connected to the network, input / output, a hard disk, etc. The storage device in the electronic device, such as ROM or hard disk, can store an online printing quality detection method based on artificial intelligence provided by the present application. Further, the electronic device may further include a user interface. Of course, the architecture shown in the present application is only exemplary, and when implementing different devices, one or more components shown in the electronic device of the present application can be omitted according to actual needs.

[0123] Embodiment 3

[0124] Referring to the figure shown, an embodiment of the present application discloses a computer-readable storage medium. Computer-readable instructions are stored on the computer-readable storage medium. When the computer-readable instructions are run by a processor, a method for on-line detection of printing quality based on artificial intelligence according to an embodiment of the present application described with reference to the above drawings can be executed. The storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory, etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0125] In addition, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, which stores machine-readable instructions that can be run by a processor to execute instructions corresponding to the method steps provided by the present application, for example: a method for on-line detection of printing quality based on artificial intelligence. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed.

[0126] The above is only the specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all of them should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

[0127] Finally: The above is only the preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A printing quality online detection method based on artificial intelligence, characterized in that: include: S1: Collect printed images; S2: preprocessing the printed image to obtain a preprocessed image; S3: Acquire a reference image corresponding to the printed image; Constructing a benchmark set, the benchmark set includes different image labels and corresponding benchmark data, and the benchmark data includes benchmark images and benchmark information; The reference information includes reference vertex coordinates and reference key point coordinates; The method for obtaining the coordinates of the benchmark key points is: using the Sobel operator to calculate the horizontal gradient and vertical gradient of each pixel in the benchmark image; Build a window and calculate the structure tensor of each pixel in the reference image; According to the structure tensor, the response value of each pixel in the reference image is calculated; The reference image is divided into regions according to the size of the window, and p regions are obtained, where p is an integer greater than 1; the maximum point of each region is obtained, a response threshold is preset, the response value of each maximum point is compared with the response threshold, and the maximum point whose response value is greater than the response threshold is marked as a reference key point, and the coordinates of the reference key point are obtained, and the maximum point whose response value is less than or equal to the response threshold is not marked; S4: aligning the preprocessed image with the reference image, calculating the overall offset, and determining whether to generate an overall offset instruction; S5: If the overall offset instruction is not generated, the preprocessed image is segmented to obtain a segmented image; S6: performing key point matching on the segmented image, calculating the local offset, and determining whether to generate a local offset instruction; The method for calculating the local offset comprises: Define the alignment matrix D, obtain the real-time key point coordinates in the segmented image, and multiply each coordinate in the real-time key point coordinates by the alignment matrix to obtain the alignment key point coordinates; Each coordinate in the alignment key point coordinates is matched one by one with each coordinate in the reference key point coordinates, and the corresponding two coordinates are taken as a set of third coordinate sets in turn; the alignment key point coordinates in each set of third coordinates are replaced with the corresponding real-time key point coordinates; the Euclidean distance between the two coordinates in each set of third coordinates is calculated in turn and used as the local offset.

2. The method for online printing quality detection based on artificial intelligence according to claim 1, characterized in that: The method for collecting printed matter images is as follows: when a printed matter is detected, the detection time t1 is obtained; the conveyor belt speed v1 and the sensor distance d1 are obtained; and the collection time t2 is obtained by adding the ratio of the sensor distance d1 to the conveyor belt speed v1 and the detection time t1. The expression of the collection time t2 is: When the acquisition time t2 is reached, the printed image is acquired; The method for obtaining the preprocessed image comprises: Extract the RGB value of each pixel in the printed image, where the RGB value includes the red channel value R, the green channel value G and the blue channel value B; The weighted average method is used for the RGB value of each pixel to calculate the corresponding gray value; the expression of the gray value is: Gray = 0.299R + 0.587G + 0.114B; where Gray is the gray value; the preprocessed image is obtained according to the gray values ​​of all pixels.

3. The method for online printing quality detection based on artificial intelligence according to claim 2, characterized in that: The image label is a digital label corresponding to the printed image, and different printed images have different corresponding digital labels. The corresponding image label is obtained according to the printed image, and then the benchmark data corresponding to the image label is obtained according to the benchmark set. The benchmark image is a printed image that meets the quality standards. The method for obtaining the reference vertex coordinates is: Using the trained pattern detection model, annotate the printed pattern in the reference image with a rectangular frame to obtain a first rectangular frame; The vertex coordinates of the first rectangular frame are used as the reference vertex coordinates; Printed patterns are patterns printed on printed materials; The training method of the pattern detection model includes: A number of different reference images are collected in advance, where a is an integer greater than 1, and a rectangular frame is marked for the printed pattern in each reference image; the marked reference images are divided into a training set and a test set, and a pattern detection model is trained using the training set, and the pattern detection model is tested using the test set; a coordinate error threshold is preset, and when the mean of the predicted coordinate errors of all reference images in the test set is less than the coordinate error threshold, the pattern detection model training is completed, and the pattern detection model is output; the pattern detection model is a convolutional neural network model.

4. The method for online printing quality detection based on artificial intelligence according to claim 3 is characterized in that: The horizontal gradient is the rate of change of the grayscale value of the pixel in the horizontal direction, and the vertical gradient is the rate of change of the grayscale value of the pixel in the vertical direction. The window is centered on the pixel to be calculated, and the pixel to be calculated is the pixel for which the structure tensor is being calculated. The expression of the structure tensor is: Where M is the structure tensor, I fx is the horizontal gradient of the f-th pixel in the window, I fy is the vertical gradient of the f-th pixel in the window, f∈[1,g], g is the number of pixels in the window; The expression of the response value is: h = A-kA', Where h is the response value, A is the determinant, A′ is the trace, k is the empirical parameter, k∈[0.04,0.06]; Sort the response values ​​corresponding to all pixels in each area from large to small to generate a sorting table, which corresponds to the area one by one; retain the pixel corresponding to the response value ranked at the front in each sorting table and mark it as a maximum point.

5. The method for online printing quality detection based on artificial intelligence according to claim 4 is characterized in that: The method for aligning the preprocessed image with the reference image comprises: Using the trained product detection model, the printed matter in the preprocessed image and the reference image are respectively annotated with rectangular frames to obtain the second rectangular frame and the third rectangular frame; the second rectangular frame is the rectangular frame corresponding to the printed matter in the preprocessed image, and the third rectangular frame is the rectangular frame corresponding to the printed matter in the reference image; the vertex coordinates of the second rectangular frame are used as the real-time product coordinates; the vertex coordinates of the third rectangular frame are used as the reference product coordinates; the training method of the product detection model is consistent with the training method of the pattern detection model, and both are convolutional neural network models; Among them, t1 and t3 control scaling and rotation, t2 and t4 control tilt, and t5 and t6 are translations; each coordinate in the real-time product coordinates is matched one by one with each coordinate in the reference product coordinates, and the corresponding two coordinates are used as a set of first coordinate sets in turn; the alignment matrix is ​​calculated based on the four sets of first coordinate sets, and the calculation expression is: D = EF -1 Where, F -1 is the inverse matrix of F, x i is the horizontal coordinate corresponding to the coordinate of the reference product in the first coordinate set of the i-th group, y i is the ordinate corresponding to the coordinate of the reference product in the first coordinate set of the i-th group, x i ′ is the horizontal coordinate corresponding to the real-time product coordinate in the first coordinate set of the i-th group, y i ′ is the ordinate corresponding to the real-time product coordinate in the i-th group of first coordinate sets, i∈[1,4]; the four groups of first coordinate sets are sequentially substituted into the calculation expression of the alignment matrix and combined to calculate the values ​​corresponding to t1, t2, t3, t4, t5, and t6 in the alignment matrix D; The trained pattern detection model is used to mark the printed pattern in the preprocessed image with a rectangular frame to obtain a fourth rectangular frame; the vertex coordinates of the fourth rectangular frame are used as real-time vertex coordinates; each vertex coordinate in the real-time vertex coordinates is multiplied by the alignment matrix to obtain aligned vertex coordinates.

6. The method for online printing quality detection based on artificial intelligence according to claim 5 is characterized in that: The method for calculating the overall offset includes: Make a one-to-one correspondence between each coordinate in the real-time vertex coordinates and each coordinate in the reference vertex coordinates, and use the corresponding two coordinates as a set of second coordinate sets in turn; replace the real-time vertex coordinates in each set of second coordinates with the corresponding aligned vertex coordinates; calculate the Euclidean distance between the two coordinates in each set of second coordinates in turn, and use it as an offset; add each offset in turn, and then divide it by 4 to obtain the overall offset; The method for determining whether to generate an overall offset instruction includes: The offset threshold is calculated dynamically, and the overall offset is compared with the offset threshold; if the overall offset is less than the offset threshold, no overall offset instruction is generated; if the overall offset is greater than or equal to the offset threshold, an overall offset instruction is generated.

7. The method for online printing quality detection based on artificial intelligence according to claim 6 is characterized in that: The step of dynamically calculating the offset threshold comprises: Step S401: setting the search interval [m,n] and precision ε; Step S402: Determine the search sequence F(j); Step S403: Determine the partition coefficient F(z) from the search sequence; Step S404: using the partition coefficient to divide the search interval to obtain two points ψ1 and ψ2; Step S405: Calculate the evaluation values ​​corresponding to ψ1 and ψ2 respectively; Step S406: updating the search interval; Step S407: Calculate the width υ of the search interval; Step S408: If the width υ is less than or equal to the precision ε, divide the width υ by 2 as the offset threshold; if the width υ is greater than the precision ε, return to step S403.

8. The method for online printing quality detection based on artificial intelligence according to claim 7, characterized in that: In step S402, the search sequence In step S403, the method for determining the partition coefficient F(z) is: calculating the initial partition coefficient δ′, Subtract the initial partition coefficient from each value in the search sequence to obtain the value difference; sort each value difference from small to large, obtain the value corresponding to the first value difference, and use it as the partition coefficient F(z); In step S404, Wherein, F(z-2) is the value of the first two digits of the search sequence before the partition coefficient F(z), and F(z-1) is the value of the first digit of the search sequence before the partition coefficient F(z); In step S405, the method for obtaining the evaluation value is: taking the numerical value corresponding to the image label and the point as analysis data, inputting the analysis data into a trained threshold evaluation model, and predicting the corresponding evaluation value, where the threshold evaluation model is a deep neural network model; In step S406, the method for updating the search interval is: marking the evaluation value corresponding to ψ1 as the first evaluation value ω1, and marking the evaluation value corresponding to ψ2 as the second evaluation value ω2; if ω1<ω2, the search interval is updated to [ω1,n], that is, m=ω1; if ω1≥ω2, the search interval is updated to [m,ω2], that is, n=ω2; In step S407, the width υ is obtained by subtracting the minimum value from the maximum value of the search interval.

9. The method for online printing quality detection based on artificial intelligence according to claim 8, characterized in that: The method for obtaining the segmented image comprises: Preset a boundary threshold; compare the gray value of each pixel in the preprocessed image with the boundary threshold; mark the pixel whose gray value is greater than or equal to the boundary threshold as a pattern point, and do not mark the pixel whose gray value is less than the boundary threshold; count the number of adjacent pixels of each pixel in the preprocessed image and mark them as adjacent numbers; sort each adjacent number from large to small, obtain the adjacent number in the front, and mark it as the maximum number; mark the pixel whose adjacent number is not the maximum number as a boundary point; Analyze the pixels adjacent to each pattern point; if the pixels adjacent to the pattern point are not all pattern points, mark the corresponding pattern point as an edge point; if the pattern point is marked as a boundary point and the adjacent pixels are all pattern points, mark the corresponding pattern point as an edge point; if the pattern point is not marked as a boundary point and the adjacent pixels are all pattern points, do not mark the corresponding pattern point; perform image segmentation on the preprocessed image based on all edge points corresponding to the preprocessed image; mark the image including all pattern points in the segmented image as a segmented image.

10. The method for online printing quality detection based on artificial intelligence according to claim 9, characterized in that: The method for acquiring the real-time key point coordinates is consistent with the method for acquiring the reference key point coordinates; the method for determining whether to generate a local offset instruction includes: Each local offset is compared with the offset threshold. If the local offset is less than the offset threshold, no local offset instruction is generated; if the local offset is greater than or equal to the offset threshold, a local offset instruction is generated and the corresponding real-time key point coordinates are marked as offset coordinates.

Images