Printing quality online detection method based on artificial intelligence

Abstract

The invention belongs to the technical field of quality detection, and discloses a printing quality online detection method based on artificial intelligence. The method comprises: collecting a printed matter image; preprocessing the printed matter image to obtain a preprocessed image; acquiring a reference image corresponding to the printed matter image; aligning the preprocessed image with the reference image, calculating an overall offset, and judging whether an overall offset instruction is generated or not; if the overall offset instruction is not generated, performing image segmentation on the preprocessed image to obtain a segmented image; key point matching is carried out on the segmented image, local offset is calculated, and whether a local offset instruction is generated or not is judged; pattern deviation in the printing process can be automatically recognized, manual intervention and errors are reduced, the rejection rate is reduced, and therefore the production efficiency and the product quality of printed matter are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of quality detection, and more specifically, to an online printing quality detection method based on artificial intelligence. Background Art

[0002] In the modern printing industry, with the continuous improvement of the quality requirements for printed products, online detection of printing quality has become a key link in ensuring product quality, reducing scrap rate and improving production efficiency. Traditional printing quality detection methods mostly rely on manual sampling or simple visual inspection equipment. There are problems such as low detection efficiency, insufficient accuracy and significant influence of human factors. It is difficult to meet the needs of high-speed and high-precision production, especially in identifying minor defects or realizing multi-dimensional detection. There are obvious limitations.

[0003] With the rapid development of artificial intelligence technology, especially deep learning and computer vision, significant progress has been made in the field of image recognition and pattern analysis. By using deep learning algorithms, especially convolutional neural networks, it is possible to perform feature extraction and defect detection on printed images to achieve high-precision and real-time quality monitoring. For example, the patent with announcement number CN118247259B discloses a high-speed online detection method for printed product quality. According to the possibility of ink spot defects, the image area occupied by several pixels is adaptively divided into new image blocks for image enhancement processing, thereby improving the accuracy of identifying ink spot defect areas. For another example, the patent with publication number CN106228562A discloses an online printed product color quality evaluation method based on a probabilistic neural network algorithm. By combining color space and probability model, the restoration and consistency of printed colors are ensured.

[0004] The above technologies realize printing quality inspection by detecting ink spot defects and color deviations respectively; however, in the printing process, in addition to ink spot defects and color deviations, quality problems also include pattern offset; pattern offset refers to the misalignment of text, graphics or logos during the printing process due to the failure to accurately align with the preset position; pattern offset not only affects the aesthetics and consistency of the printed product, but also directly reduces the product quality and use value of the printed product, especially in the fields of packaging, labeling and advertising printing that require high precision. The offset problem will bring more serious negative effects, such as affecting the brand image, barcode recognition errors and increasing the scrap rate.

[0005] In view of this, the present invention proposes an online printing quality detection method based on artificial intelligence to solve the above problems. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned purpose, the present invention provides the following technical solution: an online printing quality detection method based on artificial intelligence, comprising:

[0007] S1: Collect printed images;

[0008] S2: preprocessing the printed image to obtain a preprocessed image;

[0009] S3: Acquire a reference image corresponding to the printed image;

[0010] S4: aligning the preprocessed image with the reference image, calculating the overall offset, and determining whether to generate an overall offset instruction;

[0011] S5: If the overall offset instruction is not generated, the preprocessed image is segmented to obtain a segmented image;

[0012] S6: Perform key point matching on the segmented image, calculate the local offset, and determine whether to generate a local offset instruction.

[0013] Furthermore, the method for collecting printed matter images is: when a printed matter is detected, the detection time t1 is obtained; the conveyor belt speed v1 and the sensor distance d1 are obtained; the collection time t2 is obtained by adding the ratio of the sensor distance d1 to the conveyor belt speed v1 and the detection time t1, and the expression of the collection time t2 is: When the acquisition time t2 is reached, the printed image is acquired;

[0014] The method for obtaining the preprocessed image comprises:

[0015] Extract the RGB value of each pixel in the printed image. The RGB value includes the red channel value R, the green channel value G and the blue channel value B. Use the weighted average method to calculate the corresponding gray value for the RGB value of each pixel. The gray value expression is: Gray = 0.299R + 0.587G + 0.114B; where Gray is the gray value. Obtain the preprocessed image based on the gray values ​​of all pixels.

[0016] Further, a benchmark set is constructed, the benchmark set includes different image labels and corresponding benchmark data, and the benchmark data includes a benchmark image and benchmark information; the image label is a digital label corresponding to the printed image, and different printed images have different corresponding digital labels; the corresponding image label is obtained according to the printed image, and then the benchmark data corresponding to the image label is obtained according to the benchmark set; the benchmark image is a printed image that meets the quality standards; the benchmark information includes benchmark vertex coordinates and benchmark key point coordinates;

[0017] The method for obtaining the reference vertex coordinates is:

[0018] Using the trained pattern detection model, a printed pattern in the reference image is annotated with a rectangular frame to obtain a first rectangular frame; the vertex coordinates of the first rectangular frame are used as the reference vertex coordinates; the printed pattern is a pattern printed on a printed product; and the training method of the pattern detection model includes:

[0019] A number of different reference images are collected in advance, where a is an integer greater than 1, and a rectangular frame is marked for the printed pattern in each reference image; the marked reference images are divided into a training set and a test set, and a pattern detection model is trained using the training set, and the pattern detection model is tested using the test set; a coordinate error threshold is preset, and when the mean of the predicted coordinate errors of all reference images in the test set is less than the coordinate error threshold, the pattern detection model training is completed, and the pattern detection model is output; the pattern detection model is a convolutional neural network model.

[0020] Furthermore, the method for obtaining the coordinates of the benchmark key points is:

[0021] The Sobel operator is used to calculate the horizontal gradient and vertical gradient of each pixel in the reference image; the horizontal gradient is the rate of change of the grayscale value of the pixel in the horizontal direction, and the vertical gradient is the rate of change of the grayscale value of the pixel in the vertical direction; a window is constructed to calculate the structure tensor of each pixel in the reference image, and the window is centered on the pixel to be calculated, and the pixel to be calculated is the pixel for which the structure tensor calculation is being performed; the expression of the structure tensor is:

[0022]

[0023] Where M is the structure tensor, I fx is the horizontal gradient of the f-th pixel in the window, I fy is the vertical gradient of the f-th pixel in the window, f∈[1,g], g is the number of pixels in the window;

[0024] According to the structure tensor, the response value of each pixel in the reference image is calculated; the expression of the response value is: h = A-kA′, Where h is the response value, A is the determinant, A′ is the trace, k is the empirical parameter, k∈[0.04,0.06];

[0025] The benchmark image is divided into regions according to the size of the window to obtain p regions, where p is an integer greater than 1; the response values ​​corresponding to all pixels in each region are sorted from large to small to generate a sorting table, which corresponds to the regions one by one; the pixel points corresponding to the response values ​​at the front in each sorting table are retained and marked as maximum points; a response threshold is preset, and the response value of each maximum point is compared with the response threshold, and the maximum point with a response value greater than the response threshold is marked as a benchmark key point, and the coordinates of the benchmark key point are obtained, while the maximum point with a response value less than or equal to the response threshold is not marked.

[0026] Furthermore, the method for aligning the preprocessed image with the reference image comprises:

[0027] The trained product detection model is used to mark the printed matter in the preprocessed image and the reference image with rectangular frames to obtain the second rectangular frame and the third rectangular frame; the second rectangular frame is the rectangular frame corresponding to the printed matter in the preprocessed image, and the third rectangular frame is the rectangular frame corresponding to the printed matter in the reference image; the vertex coordinates of the second rectangular frame are used as the real-time product coordinates; the vertex coordinates of the third rectangular frame are used as the reference product coordinates; the training method of the product detection model is consistent with the training method of the pattern detection model, and both are convolutional neural network models;

[0028] Define the alignment matrix D, Among them, t1 and t3 control scaling and rotation, t2 and t4 control tilt, and t5 and t6 are translations; each coordinate in the real-time product coordinates is matched one by one with each coordinate in the reference product coordinates, and the corresponding two coordinates are used as a set of first coordinate sets in turn; the alignment matrix is ​​calculated based on the four sets of first coordinate sets, and the calculation expression is: D = EF -1 Where, F -1 is the inverse matrix of F, x i is the horizontal coordinate corresponding to the coordinate of the reference product in the first coordinate set of the i-th group, y i is the ordinate corresponding to the coordinate of the reference product in the first coordinate set of the i-th group, x i ′ is the horizontal coordinate corresponding to the real-time product coordinate in the first coordinate set of the i-th group, y i ′ is the ordinate corresponding to the real-time product coordinate in the i-th group of first coordinate sets, i∈[1,4]; the four groups of first coordinate sets are sequentially substituted into the calculation expression of the alignment matrix and combined to calculate the values ​​corresponding to t1, t2, t3, t4, t5, and t6 in the alignment matrix D;

[0029] The trained pattern detection model is used to mark the printed pattern in the preprocessed image with a rectangular frame to obtain a fourth rectangular frame; the vertex coordinates of the fourth rectangular frame are used as real-time vertex coordinates; each vertex coordinate in the real-time vertex coordinates is multiplied by the alignment matrix to obtain aligned vertex coordinates.

[0030] Furthermore, the method for calculating the overall offset includes:

[0031] Make a one-to-one correspondence between each coordinate in the real-time vertex coordinates and each coordinate in the reference vertex coordinates, and use the corresponding two coordinates as a set of second coordinate sets in turn; replace the real-time vertex coordinates in each set of second coordinates with the corresponding aligned vertex coordinates; calculate the Euclidean distance between the two coordinates in each set of second coordinates in turn, and use it as an offset; add each offset in turn, and then divide it by 4 to obtain the overall offset;

[0032] The method for determining whether to generate an overall offset instruction includes:

[0033] The offset threshold is calculated dynamically, and the overall offset is compared with the offset threshold; if the overall offset is less than the offset threshold, no overall offset instruction is generated; if the overall offset is greater than or equal to the offset threshold, an overall offset instruction is generated.

[0034] Furthermore, the step of dynamically calculating the offset threshold comprises:

[0035] Step S401: setting the search interval [m,n] and precision ε;

[0036] Step S402: Determine the search sequence F(j);

[0037] Step S403: Determine the partition coefficient F(z) from the search sequence;

[0038] Step S404: using the partition coefficient to divide the search interval to obtain two points ψ1 and ψ2;

[0039] Step S405: Calculate the evaluation values ​​corresponding to ψ1 and ψ2 respectively;

[0040] Step S406: updating the search interval;

[0041] Step S407: Calculate the width υ of the search interval;

[0042] Step S408: If the width υ is less than or equal to the precision ε, divide the width υ by 2 as the offset threshold; if the width υ is greater than the precision ε, return to step S403.

[0043] Furthermore, in step S402, the search sequence

[0044] In step S403, the method for determining the partition coefficient F(z) is: calculating the initial partition coefficient δ′, Subtract the initial partition coefficient from each value in the search sequence to obtain the value difference; sort each value difference from small to large, obtain the value corresponding to the first value difference, and use it as the partition coefficient F(z);

[0045] In step S404, Wherein, F(z-2) is the value of the first two digits of the search sequence before the partition coefficient F(z), and F(z-1) is the value of the first digit of the search sequence before the partition coefficient F(z);

[0046] In step S405, the method for obtaining the evaluation value is: taking the numerical value corresponding to the image label and the point as analysis data, inputting the analysis data into a trained threshold evaluation model, and predicting the corresponding evaluation value, where the threshold evaluation model is a deep neural network model;

[0047] In step S406, the method for updating the search interval is: marking the evaluation value corresponding to ψ1 as the first evaluation value ω1, and marking the evaluation value corresponding to ψ2 as the second evaluation value ω2; if ω1<ω2, the search interval is updated to [ω1,n], that is, m=ω1; if ω1≥ω2, the search interval is updated to [m,ω2], that is, n=ω2;

[0048] In step S407, the width υ is obtained by subtracting the minimum value from the maximum value of the search interval.

[0049] Furthermore, the method for obtaining the segmented image includes:

[0050] Preset a boundary threshold; compare the gray value of each pixel in the preprocessed image with the boundary threshold; mark the pixel whose gray value is greater than or equal to the boundary threshold as a pattern point, and do not mark the pixel whose gray value is less than the boundary threshold; count the number of adjacent pixels of each pixel in the preprocessed image and mark them as adjacent numbers; sort each adjacent number from large to small, obtain the adjacent number in the front, and mark it as the maximum number; mark the pixel whose adjacent number is not the maximum number as a boundary point;

[0051] Analyze the pixels adjacent to each pattern point; if the pixels adjacent to the pattern point are not all pattern points, mark the corresponding pattern point as an edge point; if the pattern point is marked as a boundary point and the adjacent pixels are all pattern points, mark the corresponding pattern point as an edge point; if the pattern point is not marked as a boundary point and the adjacent pixels are all pattern points, do not mark the corresponding pattern point; perform image segmentation on the preprocessed image based on all edge points corresponding to the preprocessed image; mark the image including all pattern points in the segmented image as a segmented image.

[0052] Furthermore, the method for calculating the local offset includes:

[0053] Acquire the real-time key point coordinates in the segmented image. The acquisition method of the real-time key point coordinates is consistent with the acquisition method of the reference key point coordinates; multiply each coordinate in the real-time key point coordinates by the alignment matrix to obtain the alignment key point coordinates; make one-to-one correspondence between each coordinate in the alignment key point coordinates and each coordinate in the reference key point coordinates, and sequentially use the corresponding two coordinates as a set of third coordinate sets; replace the real-time key point coordinates in each set of third coordinate sets with the corresponding alignment key point coordinates; sequentially calculate the Euclidean distance between the two coordinates in each set of third coordinate sets and use it as the local offset;

[0054] The method for determining whether to generate a local offset instruction comprises:

[0055] Each local offset is compared with the offset threshold. If the local offset is less than the offset threshold, no local offset instruction is generated; if the local offset is greater than or equal to the offset threshold, a local offset instruction is generated and the corresponding real-time key point coordinates are marked as offset coordinates.

[0056] The technical effects and advantages of the artificial intelligence-based online printing quality detection method of the present invention are as follows:

[0057] Through deep learning technology, image alignment and key point matching of the printed image and the reference image are achieved; and by calculating the overall offset and local offset, it is possible to quantitatively detect whether the printed pattern in the printed product has an overall or local offset, thereby achieving real-time detection of print quality; it can automatically identify pattern offsets in the printing process, reduce manual intervention and errors, and reduce scrap rates, thereby improving the production efficiency and product quality of printed products, meeting the modern printing industry's needs for high precision and high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of an online printing quality detection method based on artificial intelligence according to Embodiment 1 of the present invention;

[0059] Figure 2 This is a flow chart of the method for dynamically calculating the offset threshold value of embodiment 1 of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Example 1

[0062] See also Figure 1 As shown, the present embodiment provides an artificial intelligence-based online printing quality detection method, the method comprising:

[0063] S1: Collect printed images.

[0064] The printed product image is obtained by photographing the printed product with a high-resolution camera installed on the printing production line. The method for collecting the printed product image is as follows: when the printed product is detected, the detection time t1 is obtained; the conveyor belt speed v1 and the sensor distance d1 are obtained; the acquisition time t2 is obtained by adding the ratio of the sensor distance d1 to the conveyor belt speed v1 and the detection time t1. The expression of the acquisition time t2 is: When the acquisition time t2 is reached, the image of the printed product is acquired; it should be understood that the printed product is detected by a photoelectric sensor, which is installed at a position d1 away from the high-resolution camera in the opposite direction of the conveying direction of the conveyor belt in the printing production line; the conveyor belt speed v1 and the sensor distance d1 are obtained by technical personnel in this field through measurement.

[0065] S2: Preprocess the printed image to obtain a preprocessed image.

[0066] Methods for obtaining preprocessed images include:

[0067] Extract the RGB value of each pixel in the printed image. The RGB value includes the red channel value R, the green channel value G and the blue channel value B. Use the weighted average method to calculate the corresponding gray value for the RGB value of each pixel. The gray value expression is: Gray = 0.299R + 0.587G + 0.114B; where Gray is the gray value. Obtain the preprocessed image based on the gray values ​​of all pixels.

[0068] S3: Acquire a reference image corresponding to the printed image.

[0069] A benchmark set is constructed, which includes different image labels and corresponding benchmark data, and the benchmark data includes a benchmark image and benchmark information; the image label is a digital label corresponding to the printed image, and different printed images correspond to different digital labels; the corresponding image label is obtained according to the printed image, and then the benchmark data corresponding to the image label is obtained according to the benchmark set; the benchmark image is a printed image that meets the quality standards, and the benchmark image is determined by technical personnel in this field after quality inspection of the printed image; the benchmark information includes benchmark vertex coordinates and benchmark key point coordinates.

[0070] The method for obtaining the reference vertex coordinates is:

[0071] Using the trained pattern detection model, a printed pattern in the reference image is annotated with a rectangular frame to obtain a first rectangular frame; the vertex coordinates of the first rectangular frame are used as the reference vertex coordinates; the printed pattern is a pattern printed on a printed product; and the training method of the pattern detection model includes:

[0072] Collect a different reference images in advance, where a is an integer greater than 1, and mark the printed pattern in each reference image with a rectangular frame; divide the marked reference images into a training set and a test set, use 70% of the reference images as the training set, and use 30% of the reference images as the test set, use the training set to train the pattern detection model, and use the test set to test the pattern detection model; preset a coordinate error threshold, and when the mean of the predicted coordinate errors of all reference images in the test set is less than the coordinate error threshold, the pattern detection model training is completed, and the pattern detection model is output; wherein the calculation formula for the mean of the predicted coordinate error is:

[0073]

[0074] Among them ZB b is the predicted coordinate error, b is the number of the reference image, C is the number of reference images in the test set, and x be is the predicted horizontal coordinate of a vertex, x′ be is the actual horizontal coordinate of a vertex, y be is the predicted ordinate of a vertex, y′ be is the actual vertical coordinate of a vertex, e is the number of the vertex of the rectangular box, and the rectangular box has 4 vertices in total; the coordinate error threshold is pre-set according to the accuracy required by the pattern detection model; the pattern detection model is specifically a convolutional neural network model.

[0075] The method for obtaining the coordinates of the benchmark key points is:

[0076] The Sobel operator is used to calculate the horizontal gradient and vertical gradient of each pixel in the reference image. The Sobel operator is a prior art and will not be described in detail here. The horizontal gradient is the rate of change of the grayscale value of the pixel in the horizontal direction, and the vertical gradient is the rate of change of the grayscale value of the pixel in the vertical direction. A window is constructed to calculate the structure tensor of each pixel in the reference image. The window is centered on the pixel to be calculated, and the pixel to be calculated is the pixel for which the structure tensor calculation is being performed. In this embodiment, the preferred window is 3×3. The expression of the structure tensor is:

[0077]

[0078] Where M is the structure tensor, I fx is the horizontal gradient of the f-th pixel in the window, I fy is the vertical gradient of the f-th pixel in the window, f∈[1,g], g is the number of pixels in the window;

[0079] According to the structure tensor, the response value of each pixel in the reference image is calculated; the expression of the response value is: h = A-kA′, Where h is the response value, A is the determinant, A′ is the trace, k is the empirical parameter, k∈[0.04,0.06];

[0080] The reference image is divided into regions according to the size of the window to obtain p regions, where p is an integer greater than 1; the response values ​​corresponding to all pixel points in each region are sorted from large to small to generate a sorting table, and the sorting table corresponds to the region one by one; the pixel point corresponding to the response value ranked at the front in each sorting table is retained and marked as a maximum point; a preset response threshold is preset by a technician in this field according to actual conditions, the response value of each maximum point is compared with the response threshold, and the maximum point with a response value greater than the response threshold is marked as a reference key point, and the coordinates of the reference key point are obtained, and the maximum point with a response value less than or equal to the response threshold is not marked.

[0081] S4: Align the preprocessed image with the reference image, calculate the overall offset, and determine whether to generate an overall offset instruction.

[0082] Methods for aligning the preprocessed image with the reference image include:

[0083] Using the trained product detection model, the printed matter in the preprocessed image and the reference image are respectively annotated with rectangular frames to obtain the second rectangular frame and the third rectangular frame; the second rectangular frame is the rectangular frame corresponding to the printed matter in the preprocessed image, and the third rectangular frame is the rectangular frame corresponding to the printed matter in the reference image; the vertex coordinates of the second rectangular frame are used as the real-time product coordinates; the vertex coordinates of the third rectangular frame are used as the reference product coordinates; the training method of the product detection model is consistent with the training method of the pattern detection model, and both are convolutional neural network models;

[0084] Define the alignment matrix D, Among them, t1 and t3 control scaling and rotation, t2 and t4 control tilt, and t5 and t6 are translations; each coordinate in the real-time product coordinates is matched one by one with each coordinate in the reference product coordinates, and the corresponding two coordinates are used as a set of first coordinate sets in turn; the alignment matrix is ​​calculated based on the four sets of first coordinate sets, and the calculation expression is: D = EF -1 Where, F -1 is the inverse matrix of F, x i is the horizontal coordinate corresponding to the coordinate of the reference product in the first coordinate set of the i-th group, y i is the ordinate corresponding to the coordinate of the reference product in the first coordinate set of the i-th group, x i ′ is the horizontal coordinate corresponding to the real-time product coordinate in the first coordinate set of the i-th group, y i ′ is the ordinate corresponding to the real-time product coordinate in the i-th group of first coordinate sets, i∈[1,4]; the four groups of first coordinate sets are sequentially substituted into the calculation expression of the alignment matrix and combined to calculate the values ​​corresponding to t1, t2, t3, t4, t5, and t6 in the alignment matrix D;

[0085] The trained pattern detection model is used to mark the printed pattern in the preprocessed image with a rectangular frame to obtain a fourth rectangular frame; the vertex coordinates of the fourth rectangular frame are used as real-time vertex coordinates; each vertex coordinate in the real-time vertex coordinates is multiplied by the alignment matrix to obtain aligned vertex coordinates.

[0086] Methods for calculating the overall offset include:

[0087] Make a one-to-one correspondence between each coordinate in the real-time vertex coordinates and each coordinate in the reference vertex coordinates, and use the corresponding two coordinates as a set of second coordinate sets in turn; replace the real-time vertex coordinates in each set of second coordinates with the corresponding aligned vertex coordinates; calculate the Euclidean distance between the two coordinates in each set of second coordinates in turn, and use it as the offset; add each offset in turn, and then divide it by 4 to obtain the overall offset.

[0088] Methods for determining whether to generate an overall offset instruction include:

[0089] The offset threshold is calculated dynamically, and the overall offset is compared with the offset threshold; if the overall offset is less than the offset threshold, no overall offset instruction is generated; if the overall offset is greater than or equal to the offset threshold, an overall offset instruction is generated.

[0090] like Figure 2 As shown, the steps of dynamically calculating the offset threshold include:

[0091] Step S401: setting the search interval [m,n] and precision ε;

[0092] Step S402: Determine the search sequence F(j);

[0093] Step S403: Determine the partition coefficient F(z) from the search sequence;

[0094] Step S404: using the partition coefficient to divide the search interval to obtain two points ψ1 and ψ2;

[0095] Step S405: Calculate the evaluation values ​​corresponding to ψ1 and ψ2 respectively;

[0096] Step S406: updating the search interval;

[0097] Step S407: Calculate the width υ of the search interval;

[0098] Step S408: If the width υ is less than or equal to the precision ε, divide the width υ by 2 as the offset threshold; if the width υ is greater than the precision ε, return to step S403.

[0099] In the above step S401, the search interval [m, n] and the precision ε are preset by those skilled in the art according to actual conditions.

[0100] In the above step S402, the search sequence

[0101] In the above step S403, the method for determining the partition coefficient F(z) is: calculating the initial partition coefficient δ′, Subtract the initial partition coefficient from each value in the search sequence to obtain the numerical difference; sort each numerical difference from small to large, obtain the numerical value corresponding to the numerical difference at the front, and use it as the partition coefficient F(z).

[0102] In the above step S404, Wherein, F(z-2) is the value that ranks before the partition coefficient F(z) in the search sequence, and F(z-1) is the value that ranks before the partition coefficient F(z) in the search sequence.

[0103] In the above step S405, the method for obtaining the evaluation value is: using the numerical value corresponding to the image label and the point as analysis data, inputting the analysis data into the trained threshold evaluation model, and predicting the corresponding evaluation value; the training process of the threshold evaluation model includes.

[0104] Collect r groups of analysis data in advance, set corresponding evaluation values ​​for the r groups of analysis data, where r is an integer greater than 1, and convert the analysis data and the corresponding evaluation values ​​into a corresponding set of feature vectors; the evaluation values ​​corresponding to the analysis data are collected by a person skilled in the art during the historical pattern shift analysis process, and each group of analysis data is analyzed in turn in combination with actual experience, and the degree of adaptation between the image label and the point in each group of analysis data is evaluated, and the corresponding evaluation values ​​are used as the corresponding evaluation values, and the corresponding evaluation values ​​are set in turn for the a group of analysis data;

[0105] Each set of feature vectors is used as the input of the threshold evaluation model. The threshold evaluation model takes a set of predicted evaluation values ​​corresponding to each set of analysis data as output, and takes the actual evaluation value corresponding to each set of analysis data as the prediction target. The actual evaluation value is the pre-set evaluation value corresponding to the analysis data; the training goal is to minimize the sum of the prediction errors of all analysis data; wherein the calculation formula of the prediction error is η K =(β K -α K ) 2 , where η K is the prediction error, K is the group number of the eigenvector corresponding to the analyzed data, β K is the predicted evaluation value corresponding to the Kth group of analysis data, α K is the actual evaluation value corresponding to the Kth group of analysis data; the threshold evaluation model is trained until the sum of the prediction errors reaches convergence and the training is stopped.

[0106] The above threshold evaluation model is specifically a deep neural network model; it includes an input layer, a hidden layer and an output layer; each hidden layer includes multiple neurons, each neuron is connected to the neurons in the next layer, and the connection contains weights, which determine the importance and influence of data transmission in the neural network; an activation function is applied to each neuron between the hidden layer and the output layer, and the activation function introduces nonlinearity, allowing the network to learn more complex patterns and features.

[0107] In the above step S406, the method for updating the search interval is: mark the evaluation value corresponding to ψ1 as the first evaluation value ω1, and mark the evaluation value corresponding to ψ2 as the second evaluation value ω2; if ω1<ω2, the search interval is updated to [ω1,n], that is, m=ω1; if ω1≥ω2, the search interval is updated to [m,ω2], that is, n=ω2.

[0108] In the above step S407, the width υ is obtained by subtracting the minimum value from the maximum value of the search interval.

[0109] S5: If the overall offset instruction is not generated, the pre-processed image is segmented to obtain a segmented image, so as to eliminate the interference of background noise and improve the subsequent extraction accuracy of real-time key point coordinates.

[0110] Methods for obtaining segmented images include:

[0111] Preset boundary threshold; the boundary threshold is obtained by technicians in this field by analyzing multiple historical images to obtain edge points corresponding to each historical image; the historical image is a pre-processed image obtained at a historical moment, and the edge point is a pixel point corresponding to the edge of the printed pattern; the pixel values ​​of the edge points corresponding to each historical image are sorted from small to large, and the gray value at the front is obtained as the minimum gray value of the corresponding historical image; the average of the minimum gray values ​​corresponding to all historical images is used as the boundary threshold;

[0112] Compare the grayscale value of each pixel in the preprocessed image with the boundary threshold; mark the pixel whose grayscale value is greater than or equal to the boundary threshold as a pattern point, and do not mark the pixel whose grayscale value is less than the boundary threshold; count the number of adjacent pixels of each pixel in the preprocessed image and mark them as adjacent numbers; sort each adjacent number from large to small, obtain the adjacent number in the front, and mark it as the maximum number; mark the pixel whose adjacent number is not the maximum number as a boundary point;

[0113] Analyze the pixels adjacent to each pattern point; if the pixels adjacent to the pattern point are not all pattern points, mark the corresponding pattern point as an edge point; if the pattern point is marked as a boundary point and the adjacent pixels are all pattern points, mark the corresponding pattern point as an edge point; if the pattern point is not marked as a boundary point and the adjacent pixels are all pattern points, do not mark the corresponding pattern point; perform image segmentation on the preprocessed image based on all edge points corresponding to the preprocessed image; mark the image including all pattern points in the segmented image as a segmented image.

[0114] S6: Perform key point matching on the segmented image, calculate the local offset, and determine whether to generate a local offset instruction.

[0115] Methods for calculating local offsets include:

[0116] The real-time key point coordinates in the segmented image are obtained in the same way as the reference key point coordinates; each coordinate in the real-time key point coordinates is multiplied by the alignment matrix to obtain the alignment key point coordinates; each coordinate in the alignment key point coordinates is matched one-to-one with each coordinate in the reference key point coordinates, and the corresponding two coordinates are used as a set of third coordinate sets in turn; the real-time key point coordinates in each set of third coordinates are replaced with the corresponding alignment key point coordinates; the Euclidean distance between the two coordinates in each set of third coordinates is calculated in turn and used as the local offset.

[0117] Methods for determining whether to generate a local offset instruction include:

[0118] Each local offset is compared with the offset threshold. If the local offset is less than the offset threshold, no local offset instruction is generated. If the local offset is greater than or equal to the offset threshold, a local offset instruction is generated and the corresponding real-time key point coordinates are marked as offset coordinates to quickly identify the specific offset position of the printed pattern in the printed image.

[0119] This embodiment uses deep learning technology to achieve image alignment and key point matching between the printed image and the reference image; and by calculating the overall offset and the local offset, quantitatively detects whether the printed pattern in the printed product has an overall or local offset, thereby achieving real-time detection of the quality of the printed product; it can automatically identify pattern offsets in the printing process, reduce manual intervention and errors, and reduce the scrap rate, thereby improving the production efficiency and product quality of printed products, and meeting the modern printing industry's needs for high precision and high efficiency.

[0120] Example 2

[0121] The present application also provides an electronic device. The electronic device may include one or more processors and one or more memories. The memories store computer-readable codes, and when the computer-readable codes are executed by the one or more processors, the above-mentioned online printing quality detection method based on artificial intelligence can be executed.

[0122] The method or system according to the implementation mode of the present application can also be implemented with the aid of the architecture of the electronic device shown in the present application. The electronic device may include a bus, one or more CPUs, ROM, RAM, a communication port connected to a network, input / output, a hard disk, etc. A storage device in the electronic device, such as a ROM or a hard disk, can store an online printing quality detection method based on artificial intelligence provided in the present application. Furthermore, the electronic device may also include a user interface. Of course, the architecture shown in the present application is only exemplary. When implementing different devices, one or more components in the electronic device shown in the present application may be omitted according to actual needs.

[0123] Example 3

[0124] As shown, one embodiment of the present application discloses a computer-readable storage medium. Computer-readable instructions are stored on the computer-readable storage medium. When the computer-readable instructions are executed by the processor, an artificial intelligence-based online printing quality detection method according to the embodiment of the present application described with reference to the above figures can be executed. The storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0125] In addition, according to the implementation of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided by the present application, for example: an online printing quality detection method based on artificial intelligence. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed.

[0126] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0127] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A printing quality online detection method based on artificial intelligence, characterized in that: include: S1: Collect printed images; S2: preprocessing the printed image to obtain a preprocessed image; S3: Acquire a reference image corresponding to the printed image; S4: aligning the preprocessed image with the reference image, calculating the overall offset, and determining whether to generate an overall offset instruction; S5: If the overall offset instruction is not generated, the preprocessed image is segmented to obtain a segmented image; S6: Perform key point matching on the segmented image, calculate the local offset, and determine whether to generate a local offset instruction.

2. The method for online printing quality detection based on artificial intelligence according to claim 1, characterized in that: The method for collecting printed matter images is as follows: when a printed matter is detected, the detection time t1 is obtained; the conveyor belt speed v1 and the sensor distance d1 are obtained; and the collection time t2 is obtained by adding the ratio of the sensor distance d1 to the conveyor belt speed v1 and the detection time t1. The expression of the collection time t2 is: When the acquisition time t2 is reached, the printed image is acquired; The method for obtaining the preprocessed image comprises: Extract the RGB value of each pixel in the printed image, where the RGB value includes the red channel value R, the green channel value G and the blue channel value B; The weighted average method is used for the RGB value of each pixel to calculate the corresponding gray value; the expression of the gray value is: Gray = 0.299R + 0.587G + 0.114B; where Gray is the gray value; the preprocessed image is obtained according to the gray values ​​of all pixels.

3. The method for online printing quality detection based on artificial intelligence according to claim 2, characterized in that: Construct a benchmark set, which includes different image labels and corresponding benchmark data, and the benchmark data includes a benchmark image and benchmark information; the image label is a digital label corresponding to the printed image, and different printed images have different corresponding digital labels; the corresponding image label is obtained according to the printed image, and then the benchmark data corresponding to the image label is obtained according to the benchmark set; the benchmark image is a printed image that meets the quality standards; the benchmark information includes the coordinates of the benchmark vertex and the coordinates of the benchmark key point; The method for obtaining the reference vertex coordinates is: Using the trained pattern detection model, annotate the printed pattern in the reference image with a rectangular frame to obtain a first rectangular frame; The vertex coordinates of the first rectangular frame are used as the reference vertex coordinates; Printed patterns are patterns printed on printed materials; The training method of the pattern detection model includes: A number of different reference images are collected in advance, where a is an integer greater than 1, and a rectangular frame is marked for the printed pattern in each reference image; the marked reference images are divided into a training set and a test set, and a pattern detection model is trained using the training set, and the pattern detection model is tested using the test set; a coordinate error threshold is preset, and when the mean of the predicted coordinate errors of all reference images in the test set is less than the coordinate error threshold, the pattern detection model training is completed, and the pattern detection model is output; the pattern detection model is a convolutional neural network model.

4. The method for online printing quality detection based on artificial intelligence according to claim 3 is characterized in that: The method for obtaining the coordinates of the benchmark key points is: The Sobel operator is used to calculate the horizontal gradient and vertical gradient of each pixel in the reference image; the horizontal gradient is the rate of change of the grayscale value of the pixel in the horizontal direction, and the vertical gradient is the rate of change of the grayscale value of the pixel in the vertical direction; Construct a window to calculate the structure tensor of each pixel in the reference image. The window is centered on the pixel to be calculated. The pixel to be calculated is the pixel for which the structure tensor is being calculated. The expression of the structure tensor is: Where M is the structure tensor, I fx is the horizontal gradient of the f-th pixel in the window, I fy is the vertical gradient of the f-th pixel in the window, f∈[1,g], g is the number of pixels in the window; According to the structure tensor, the response value of each pixel in the reference image is calculated; the expression of the response value is: h = A-kA′, Where h is the response value, A is the determinant, A′ is the trace, k is the empirical parameter, k∈[0.04,0.06]; Divide the reference image into regions according to the size of the window to obtain p regions, where p is an integer greater than 1; sort the response values ​​corresponding to all pixels in each region from large to small to generate a sorting table, which corresponds to the regions one by one; retain the pixel corresponding to the response value ranked at the front in each sorting table and mark it as a maximum point; A response threshold is preset, and the response value of each maximum point is compared with the response threshold. The maximum points with response values ​​greater than the response threshold are marked as benchmark key points, and the coordinates of the benchmark key points are obtained. The maximum points with response values ​​less than or equal to the response threshold are not marked.

5. The method for online printing quality detection based on artificial intelligence according to claim 4 is characterized in that: The method for aligning the preprocessed image with the reference image comprises: Using the trained product detection model, the printed matter in the preprocessed image and the reference image are respectively annotated with rectangular frames to obtain the second rectangular frame and the third rectangular frame; the second rectangular frame is the rectangular frame corresponding to the printed matter in the preprocessed image, and the third rectangular frame is the rectangular frame corresponding to the printed matter in the reference image; the vertex coordinates of the second rectangular frame are used as the real-time product coordinates; the vertex coordinates of the third rectangular frame are used as the reference product coordinates; the training method of the product detection model is consistent with the training method of the pattern detection model, and both are convolutional neural network models; Define the alignment matrix D, Among them, t1 and t3 control scaling and rotation, t2 and t4 control tilt, and t5 and t6 are translations; each coordinate in the real-time product coordinates is matched one by one with each coordinate in the reference product coordinates, and the corresponding two coordinates are used as a set of first coordinate sets in turn; the alignment matrix is ​​calculated based on the four sets of first coordinate sets, and the calculation expression is: D = EF -1 Where, F -1 is the inverse matrix of F, x i is the horizontal coordinate corresponding to the coordinate of the reference product in the first coordinate set of the i-th group, y i is the ordinate corresponding to the coordinate of the reference product in the first coordinate set of the i-th group, x i ′ is the horizontal coordinate corresponding to the real-time product coordinate in the first coordinate set of the i-th group, y i ′ is the ordinate corresponding to the real-time product coordinate in the i-th group of first coordinate sets, i∈[1,4]; the four groups of first coordinate sets are sequentially substituted into the calculation expression of the alignment matrix and combined to calculate the values ​​corresponding to t1, t2, t3, t4, t5, and t6 in the alignment matrix D; The trained pattern detection model is used to mark the printed pattern in the preprocessed image with a rectangular frame to obtain a fourth rectangular frame; the vertex coordinates of the fourth rectangular frame are used as real-time vertex coordinates; each vertex coordinate in the real-time vertex coordinates is multiplied by the alignment matrix to obtain aligned vertex coordinates.

6. The method for online printing quality detection based on artificial intelligence according to claim 5 is characterized in that: The method for calculating the overall offset includes: Make a one-to-one correspondence between each coordinate in the real-time vertex coordinates and each coordinate in the reference vertex coordinates, and use the corresponding two coordinates as a set of second coordinate sets in turn; replace the real-time vertex coordinates in each set of second coordinates with the corresponding aligned vertex coordinates; calculate the Euclidean distance between the two coordinates in each set of second coordinates in turn, and use it as an offset; add each offset in turn, and then divide it by 4 to obtain the overall offset; The method for determining whether to generate an overall offset instruction includes: The offset threshold is calculated dynamically, and the overall offset is compared with the offset threshold; if the overall offset is less than the offset threshold, no overall offset instruction is generated; if the overall offset is greater than or equal to the offset threshold, an overall offset instruction is generated.

7. The method for online printing quality detection based on artificial intelligence according to claim 6 is characterized in that: The step of dynamically calculating the offset threshold comprises: Step S401: setting the search interval [m,n] and precision ε; Step S402: Determine the search sequence F(j); Step S403: Determine the partition coefficient F(z) from the search sequence; Step S404: using the partition coefficient to divide the search interval to obtain two points ψ1 and ψ2; Step S405: Calculate the evaluation values ​​corresponding to ψ1 and ψ2 respectively; Step S406: updating the search interval; Step S407: Calculate the width υ of the search interval; Step S408: If the width υ is less than or equal to the precision ε, divide the width υ by 2 as the offset threshold; if the width υ is greater than the precision ε, return to step S403.

8. The method for online printing quality detection based on artificial intelligence according to claim 7, characterized in that: In step S402, the search sequence In step S403, the method for determining the partition coefficient F(z) is: calculating the initial partition coefficient δ′, Subtract the initial partition coefficient from each value in the search sequence to obtain the value difference; sort each value difference from small to large, obtain the value corresponding to the first value difference, and use it as the partition coefficient F(z); In step S404, Wherein, F(z-2) is the value of the first two digits of the search sequence before the partition coefficient F(z), and F(z-1) is the value of the first digit of the search sequence before the partition coefficient F(z); In step S405, the method for obtaining the evaluation value is: taking the numerical value corresponding to the image label and the point as analysis data, inputting the analysis data into a trained threshold evaluation model, and predicting the corresponding evaluation value, where the threshold evaluation model is a deep neural network model; In step S406, the method for updating the search interval is: marking the evaluation value corresponding to ψ1 as the first evaluation value ω1, and marking the evaluation value corresponding to ψ2 as the second evaluation value ω2; if ω1<ω2, the search interval is updated to [ω1,n], that is, m=ω1; if ω1≥ω2, the search interval is updated to [m,ω2], that is, n=ω2; In step S407, the width υ is obtained by subtracting the minimum value from the maximum value of the search interval.

9. The method for online printing quality detection based on artificial intelligence according to claim 8, characterized in that: The method for obtaining the segmented image comprises: Preset a boundary threshold; compare the gray value of each pixel in the preprocessed image with the boundary threshold; mark the pixel whose gray value is greater than or equal to the boundary threshold as a pattern point, and do not mark the pixel whose gray value is less than the boundary threshold; count the number of adjacent pixels of each pixel in the preprocessed image and mark them as adjacent numbers; sort each adjacent number from large to small, obtain the adjacent number in the front, and mark it as the maximum number; mark the pixel whose adjacent number is not the maximum number as a boundary point; Analyze the pixels adjacent to each pattern point; if the pixels adjacent to the pattern point are not all pattern points, mark the corresponding pattern point as an edge point; if the pattern point is marked as a boundary point and the adjacent pixels are all pattern points, mark the corresponding pattern point as an edge point; if the pattern point is not marked as a boundary point and the adjacent pixels are all pattern points, do not mark the corresponding pattern point; perform image segmentation on the preprocessed image based on all edge points corresponding to the preprocessed image; mark the image including all pattern points in the segmented image as a segmented image.

10. The method for online printing quality detection based on artificial intelligence according to claim 9, characterized in that: The method for calculating the local offset comprises: Acquire the real-time key point coordinates in the segmented image. The acquisition method of the real-time key point coordinates is consistent with the acquisition method of the reference key point coordinates; multiply each coordinate in the real-time key point coordinates by the alignment matrix to obtain the alignment key point coordinates; make one-to-one correspondence between each coordinate in the alignment key point coordinates and each coordinate in the reference key point coordinates, and sequentially use the corresponding two coordinates as a set of third coordinate sets; replace the real-time key point coordinates in each set of third coordinate sets with the corresponding alignment key point coordinates; sequentially calculate the Euclidean distance between the two coordinates in each set of third coordinate sets and use it as the local offset; The method for determining whether to generate a local offset instruction comprises: Each local offset is compared with the offset threshold. If the local offset is less than the offset threshold, no local offset instruction is generated; if the local offset is greater than or equal to the offset threshold, a local offset instruction is generated and the corresponding real-time key point coordinates are marked as offset coordinates.

Images