Printing quality detection and correction method and system based on image recognition and storage medium

Abstract

The invention relates to a printing quality detection and correction method and system based on image recognition and a storage medium, and belongs to the technical field of image detection.The method comprises the steps that image information of a printed product is obtained, the image information is preprocessed, a preprocessing result is obtained, and a printing area of the current printed product is determined according to the preprocessing result; the method comprises the steps of constructing an image recognition model, inputting pre-selected and trained image data into the image recognition model, obtaining feature data of incomplete frequent data of a current printing area, constructing a knowledge graph, and inputting the feature data into the knowledge graph to obtain associated information, printing parameters of the current printing equipment are corrected according to the associated information, and a correction instruction is generated. And the quality inspection quality and efficiency of printed products are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image detection, and in particular, relates to a printing quality detection and correction method and system based on image recognition, and a storage medium. Background Art

[0002] In the prior art, when performing printed product defect detection, the paper is generally moved by rotating two parallel rollers, and a snapshot camera is placed above the paper to capture the printed image on the paper. By matching the printed information in the captured image with the base plate information, it is determined whether the printed product has a printing curve.

[0003] However, there are the following difficulties in the intelligent quality inspection process of printed products: the existing technology uses a camera to take pictures, compares the pictures with standard pictures, removes unqualified products, and stacks qualified products. This method still requires manual operation and control, and there are still missed inspections and wrong inspections, which lead to unqualified products entering the market. Summary of the invention

[0004] In order to solve the technical problems existing in the above background technology, the present invention provides a printing quality detection and correction method based on image recognition, comprising the following steps:

[0005] The image information of the printed product is obtained, and the image information is preprocessed to obtain a preprocessing result, and the printing area of ​​the current printed product is determined according to the preprocessing result. The preprocessing of the image information includes performing a random deformation operation on the defect image to obtain a new defect image, performing a random deformation operation on the defect image to obtain a new defect image, and the deformation operation process formula is as follows:

[0006] D′(x, y)=T[D(x, y)],

[0007] Where: T is the random deformation operation; D(x, y) is the intercepted curve image; D′(x, y) is the new defect image obtained after the deformation operation;

[0008] Constructing an image recognition model, and inputting the pre-selected image data into the image recognition model to obtain a trained image recognition model, and obtaining an output result of the image information of the current printed product through the image recognition model;

[0009] If the output result is the preset output result, the defective area of ​​each printed product is obtained, feature data of the defective area is extracted and counted, and frequent defect data and occasional defect data are obtained based on the feature data;

[0010] Acquire feature data of frequent incomplete data in the current printing area, construct a knowledge graph, and input the feature data into the knowledge graph to obtain associated information. Correct the printing parameters of the current printing device according to the associated information, generate correction instructions, and transmit the correction instructions to the remote printing control terminal.

[0011] Furthermore, a random rotation operation of 0° to 180° is performed on the acquired image, and a number of images are randomly generated for each defect type and saved as PNG (Portable Network Graphics) format images containing transparency information.

[0012] Furthermore, a standard printed sample for detection is selected, and the randomly generated defective images are randomly superimposed on different positions of the standard printed sample to generate a data set of defective samples of several printed products containing different defects, which is expressed by the following formula:

[0013] f(x, y) i =B(x, y)+D i (x, y, θ),

[0014] Where: i∈{1, 2, …, n}, n is the number of generated data sets; (x, y) represents the position of the image pixel, f(x, y) i is the image generated after superposition; B(x, y) represents the sample image without defects; θ is the rotation angle of the defective image, and Di(x, y, θ) is the defect obtained after random deformation and rotation.

[0015] Furthermore, preprocessing the image information also includes adding noise to the randomly generated print defect data set, and the added noise types include salt and pepper noise and Gaussian noise.

[0016] Furthermore, the image is blurred on the basis of adding noise, wherein the image blurring includes mean filtering and median filtering. The model of data enhancement is expressed as follows:

[0017] B i (x, y) = h(x, y) * f(x, y) i +n(x,y),

[0018] Where: i∈{1, 2, …, n}, n is the number of generated data sets; h(x, y) represents the filtering system; n(x, y) represents the noise added to the defect image.

[0019] Furthermore, the image dataset B i (x, y) is divided into blocks with a size of 100×100 pixels, and image blocks that do not meet the size are padded with a pixel value of 255.

[0020] Furthermore, the image recognition model is constructed, and the pre-selected image data is input into the image recognition model to obtain a trained image recognition model, and the image information of the current printed product is outputted by the image recognition model, specifically including the following steps:

[0021] An image recognition model is constructed based on a convolutional neural network, and the training set is input into the convolution layer of the image recognition model for convolution operation to obtain the convolution feature value;

[0022] The convolution feature value is input into the pooling layer, and the convolution feature value is reddened by using the maximum pooling method;

[0023] The parameter propagation training is performed through the cross entropy loss function until the error value converges to the preset value, and finally connected with the softmax classifier. The current image recognition model is tested through the test set until the test result meets the preset test result and the model parameters are saved.

[0024] The image information of the current printed product is input into the image recognition model for recognition to obtain an output result.

[0025] Furthermore, the printing parameters of the current printing device are corrected according to the associated information, which specifically includes the following steps:

[0026] G7 uses chromaticity values ​​to clearly define "gray" based on the color of the substrate, and uses a formula to calculate the chromaticity values ​​of each tone when the gray balance of different substrates is calculated. The formula is as follows:

[0027] a t =a p ×(1-c)

[0028] b t =b p ×(1-c)

[0029] where a t is the target chromaticity, a p is the chromaticity value of the substrate, and c is the tone value;

[0030] Defines the tones in image reproduction and provides a correction method for the gray balance curve.

[0031] In a second aspect, the present invention provides a printing quality detection and correction system based on image recognition, comprising a memory and a processor;

[0032] The memory stores computer-executable instructions;

[0033] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the aforementioned method.

[0034] In a third aspect, the present invention provides a computer readable storage medium, wherein the computer readable storage medium stores computer executable instructions, and when the computer executable instructions are executed, the method as claimed in the preceding claim is implemented.

[0035] Beneficial effects of the present invention:

[0036] The present invention discloses a printing quality detection and correction method based on image recognition, which obtains a variety of possible defective images by performing random deformation operations on the acquired image information, and constructs an image recognition model based on the various possible defective images, thereby improving the accuracy of the model. After the prediction result is obtained through the model, the defective area of ​​the printed product is obtained, the feature data of the defective area is extracted and counted, a knowledge graph is constructed, and the feature data is input into the knowledge graph to obtain associated information, and the printing parameters of the current printing equipment are corrected according to the associated information to generate correction instructions, thereby solving the technical problems in the prior art of low efficiency in shooting the target printed product with a camera and the need for manual inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0038] Figure 1 It is a flowchart of the overall steps of a printing quality detection and correction method based on image recognition in an embodiment of the present invention;

[0039] Figure 2 4 is a flowchart of the specific steps of step S2 in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] The present invention provides a printing quality detection and correction method based on image recognition, such as Figure 1 As shown, the following steps are included:

[0042] Step S1, acquiring image information of a printed product, and preprocessing the image information to obtain a preprocessing result, and determining a printing area of ​​a current printed product according to the preprocessing result;

[0043] Furthermore, the preprocessing of the image information includes performing a random deformation operation on the defect image to obtain a new defect image;

[0044] For printing production, the number of defect samples is limited, and collecting some common print defect samples and regenerating the data set is an effective way to expand the samples. The defect content in the print sample is intercepted, the defect content is extracted, and the non-defective image and text part are removed to obtain the defect image. The defect image is randomly deformed to obtain a new defect image. Specifically, the deformation operation process formula is as follows:

[0045] D′(x, y)=T[D(x, y)],

[0046] Where: T is the random deformation operation; D(x, y) is the intercepted curve image; D′(x, y) is the new defect image obtained after the deformation operation.

[0047] Specifically, in the embodiment of the present application, a random rotation operation of 0° to 180° is also performed on the acquired image, and the random rotation operation can increase the diversity of the sample. Several images are randomly generated for each defect type and saved as PNG (Portable Network Graphics) format images containing transparency information. Specifically, in the embodiment of the present application, the defect types include missing prints, dirty spots, and satellite ink droplets. A standard printed sample for detection is selected, and the randomly generated defective images are randomly superimposed on different positions of the standard printed sample to generate a data set of 4,000 printed defective samples containing different defects, which is expressed by the following formula:

[0048] f(x, y) i =B(x, y)+D i (x, y, θ),

[0049] Where: i∈{1, 2, …, n}, n is the number of generated data sets; (x, y) represents the position of the image pixel, f(x, y) i is the image generated after superposition; B(x, y) represents the sample image without defects; θ is the rotation angle of the defective image, D i (x, y, θ) is the defect obtained after random deformation and rotation.

[0050] In an embodiment of the present application, defects can be superimposed at any position to generate a variety of defect images. Defects can be superimposed at any position to generate a variety of defect images. In a short period of time, some of the generated defect images cannot even be easily located and identified. The generated printed product defect data set can better simulate the actual printed product defect situation.

[0051] Furthermore, the preprocessing of the image information also includes adding noise to the randomly generated print defect data set, and the added noise types include salt and pepper noise and Gaussian noise.

[0052] Specifically, in order to simulate the camera shake in production, the image is blurred on the basis of adding noise to achieve data enhancement, where the image blurring includes mean filtering and median filtering. The data enhancement model is expressed as follows:

[0053] B i (x, y) = h(x, y) * f(x, y) i +n(x,y),

[0054] Where: i∈{1, 2, …, n}, n is the number of generated data sets; h(x, y) represents the filtering system; n(x, y) represents the noise added to the defect image.

[0055] In the embodiment of the present application, the noise ratio of salt and pepper noise is 0.0005, the mean parameter of Gaussian noise is 0, and the variance is set to 0.001. The two noises are randomly selected and added to generate the data set f(x, y) i Different positions of B i (x, y) is the generated defective print data set.

[0056] Furthermore, the processed image dataset B i (x, y) is divided into blocks with a size of 100×100 pixels. Image blocks that do not meet the size requirement are padded with a pixel value of 255 to meet the size requirement.

[0057] Furthermore, the segmented image blocks are cleaned to identify image blocks containing defective targets, and are classified according to their similarity. The classified data sets are used for model training.

[0058] Step S2, constructing an image recognition model, and inputting the pre-selected image data into the image recognition model to obtain a trained image recognition model, and obtaining an output result of the image information of the current printed product through the image recognition model;

[0059] Further, in a preferred embodiment of the present application, Figure 2As shown, step S2 also includes the following steps:

[0060] Step S210: construct an image recognition model based on a convolutional neural network, and input the training set into the convolution layer of the image recognition model to perform a convolution operation to obtain a convolution feature value;

[0061] Step S220: input the convolution feature value into the pooling layer, and redden the convolution feature value by using the maximum pooling method;

[0062] Step S230, perform parameter propagation training through the cross entropy loss function until the error value converges to a preset value, finally connect with the softmax classifier, and test the current image recognition model through the test set until the test result meets the preset test result, and save the model parameters;

[0063] Step S240: input the image information of the barcode of the current printed product into the image recognition model for recognition to obtain an output result.

[0064] Step S3: if the output result is the preset output result, the defective area of ​​each printed product is obtained, feature data of the defective area is extracted and counted, and frequent defect data and occasional defect data are obtained based on the feature data;

[0065] Step S4, obtain feature data of frequent incomplete data in the current printing area, construct a knowledge graph, and input the feature data into the knowledge graph to obtain associated information, correct the printing parameters of the current printing device according to the associated information, generate correction instructions, and transmit the correction instructions to the remote printing control terminal.

[0066] Further, in the embodiment of the present application, in step S4, the printing parameters of the current printing device are corrected according to the associated information, which specifically includes the following steps:

[0067] G7 uses chromaticity values ​​to clearly define "gray" based on the color of paper (substrate), and uses a formula to calculate the chromaticity values ​​of each tone when the gray balance of different printing materials is achieved. The formula is as follows:

[0068] a t =a p ×(1-c)

[0069] b t =b p ×(1-c)

[0070] where a t is the target chromaticity, a p is the chromaticity value of the substrate, and c is the tone value;

[0071] Tone is defined in image reproduction and a correction method for the gray balance curve is provided. Tone and gray balance are equally important factors in image reproduction.

[0072] Regarding "tone", the G7 specification provides two standard NPDC curves (Neutral Printing Density Curve), one is the K curve and the other is the CMY curve. After providing two standard NPDC curves, G7 also proposed a correction method for the gray balance curve. It can be done manually or automatically.

[0073] In the printing industry, equal amounts of C, M, and Y primary inks should produce grays of different brightness. Due to defects in ink manufacturing, equal amounts of three-color inks cannot produce neutral gray. Therefore, it is necessary to find the appropriate gradation values ​​of the three color plates when the three-color inks are superimposed to form neutral gray. This application is based on the control method of gray balance in the G7 process control. Through the analysis of experimental data, it can be known that before the gray balance correction, the ab value of the gray balance color block fluctuates between -4 and 4, which is unstable and inconsistent with the ab value of the theoretical gray balance data; after the correction, the ab value, although there are still some fluctuations, the difference between the theoretical gray balance data and the theoretical gray balance data does not exceed 0.5, which is normal. This proves that the gray balance control process based on G7 studied in this paper is simple and effective. On this basis, a gray balance control process based on G7 is designed, and the feasibility of the control process is verified by taking a large-format inkjet printer as an example.

[0074] On the other hand, the present application also implements printed image correction through the following steps:

[0075] Use bilateral filter to remove the noise generated during image acquisition and transmission;

[0076] Use the Canny operator to perform edge detection on the filtered image;

[0077] Perform thinning processing on the detected edge image to remove redundant pixels and extract skeleton information;

[0078] Extract the longest line information of the printing area (i.e. the edge contour of the printing area), improve the tanh function to correct the depth information of the printing area image, and control the range and intensity of the depth information change through parameters;

[0079] The Gamma function is improved to correct the depth information of the image, and the gray value range is controlled by the power function.

[0080] Specifically, the Gamma function expression is as follows:

[0081] y=α(1-x)γ

[0082] Among them, y is the gray value of the output pixel, x is the distance between the current pixel and the pollen center coordinates, γ is the parameter that controls the brightness change from the center point to the edge of the printing area, and α is the pre-assigned parameter of the Gamma function, and α is set to be less than 1.

[0083] On the other hand, the embodiment of the present application also includes a printing quality detection and correction system based on image recognition, including a memory and a processor, wherein the memory stores computer-executable instructions;

[0084] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the aforementioned printing quality detection and correction method based on image recognition.

[0085] On the other hand, an embodiment of the present application also includes a computer storable medium for implementing the aforementioned printing quality detection and correction method based on image recognition.

[0086] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0087] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0090] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

[0091] In the description of the specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0092] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A printing quality detection and correction method based on image recognition, characterized in that: The following steps are involved: The image information of the printed product is obtained, and the image information is preprocessed to obtain a preprocessing result, and the printing area of ​​the current printed product is determined according to the preprocessing result. The preprocessing of the image information includes performing a random deformation operation on the defective image to obtain a new defective image. The deformation operation process formula is as follows: Where: T is the random deformation operation; is the intercepted curve picture; is the new defect image obtained after the deformation operation; Constructing an image recognition model, and inputting the pre-selected image data into the image recognition model to obtain a trained image recognition model, and obtaining an output result of the image information of the current printed product through the image recognition model; If the output result is the preset output result, the defective area of ​​each printed product is obtained, feature data of the defective area is extracted and counted, and frequent defect data and occasional defect data are obtained based on the feature data; Acquire feature data of frequent incomplete data in the current printing area, construct a knowledge graph, and input the feature data into the knowledge graph to obtain associated information. Correct the printing parameters of the current printing device according to the associated information, generate correction instructions, and transmit the correction instructions to the remote printing control terminal.

2. The printing quality detection and correction method based on image recognition according to claim 1 is characterized in that: The acquired image is subjected to a random rotation operation of 0° to 180°, and a number of images are randomly generated for each defect type and saved as PNG (Portable Network Graphics) format images containing transparency information.

3. The printing quality detection and correction method based on image recognition according to claim 2 is characterized in that: Select a standard printed sample for detection, randomly superimpose the randomly generated defective images on different positions of the standard printed sample, and generate a data set of several printed defective samples containing different defects, which is expressed as follows: Where: i∈{1, 2, …, n}, n is the number of generated data sets; represents the position of the image pixel, is the image generated after superposition; represents a sample image without defects; θ is the rotation angle of the defective image, It is a defect obtained after random deformation and rotation.

4. The printing quality detection and correction method based on image recognition according to claim 1 is characterized in that: The preprocessing of the image information also includes adding noise to the randomly generated print defect data set, and the added noise types include salt and pepper noise and Gaussian noise.

5. The printing quality detection and correction method based on image recognition according to claim 4 is characterized in that: On the basis of adding noise, the image is also blurred, where the image blurring includes mean filtering and median filtering. The data enhancement model is expressed as follows: Where: i∈{1, 2, …, n}, n is the number of generated data sets; represents the filtering system; Represents the noise added to the defective image.

6. The printing quality detection and correction method based on image recognition according to claim 5 is characterized in that: Image dataset The image is divided into blocks with a size of 100×100 pixels. The image blocks that do not meet the size are padded with a pixel value of 255.

7. The printing quality detection and correction method based on image recognition according to claim 1 is characterized in that: The image recognition model is constructed, and the pre-selected image data is input into the image recognition model to obtain a trained image recognition model, and the image information of the current printed product is output by the image recognition model, specifically including the following steps: An image recognition model is constructed based on a convolutional neural network, and the training set is input into the convolution layer of the image recognition model for convolution operation to obtain the convolution feature value; The convolution feature value is input into the pooling layer, and the convolution feature value is reddened by using the maximum pooling method; The parameter propagation training is performed through the cross entropy loss function until the error value converges to the preset value, and finally connected with the softmax classifier. The current image recognition model is tested through the test set until the test result meets the preset test result and the model parameters are saved. The image information of the current printed product is input into the image recognition model for recognition to obtain an output result.

8. The printing quality detection and correction method based on image recognition according to claim 1 is characterized in that: Correcting the printing parameters of the current printing device according to the associated information specifically includes the following steps: G7 uses chromaticity values ​​to clearly define grayscale based on the color of the substrate, and uses a formula to calculate the chromaticity values ​​of each tone when the gray balance of different substrates is achieved. The formula is as follows: a t =a p ×(1-c) b t =b p ×(1-c) where a t is the target chromaticity, a p is the chromaticity value of the substrate, and c is the tone value; Defines the tones in image reproduction and provides a correction method for the gray balance curve.

9. A printing quality detection and correction system based on image recognition, comprising a memory and a processor; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 8.

10. A computer readable storage medium, wherein computer executable instructions are stored in the computer readable storage medium, and when the computer executable instructions are executed, the method according to any one of claims 1 to 8 is implemented.

Images