A color text printing ghosting quality inspection method and device based on image processing

Abstract

The present invention discloses a method and device for color text printing ghosting quality inspection based on image processing, and relates to the technical field of image quality inspection. The method proposed by the present invention first images the printing area of ​​a card or paper, then locates the text area in the image and classifies it to obtain a color text area, then divides the color text area to obtain a text block, and uses a classification algorithm to determine whether the text block has a printing offset, that is, "ghosting". Finally, the judgment results of all color text blocks are combined to perform voting statistics. When the voting result exceeds the set threshold, it is determined that the color printing area of ​​the target to be inspected has a ghosting defect, thereby reducing the risk of misjudgment. The printing quality of the color text in the printing area can be automatically inspected without manual processing, and the accuracy can be improved at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of image quality inspection, and in particular to a method and device for color text printing ghosting quality inspection based on image processing. Background Art

[0002] On some automated equipment, such as card making equipment, it is usually necessary to print colored text on a card or paper with a background pattern. Color printers use 4 or 6 basic colors to create different colors. Taking a printer with 4 basic colors as an example, various colors are mixed by combining cyan, magenta, yellow and black. If the print head of the printer is not aligned, the printing position of the printed colored text on different color channels will be offset, resulting in "ghosting" of the printed colored text. This ghosting will affect the clarity of the text, resulting in unqualified cards or printed paper. In some occasions where strict requirements are placed on the quality of cards, it is necessary to manually check the printed cards or paper to check whether the printed colored fonts have "ghosting". This manual processing is not only time-consuming and labor-intensive, but also prone to missed inspections, resulting in unqualified cards or paper being missed. Summary of the invention

[0003] In order to solve the technical defects of the prior art in the practice of color text printing ghosting, the present invention provides a color text printing ghosting quality inspection method and device based on image processing. The following technical solutions are adopted:

[0004] A color text printing ghosting quality inspection method based on image processing comprises the following steps:

[0005] Step 1, obtaining an image of the area to be inspected of the color printing area of ​​the target to be inspected;

[0006] Step 2, performing text detection on the image of the inspection area to obtain the text area;

[0007] Step 3, classify the text area to obtain a color text area;

[0008] Step 4, segmenting the colored text area to obtain multiple colored text blocks;

[0009] Step 5, using a classification algorithm to classify the multiple colored text blocks respectively, the classification categories include text with ghosting and text without ghosting;

[0010] Step 6, voting statistics are performed based on the classification judgment results of multiple color text blocks. When the number of classification statistics results of text with ghosting is greater than the set statistical threshold, it is determined that the color printing area of ​​the target to be inspected has a ghosting defect.

[0011] By adopting the above technical scheme, the color printing area of ​​the quality inspection target is first imaged, and then the text area is located in the image of the quality inspection area and classified to obtain the color text area, and then the color text area is segmented to obtain multiple color text blocks, and a classification algorithm is used to determine whether the color text block has printing deviation, that is, ghosting defect. Since errors may occur when a single color text block is judged by classification, based on the principle of multi-channel color printing, the occurrence of ghosting defects must be that each color text block should have a ghosting defect. Finally, the judgment results of all color text blocks are combined to perform voting statistics. When more than a set proportion of color text blocks are judged to have ghosting defects, the color printing area of ​​the quality inspection target is judged to have a ghosting defect, thereby reducing the risk of misjudgment. Or, when the average confidence value of all color text blocks classified as ghosting defects exceeds a set threshold, the color printing area of ​​the quality inspection target is judged to have a ghosting defect.

[0012] The colored text blocks and annotation results are used as training samples to input into the classification algorithm. Usually, a colored text area contains multiple colored text blocks, which greatly increases the amount of training samples and can improve the accuracy of the classification results.

[0013] Optionally, in step 1, the object to be inspected is a card or paper, and the text printing area of ​​the card or paper is imaged by a scanner or a camera to obtain an image of the area to be inspected.

[0014] By adopting the above technical solution, whether it is a scanner, a camera or other shooting equipment, a high-quality image of the area to be inspected can be obtained.

[0015] Optionally, in step 2, a text line detection algorithm is used to detect each text line, and the location of the text line is the text area.

[0016] By adopting the above technical solution, the text line detection algorithm here can be YOLO, PSENet or DBNet, which can accurately detect text lines.

[0017] Optionally, in step 3, SVM, adaboost, random forest, and deep learning-based classification algorithms are used to extract features, and softMax is used for classification.

[0018] By adopting the above technical solution, the classification algorithm here can be a traditional SVM, adaboost or random forest, or a classification algorithm based on deep learning, such as using ResNet, VGGNet, mobileNet to extract features and using softMax for classification, which can efficiently and accurately obtain the color text area.

[0019] Optionally, the specific method for text segmentation of the colored text area in step 4 is: segmenting the text lines of the colored text according to a fixed width to obtain multiple colored text blocks after segmentation, and the fixed width is the height of the text or a multiple of the text height.

[0020] By adopting the above technical solution, the color text area is divided into text blocks. The division can be performed in units of words or characters, so that the divided text blocks contain at least one word or character. The division can also be performed with a fixed width, and the division can be overlapped. Since the general printed font shape is close to a square, considering the interval between words, the fixed width can be set to the height of the word or a multiple of the height of the word.

[0021] Optionally, in step 5, multiple colored text blocks are scaled to the same resolution, feature extraction is performed on the colored text blocks using a convolutional neural network, and then the probability of each category is output using a fully connected layer and a softmax layer, and finally training is performed using a cross entropy loss function; in the testing phase, the colored text blocks to be tested are scaled to the same resolution, feature extraction is performed using a convolutional neural network, and then the probability of each category is output using a fully connected layer and a softmax layer, and the class with the highest probability is selected as the classification result, and the probability of belonging to the class is used as the confidence of the classification.

[0022] By adopting the above technical solution, when the printed colored text has ghosting, all the colored texts on the card or paper will have ghosting, so it can be judged whether there is ghosting based on a single word.

[0023] Optionally, the cross entropy loss function in step 5 is focalLoss, and the training samples of focalLoss are obtained by the following method:

[0024] First, the printed color text is split into four color channels by using a simulated ghosting generation method, and one or more color channels are randomly selected, and the printed part of the channel is randomly offset and pasted on the printed background image;

[0025] The colored text area is then segmented according to characters, and each segmented colored text block is a sample.

[0026] The text blocks are detected and segmented on real samples, and the obtained colored text blocks can also be used as training samples to fine-tune the classification network and improve the classification accuracy.

[0027] By adopting the above technical solution, each segmented colored text block is a sample, rather than the entire colored area as a sample, which greatly increases the amount of training samples. First, a large number of training samples are obtained by generating simulated ghost images to train the classification network, and then a small number of real samples are used to fine-tune the classification network, which can improve the accuracy of the classification network and solve the problem of low classification accuracy of deep learning models under the condition of lack of training samples.

[0028] Optionally, in step 6, a specific method for determining whether a ghosting defect exists in the color printing area of ​​the target to be inspected is: using a trained classification network to determine whether a ghosting defect exists in the color printing area of ​​the target to be inspected.

[0029] The specific method is:

[0030] (1) Use the trained classification network to classify each colored text block;

[0031] (2) Count the proportion of the number of colored text blocks classified as ghosting to the total number of text blocks;

[0032] (3) If the ratio in (2) is higher than the threshold value T, it is considered that the color printing area of ​​the target to be inspected has a ghosting defect, otherwise it is considered that there is no ghosting defect. The threshold value T can be set to 0.5.

[0033] You can also use the following method:

[0034] (1) Use the trained classification network to classify each colored text block and record the confidence level of its classification as a ghosting defect;

[0035] (2) Calculate the average confidence level of all colored text blocks classified as ghost defects;

[0036] (3) If the average value in step (2) is greater than the threshold value C, it is considered that there is a ghosting defect in the color printing area to be inspected, otherwise it is considered that there is no ghosting defect. The threshold value C can be set to 0.5.

[0037] By adopting the above technical solution, voting is performed on the classification results of all color text blocks, and it is determined whether double images appear in the color text printed on the card or paper according to the voting results.

[0038] A color text printing ghosting quality inspection device based on image processing is characterized in that it includes an image acquisition device, a memory and a processor, the image acquisition device is used to capture the image of the color printing area of ​​the target to be inspected, and communicate with the memory to exchange the image of the color printing area of ​​the target to be inspected, the memory stores a quality inspection program designed using a color text printing ghosting quality inspection method based on image processing, and the processor is communicated with the memory, runs the quality inspection program and outputs the quality inspection results.

[0039] By adopting the above technical solution, the image acquisition device is responsible for imaging the produced card or paper and acquiring the printed image. The processor is responsible for running the quality inspection program and performing quality inspection on the printed image. The processor can be a CPU, a GPU, an embedded processor, or a DSP. The memory is responsible for storing the color printed area image of the target to be inspected, the quality inspection program, and the quality inspection results.

[0040] A memory stores a quality inspection program designed using a color text printing ghosting quality inspection method based on image processing.

[0041] In summary, the present invention includes at least the following beneficial technical effects:

[0042] The present invention can provide a method and device for color text printing ghosting quality inspection based on image processing. The proposed method first images the printing area of ​​a card or paper, then locates the text area in the image and classifies it to obtain a color text area, then divides the color text area into text blocks, and uses a classification algorithm to determine whether the text blocks have printing deviation, i.e., "ghosting". Finally, the final result is obtained by comprehensively combining the judgments of all text blocks. The printing quality of the color text in the printing area can be automatically inspected without manual processing, and the accuracy can be improved at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of a method for quality inspection of color text printing ghosting based on image processing according to the present invention;

[0044] Figure 2 The figure is a comparison diagram of normally printed color text and ghosted color text. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0046] The embodiment of the invention discloses a method and a device for color text printing ghosting quality inspection based on image processing.

[0047] Reference Figure 1 and Figure 2 , a color text printing ghosting quality inspection method based on image processing, comprising the following steps:

[0048] Step 1, obtaining an image of the area to be inspected of the color printing area of ​​the target to be inspected;

[0049] Step 2, performing text detection on the image of the inspection area to obtain the text area;

[0050] Step 3, classify the text area to obtain a color text area;

[0051] Step 4, segmenting the colored text area to obtain multiple colored text blocks;

[0052] Step 5, using a classification algorithm to classify the multiple colored text blocks respectively, the classification categories include text with ghosting and text without ghosting;

[0053] Step 6, voting statistics are performed based on the classification judgment results of multiple color text blocks. When the number of classification statistics results of text with ghosting is greater than the set statistical threshold, it is determined that the color printing area of ​​the target to be inspected has a ghosting defect.

[0054] First, the color printing area of ​​the inspection target is imaged, and then the text area is located in the image of the inspection area and classified to obtain the color text area. Then, the color text area is segmented to obtain multiple color text blocks, and a classification algorithm is used to determine whether the color text block has printing deviation, that is, ghosting defect. Since there will be errors in the classification judgment of a single color text block, based on the principle of multi-channel color printing, the occurrence of ghosting defects must be that each color text block should have a ghosting defect. Finally, the judgment results of all color text blocks are combined to perform voting statistics. When more than a set proportion of color text blocks are judged to have ghosting defects, the color printing area of ​​the inspection target is judged to have ghosting defects, thereby reducing the risk of misjudgment.

[0055] The color text blocks and the annotation results are used as training samples to input the classification algorithm. Usually, a color text area contains multiple color blocks, which greatly increases the number of training samples and can improve the accuracy of the classification results. In actual applications, the number of samples with ghosting defects is small. The color printing area is divided into color blocks as training samples by splitting blocks, which are used to fine-tune the classification network and improve the classification accuracy. On the other hand, using color blocks as the input of the classification network can also reduce the size of the input image of the ghosting classification network, thereby reducing the amount of calculation.

[0056] In step 1, the object to be inspected is a card or paper, and the text printing area of ​​the card or paper is imaged by a scanner or a camera to obtain an image of the area to be inspected.

[0057] Whether it is a scanner, camera or other shooting equipment, it is sufficient to obtain a high-quality image of the area to be inspected.

[0058] In step 2, a text line detection algorithm is used to detect each text line, and the location of the text line is the text area.

[0059] The text line detection algorithm here can be YOLO, PSENet or DBNet. For the target to be inspected with printed version rules, such as cards, YOLO can usually detect the text line. For other targets to be inspected, PSENet or DBNet can be used to improve the accuracy of text line detection.

[0060] In step 3, SVM, adaboost, and random forest are used to extract features based on deep learning classification algorithms, and softMax is used for classification.

[0061] The classification algorithm here can be a traditional SVM, adaboost or random forest, or a classification algorithm based on deep learning, such as using ResNet, VGGNet, mobileNet to extract features and using softMax for classification, which can efficiently and accurately obtain the color text area.

[0062] The specific method of text segmentation for the color text area in step 4 is: segment the text line of the color text according to a fixed width to obtain multiple color text blocks after segmentation, and the fixed width is the height of the text or a multiple of the text height.

[0063] The colored text area is divided into text blocks. The division can be performed in units of words or characters, so that each divided text block contains at least one word or character. The division can be performed with a fixed width, and the division can be overlapped. The fixed width can be the height of the text or a multiple of the text height.

[0064] In step 5, multiple colored text blocks are scaled to the same resolution, and feature extraction is performed on the colored text blocks using a convolutional neural network. Then, the fully connected layer and the softmax layer are used to output the probabilities of each category, and finally, the cross-entropy loss function is used for training. In the test phase, the colored text blocks to be tested are scaled to the same resolution, feature extraction is performed using a convolutional neural network, and the probabilities of each category are output using a fully connected layer and a softmax layer. The class with the highest probability is selected as the classification result, and the probability of belonging to this class is used as the confidence of the classification.

[0065] When printed color text has ghosting, all the color text on the card or paper will have ghosting, so it is possible to determine whether there is ghosting based on individual words.

[0066] The cross entropy loss function in step 5 is focalLoss, and the training samples of focalLoss are obtained by the following method:

[0067] First, the printed color text is split into four color channels by using a simulated ghosting generation method, and one or more color channels are randomly selected, and the printed part of the channel is randomly offset and pasted on the printed background image;

[0068] The colored text area is then segmented according to characters, and each segmented colored text block is a sample.

[0069] Each segmented colored text block is a sample, rather than taking the entire colored area as a sample, which greatly increases the amount of training samples.

[0070] The text blocks are detected and segmented on real samples, and the obtained colored text blocks can also be used as training samples to fine-tune the classification network and improve the classification accuracy.

[0071] In step 6, the specific method for determining whether the color printing area of ​​the target to be inspected has a ghosting defect is: using a trained classification network to determine whether the color printing area of ​​the target to be inspected has a ghosting defect.

[0072] The specific method is:

[0073] (1) Use the trained classification network to classify each colored text block;

[0074] (2) Count the proportion of colored text blocks classified as ghosting to the total text blocks;

[0075] (3) If the ratio in (2) is higher than the threshold value T, it is considered that the color printing area of ​​the target to be inspected has a ghosting defect, otherwise it is considered that there is no ghosting defect. The threshold value T can be set to 0.5.

[0076] You can also use the following method:

[0077] (1) Use the trained classification network to classify each colored text block and record the confidence level of its classification as a ghosting defect;

[0078] (2) Calculate the average confidence level of all colored text blocks classified as ghost defects;

[0079] (3) If the average value in step (2) is greater than the threshold value C, it is considered that there is a ghosting defect in the color printing area to be inspected, otherwise it is considered that there is no ghosting defect. The threshold value C can be set to 0.5.

[0080] Voting is performed on the classification results of all color text blocks, and judging whether the color text printed on the card or paper has double images is based on the voting results.

[0081] The device for inspecting ghost images of color text printing based on image processing is characterized in that it includes an image acquisition device, a memory and a processor. The image acquisition device is used to capture the image of the color printing area of ​​the target to be inspected, and communicate with the memory to exchange the image of the color printing area of ​​the target to be inspected. The memory stores a quality inspection program designed using a color text printing ghosting quality inspection method based on image processing. The processor is connected to the memory for communication, runs the quality inspection program and outputs the quality inspection results.

[0082] The image acquisition device is responsible for imaging the produced card or paper and acquiring the printed image. The processor is responsible for running the quality inspection program and performing quality inspection on the printed image. The processor can be a CPU, GPU, embedded processor, or DSP. The memory is responsible for storing the color printed area image of the target to be inspected, the quality inspection program, and the quality inspection results.

[0083] A memory stores a quality inspection program designed using a color text printing ghosting quality inspection method based on image processing. Specific embodiments

[0084] 1. Obtain an image of the area to be inspected of the color printed area of ​​the target to be inspected;

[0085] 2. Perform text line detection on the image of the inspection area to obtain the positions of all text lines;

[0086] 3. Classify each text line as a colored text line to obtain all colored text lines;

[0087] 4. For each color text line, the text line is segmented with the height of the text line as a fixed width to obtain multiple color text blocks;

[0088] 5. Classify each text block as ghosting or not, and obtain the category and confidence of each text block;

[0089] 6. The ghost classification network uses mobileNet to extract features, connects a fully connected layer and a softmax layer, and uses focalLoss as the loss function; the training process includes two steps: (1) using samples obtained by simulating ghost generation to train the model; (2) using real ghost samples to fine-tune the model in step (1);

[0090] 7. Perform voting statistics based on the classification results of multiple color text blocks. For example, 5 color text blocks are cut out from the image of the area to be inspected, and the classification results and confidence of each text block are (ghost, 0.6), (ghost, 0.8), (ghost, 0.9), (non-ghost, 0.48), (non-ghost, 0.56), then the votes are counted in the following way: the number of text blocks judged as ghosts is 3, the number of text blocks without ghosts is 2, the proportion of text blocks judged as ghosts is 3 / 5=0.6, the threshold T=0.5 is set, and according to 0.6>0.5, the target to be inspected is judged to be a ghost. The voting results can also be counted in the following way: the average confidence of the ghost is judged to be {0.6+0.8+0.9+(1-0.48)+(1-0.56)} / 5=0.652, the threshold C is set to 0.5, and according to 0.652>0.5, the target to be inspected is judged to be a ghost.

[0091] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A color text printing ghosting quality inspection method based on image processing, characterized in that: The following steps are involved: Step 1, obtaining an image of the area to be inspected of the color printing area of ​​the target to be inspected; Step 2, performing text detection on the image of the inspection area to obtain the text area; Step 3, classify the text area to obtain a color text area; Step 4, segmenting the colored text area to obtain multiple colored text blocks; Step 5, using a classification algorithm to classify the multiple colored text blocks respectively, the classification categories include text with ghosting and text without ghosting; Step 6, voting statistics are performed based on the classification judgment results of multiple color text blocks. When the number of classification statistics results of text with ghosting is greater than the set statistical threshold, it is determined that the color printing area of ​​the target to be inspected has a ghosting defect; In step 5, multiple color text blocks are scaled to the same resolution, and a convolutional neural network is used to extract features of the color text blocks. Then, the fully connected layer and the softmax layer are used to output the probability of each category, and finally the cross entropy loss function is used for training. During the testing phase, the colored text blocks to be tested are scaled to the same resolution, and feature extraction is performed through a convolutional neural network. The probabilities of each category are output through a fully connected layer and a softmax layer. The class with the highest probability is selected as the classification result, and the probability of belonging to this category is used as the confidence of the classification.

2. The method for color text printing ghosting quality inspection based on image processing according to claim 1, characterized in that: In step 1, the object to be inspected is a card or paper, and the text printing area of ​​the card or paper is imaged by a scanner or a camera to obtain an image of the area to be inspected.

3. The method for color text printing ghosting quality inspection based on image processing according to claim 2, characterized in that: In step 2, a text line detection algorithm is used to detect each text line, and the location of the text line is the text area.

4. The method for color text printing ghosting quality inspection based on image processing according to claim 3, characterized in that: In step 3, SVM, adaboost, and random forest are used to extract features based on deep learning classification algorithms, and softMax is used for classification.

5. The method for color text printing ghosting quality inspection based on image processing according to claim 4, characterized in that: The specific method of text segmentation for the color text area in step 4 is: segment the text line of the color text according to a fixed width to obtain multiple color text blocks after segmentation, and the fixed width is the height of the text or a multiple of the text height.

6. The method for color text printing ghosting quality inspection based on image processing according to claim 5, characterized in that: The cross entropy loss function in step 5 is focalLoss, and the training samples of focalLoss are obtained by the following method: First, the printed color text is split into four color channels by using a simulated ghosting generation method, and one or more color channels are randomly selected, and the printed part of the channel is randomly offset and pasted on the printed background image; The colored text area is then segmented according to characters, and each segmented colored text block is a sample.

7. The method for color text printing ghosting inspection based on image processing according to claim 6, characterized in that: In step 6, the specific method for determining whether the color printing area of ​​the target to be inspected has a ghosting defect is: using a trained classification network to determine whether the color printing area of ​​the target to be inspected has a ghosting defect.

8. A color text printing ghosting quality inspection device based on image processing, characterized in that: The invention comprises an image acquisition device, a memory and a processor, wherein the image acquisition device is used to capture an image of a color printing area of ​​a target to be inspected, and communicate with the memory to exchange the image of the color printing area of ​​the target to be inspected, the memory stores a quality inspection program designed using a color text printing ghosting quality inspection method based on image processing as described in any one of claims 1 to 7, and the processor is connected in communication with the memory to run the quality inspection program and output the quality inspection result.

9. A memory, characterized in that: The memory stores a quality inspection program designed using a color text printing ghosting quality inspection method based on image processing as described in any one of claims 1 to 7.

Images