A method and device for detecting printed text defects based on contour structure analysis

Abstract

The present invention discloses a method and device for detecting defects of printed text based on contour structure analysis, and relates to the technical field of image quality inspection. First, a printed image to be detected is obtained; then, the printed image to be detected is preprocessed and aligned to an electronic printed original image; for the electronic printed original image, each character is cut out to obtain a character image block, and a candidate area of ​​the character is found to obtain a candidate image block. Binary segmentation is performed on the character image block and the candidate image block respectively. With a template matching method, a binary character template image is used as a template to search for the best matching position of the character in the binary candidate image. A picture of the same size as the binary character image is cut out at the best matching position in the binary candidate image, which is called a matching image. Contour analysis is performed on the matching image, the binary character image and the same image respectively to obtain respective contour trees, and finally, defect detection is achieved through contour tree matching, thereby solving the problem of missed detection in the prior art when small areas of printed text are missing.

Description

Technical Field

[0001] The present invention relates to the technical field of image quality inspection, and in particular to a method and device for detecting printed text defects based on contour structure analysis. Background Art

[0002] On some automated equipment, text printing is usually required. Sometimes the print head of the printing equipment fails, or impurities are mixed into the printer, which will cause defects such as less printing in some areas of the printed text or impurities. In the field of key certificate production, such as identity cards, Hong Kong and Macau passes, etc., strict requirements are placed on the quality of text printing. In order to ensure the quality of printing, it is usually necessary to manually check the printed text to ensure that the printed text is not missing, mixed with impurities, and other defects. This inspection usually consumes a lot of manpower and is prone to errors. At the same time, certificates are usually accompanied by complex backgrounds, which increases the difficulty of quality inspection of printed text.

[0003] During the imaging process, due to the influence of the shooting angle of the scanner or camera, the printed text may be stretched and deformed, resulting in false detection or missed detection when directly using the template matching method for defect detection: for example, after the letter O is flattened, using template matching for defect detection will result in false detection on the left and right sides of the text; and when a stroke is missing somewhere and causing the stroke to be broken, the area of ​​the missing print is very small and will be submerged in the noise, resulting in missed detection. Summary of the invention

[0004] In order to solve the defects of the prior art in printed text defect detection, the present invention provides a printed text defect detection method and device based on contour structure analysis. The following technical solutions are adopted:

[0005] A method for detecting printed text defects based on contour structure analysis comprises the following steps:

[0006] Step 1, obtaining a printed image to be detected;

[0007] Step 2: preprocessing the printed image to be inspected, including target segmentation and alignment, and aligning to the electronic printed original image;

[0008] Step 3, segmenting all individual characters from the electronic printed original image to obtain character image blocks;

[0009] Step 4, according to the position of each character image block in the electronic print original image, determine the position in the aligned print image, and appropriately enlarge it as a candidate area, and crop the candidate area to obtain a candidate image block; when enlarging, consider the size of the character image block as the basis, the width of the candidate area is 3 times the width of the character image block, and the height is 3 times the height of the character image block;

[0010] Step 5, binarizing the character image block and the candidate image block respectively;

[0011] Step 6, using a template matching method, taking the binary character image as a template, searching in the binary candidate images to find the best matching position of the template;

[0012] Step 7, perform pixel-by-pixel AND operation on the matching image and the binary character image to obtain the same image. In the binary image here, the value of each pixel can only be 0 or 1. The AND operation of two pixels is defined as a logical AND operation: when both pixels are 1, the result of the pixel AND is 1, and in other cases it is 0. The pixel-by-pixel AND operation means that for two binary images of the same size, the AND operation is performed pixel by pixel;

[0013] Step 8, perform contour analysis on the matching image, binary character image and same image respectively, find all outer contours and inner contours, and establish a contour tree structure: when one contour completely contains another contour, the latter is called the child node of the former;

[0014] Step 9: Match the contour tree of the binary character image with the contour tree of the same image. If they cannot match, it is considered that a missing print defect occurs.

[0015] By adopting the above technical solution, firstly, a camera or a scanner is used to shoot or scan the printed target to obtain the printed image to be detected; then the printed image to be detected is preprocessed and aligned to the electronic printed original image; the electronic printed original image is the electronic image sent to the printer for document printing. For the electronic printed original image, each character is cut out to obtain a character image block, and the candidate area of ​​the character is found in the aligned printed image to obtain the candidate image block. The character image block and the candidate image block are binary segmented respectively to obtain a binary character template image and a binary candidate image. Using the template matching method, the binary character template image is used as a template to search for the best matching position of the character in the binary candidate image. A picture of the same size as the binary character image is cut out at the best matching position in the binary candidate image, which is called a matching image. The matching and template images are pixel-by-pixel ANDed to obtain the same image. The matching image, the binary character image and the same image are contour analyzed respectively to obtain their respective contour trees, and finally defect detection is achieved through contour tree matching, solving the problem of missed detection of printing defects in small areas in the prior art.

[0016] Optionally, in step 9, the contour tree matching method is:

[0017] If the two trees have unequal levels or unequal numbers of nodes, the two trees are considered mismatched;

[0018] If the number of layers of both trees is 1 and the number of nodes in the layer is equal, the two trees are considered to match;

[0019] If the number of layers of two trees is greater than or equal to 2, when the number of subtrees of their root nodes is equal and there is a set of corresponding relationships, their subtrees are paired separately, and the subtrees in each pairing are matched, then the two trees are judged to match.

[0020] By adopting the above technical solution, when a small area of ​​printing missing causes a stroke break, the contour tree will undergo a large change. Capturing this change helps to improve the detection accuracy of small area printing missing.

[0021] Optionally, before the binarization process is performed in step 5, step 41 is further included;

[0022] Step 41, converting the character image block and the candidate image block into color space respectively.

[0023] Optionally, the specific method of step 41 is: perform color space conversion on the character image block and the candidate image block respectively, convert them into CMYK color space, take K channel for black fonts, and take C, M and Y channels for color fonts.

[0024] By adopting the above technical solution, it is possible to better perform text defect detection based on the principle of the printer: black fonts are usually printed using black ink, and the printed characters are more prominent on the K channel. Selecting the K channel can reduce the impact of background patterns on print defect detection. Similarly, for color fonts, the printer uses cyan (C), magenta (M), and yellow (Y) inks to synthesize color fonts. Using the C, M, and Y channels for text defect detection and analysis can not only reduce the impact of background patterns on defect detection, but also help analyze print color anomalies. For example, a red font requires magenta and yellow to be synthesized. If magenta is missing, the printed font will be yellow, and this defect can be detected in the M channel.

[0025] Optionally, the single-channel image block is binarized to obtain the character area and the background area, and the OTSU threshold segmentation algorithm or the adaptive threshold segmentation algorithm is used for binarization, or a segmentation network is trained using deep learning to segment the font. When the image background is relatively simple, the OTSU threshold segmentation algorithm can accurately segment the character area and the background area, but if the lighting is uneven, an adaptive threshold segmentation algorithm is required to obtain a better segmentation result. Furthermore, if the background is complex and the lighting is uneven, it becomes necessary to train a segmentation network using deep learning, and use a large amount of training data to train a robust segmentation model to improve the accuracy of segmentation.

[0026] A printed text defect detection device based on contour structure analysis includes an imaging device, a memory and a processor. The imaging device is used to capture a printed image to be detected and store it in the memory. The memory stores a printed text defect detection program designed using a printed text defect detection method based on contour structure analysis. The processor is communicatively connected to the memory, runs the printed text defect detection program, inputs the printed image to be detected into the printed text defect detection program, and outputs the detection result.

[0027] Optionally, a display is also included, and the processor controls the display to display the detection results.

[0028] A memory stores a printed text defect detection program designed using a printed text defect detection method based on contour structure analysis.

[0029] In summary, the present invention includes at least the following beneficial technical effects:

[0030] The present invention can provide a method and device for detecting defects in printed text based on contour structure analysis. First, a camera or a scanner is used to shoot or scan a printed target to obtain a printed image to be detected. Then, the printed image to be detected is preprocessed and aligned to an electronic printed original image. For the electronic printed original image, each character is cut out to obtain a character image block, and a candidate area of ​​the character is found in the aligned printed image to obtain a candidate image block. The character image block and the candidate image block are binary segmented respectively to obtain a binary character template image and a binary candidate image. With a template matching method, the binary character template image is used as a template to search for the best matching position of the character in the binary candidate image. A picture of the same size as the binary character image is cut out at the best matching position in the binary candidate image, which is called a matching image. The matching image and the template image are pixel-by-pixel ANDed to obtain the same image. Contour analysis is performed on the matching image, the binary character image and the same image respectively to obtain their respective contour trees. Finally, defect detection is achieved through contour tree matching, thereby solving the problem of missed detection in the detection of defects in printed text in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of a method for detecting printed text defects based on contour structure analysis of the present invention;

[0032] Figure 2 is a schematic diagram of the printed text defect detection effect in a specific embodiment;

[0033] Figure 3 is a schematic diagram of the effect of the profile tree in a specific embodiment;

[0034] Figure 4 It is a schematic diagram of the effect after contour merging in a specific embodiment.

[0035] Explanation of the accompanying figures: (a) represents a character image block; (b) represents a candidate image block; (c) represents a matching image; (d) represents a difference map; (e) represents a same image. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0037] The embodiment of the invention discloses a method and device for detecting printed text defects based on contour structure analysis.

[0038] Reference Figure 1-Figure 4 , a printed text defect detection method based on contour structure analysis, comprising the following steps:

[0039] Step 1, obtaining a printed image to be detected;

[0040] Step 2: preprocessing the printed image to be inspected, including target segmentation and alignment, and aligning to the electronic printed original image;

[0041] Step 3, segmenting all individual characters from the electronic printed original image to obtain character image blocks;

[0042] Step 4, according to the position of each character image block in the electronic print original image, determine the position in the aligned print image, and appropriately enlarge it as a candidate area, and crop the candidate area to obtain a candidate image block;

[0043] Step 5, binarizing the character image block and the candidate image block respectively;

[0044] Step 6, using a template matching method, taking the binary character image as a template, searching in the binary candidate images to find the best matching position of the template;

[0045] Step 7, perform pixel-by-pixel AND operation on the matching image and the binary character image to obtain the same image. In the binary image here, the value of each pixel can only be 0 or 1. The AND operation of two pixels is defined as a logical AND operation: when both pixels are 1, the result of the pixel AND is 1, and in other cases it is 0. The pixel-by-pixel AND operation means that for two binary images of the same size, the AND operation is performed pixel by pixel;

[0046] Step 8, perform contour analysis on the matching image, binary character image and same image respectively, find all outer contours and inner contours, and establish a contour tree structure: when one contour completely contains another contour, the latter is called the child node of the former;

[0047] Step 9: Match the contour tree of the binary character image with the contour tree of the same image. If they cannot match, it is considered that a missing print defect occurs.

[0048] First, use a camera or scanner to shoot or scan the printed target to obtain the printed image to be detected; then pre-process the printed image to be detected and align it to the electronic printed original image; the electronic printed original image refers to the electronic image sent to the printing device. For the electronic printed original image, cut out each character to obtain a character image block, and find the candidate area of ​​the character in the aligned printed image to obtain the candidate image block. Perform binary segmentation on the character image block and the candidate image block respectively to obtain a binary character template image and a binary candidate image. Using the template matching method, use the binary character template image as a template to search for the best matching position of the character in the binary candidate image. Cut out a picture of the same size as the binary character image at the best matching position in the binary candidate image, which is called a matching image. Perform pixel-by-pixel AND operations on the matching and template images to obtain the same image. Perform contour analysis on the matching image, the binary character image and the same image respectively to obtain their respective contour trees. Finally, defect detection is achieved through contour tree matching to solve the problem of missed detection of printing defects in small areas in the prior art.

[0049] In step 9, the contour tree matching method is:

[0050] If the two trees have unequal levels or unequal numbers of nodes, the two trees are considered mismatched;

[0051] If the number of layers of both trees is 1 and the number of nodes in the layer is equal, the two trees are considered to match;

[0052] If the number of layers of two trees is greater than or equal to 2, when the number of subtrees of their root nodes is equal and there is a set of corresponding relationships, their subtrees are paired separately, and the subtrees in each pairing are matched, then the two trees are judged to match.

[0053] By adopting the above technical solution, when a small area of ​​printing missing causes a stroke break, the contour tree will undergo a large change. Capturing this change helps to improve the detection accuracy of small area printing missing.

[0054] Before the binarization process is performed in step 5, step 41 is also included;

[0055] Step 41, converting the character image block and the candidate image block into color space respectively.

[0056] The specific method of step 41 is: perform color space conversion on the character image block and the candidate image block respectively, convert them into CMYK color space, take K channel for black fonts, and take C, M and Y channels for color fonts.

[0057] By adopting the above technical solution, it is possible to better perform text defect detection based on the principle of the printer: black fonts are usually printed using black ink, and the printed characters are more prominent on the K channel. Selecting the K channel can reduce the impact of background patterns on print defect detection. Similarly, for color fonts, the printer uses cyan (C), magenta (M), and yellow (Y) inks to synthesize color fonts. Using the C, M, and Y channels for text defect detection and analysis can not only reduce the impact of background patterns on defect detection, but also help analyze print color anomalies. For example, a red font requires magenta and yellow to be synthesized. If magenta is missing, the printed font will be yellow, and this defect can be detected in the M channel.

[0058] Binarize the single-channel image block to obtain the character area and the background area, use the OTSU threshold segmentation algorithm or the adaptive threshold segmentation algorithm for binarization, or use deep learning to train a segmentation network to segment the font. When the image background is simple, the OTSU threshold segmentation algorithm can accurately segment the character area and the background area, but if the lighting is uneven, then an adaptive threshold segmentation algorithm is needed to obtain a better segmentation result. Furthermore, if the background is complex and the lighting is uneven, it becomes necessary to use deep learning to train a segmentation network, and use a large amount of training data to train a robust segmentation model to improve the accuracy of segmentation.

[0059] A printed text defect detection device based on contour structure analysis includes an imaging device, a memory and a processor. The imaging device is used to capture a printed image to be detected and store it in the memory. The memory stores a printed text defect detection program designed using a printed text defect detection method based on contour structure analysis. The processor is connected to the memory for communication, runs the printed text defect detection program, inputs the printed image to be detected into the printed text defect detection program, and outputs the detection result.

[0060] The device also includes a display, and the processor controls the display to display the detection result.

[0061] A memory stores a printed text defect detection program designed using a printed text defect detection method based on contour structure analysis.

[0062] The following specific embodiments are used to illustrate the implementation principle of the present invention:

[0063] 1. Use a scanner to scan the card and cut out the card area.

[0064] 2. Scale the card area to the same resolution as the electronically printed original.

[0065] 3. For the electronically printed original image, determine the position of each text line according to the certificate printing standard; for each text line, segment out a single character image block.

[0066] 4. For each character, determine its position in the aligned printed image according to its position in the electronic printed original image, and appropriately enlarge it as a candidate area, and crop the area to obtain a candidate image block.

[0067] 5. Perform color space conversion on the character image block and the candidate image block respectively, and convert them into CMYK color space. For black fonts, take its K channel, and for color fonts, take C, M and Y channels.

[0068] 6. Binarize the single-channel image blocks. You can use the OTSU threshold segmentation algorithm or the adaptive threshold segmentation algorithm, or you can use deep learning to train a segmentation network to segment the fonts.

[0069] 7. Using the template matching method, take the binary character image as the template, search in the binary candidate image, and find the best matching position of the character. Since the template matching is performed on the binary image, the matching degree can be calculated by pixel and. A matching image of the same size as the binary character image is cut out at the best matching position in the binary candidate image.

[0070] 8. Perform pixel-by-pixel AND operations on the matching image and the binary character image to obtain the same image.

[0071] 9. Perform contour analysis on the matching image, binary character image and same image respectively, find all outer contours and inner contours, and establish a contour tree structure: when one contour completely contains another contour, the latter is called the child node of the former.

[0072] 10. Match the contour tree of the binary character image with the contour tree of the same image. If they cannot match, it is considered that a missing print defect has occurred. Figure 2 In the first row, the binary character image has only one outer contour and one inner contour, and the contour tree has two layers, each with only one node; the same image also has only one outer contour and one inner contour, and the contour tree has two layers, each with only one node. The two are completely matched, so it is considered that there is no missing printing defect, as shown in the contour tree. Figure 3 As shown. Figure 2 In the second row, the binary character image has 4 contours, the contour tree has 3 layers, and the number of nodes in each layer is 1, 2, and 1 respectively. The same image has only 3 contours, the contour tree has 2 layers, and the number of nodes in each layer is 2 and 1 respectively. The missed printing area causes the strokes to be broken, and the two contours are merged into one, see Figure 4As shown in the figure. If we only look at the difference map, the area of ​​the difference region in the first row is obviously larger than that in the second row, and we cannot distinguish the two cases. Using contour tree matching, we can easily separate the two cases.

[0073] 11. Match the contour trees of the matching image and the same image. If they cannot match, it is considered that there is a defect in the printing area.

[0074] 12. The contour tree matching method is:

[0075] (1) If the number of layers or the number of nodes of two trees are not equal, the two trees are considered to be mismatched. Figure 1 As shown, the contour tree of the binary character image does not match the contour tree of the same image;

[0076] (2) If the number of layers of both trees is 1 and the number of nodes in the layer is equal, the two trees are considered to match;

[0077] (3) If the number of layers of two trees is greater than or equal to 2, when the number of subtrees of their root nodes is equal and there is a set of corresponding relationships, their subtrees are paired separately, and the subtrees in each pairing are matched, then the two trees are considered to match.

[0078] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A printed text defect detection method based on contour structure analysis, It is characterized in that The following steps are involved: Step 1, obtaining a printed image to be detected; Step 2: preprocessing the printed image to be inspected, including target segmentation and alignment, and aligning to the electronic printed original image; Step 3, segmenting all individual characters from the electronic printed original image to obtain character image blocks; Step 4, according to the position of each character image block in the electronic print original image, determine the position in the aligned print image, and enlarge it as a candidate area, and crop the candidate area to obtain a candidate image block; Step 5, binarizing the character image block and the candidate image block respectively; Step 6, using a template matching method, taking the binary character image as a template, searching in the binary candidate images to find the best matching position of the template; Step 7, performing pixel-by-pixel AND operation on the matching image and the binary character image to obtain the same image; Step 8, perform contour analysis on the matching image, binary character image and same image respectively, find all outer contours and inner contours, and establish a contour tree structure: when one contour completely contains another contour, the latter is called the child node of the former; Step 9: Match the contour tree of the binary character image with the contour tree of the same image. If they cannot match, it is considered that a missing print defect occurs.

2. A printed text defect detection method based on contour structure analysis according to claim 1, It is characterized in that In step 9, the contour tree matching method is: If the two trees have unequal levels or unequal numbers of nodes, the two trees are considered mismatched; If the number of layers of both trees is 1 and the number of nodes in the layer is equal, the two trees are considered to match; If the number of layers of two trees is greater than or equal to 2, when the number of subtrees of their root nodes is equal and there is a set of corresponding relationships, their subtrees are paired separately, and the subtrees in each pairing are matched, then the two trees are judged to match.

3. The method for detecting printed text defects based on contour structure analysis according to claim 1, It is characterized in that Before the binarization process is performed in step 5, step 41 is also included; Step 41, converting the character image block and the candidate image block into color space respectively.

4. The method for detecting printed text defects based on contour structure analysis according to claim 3, It is characterized in that The specific method of step 41 is: perform color space conversion on the character image block and the candidate image block respectively, convert them into CMYK color space, take K channel for black fonts, and take C, M and Y channels for color fonts.

5. The method for detecting printed text defects based on contour structure analysis according to claim 4, It is characterized in that The single-channel image block is binarized to obtain the character area and the background area, and the OTSU threshold segmentation algorithm or the adaptive threshold segmentation algorithm is used for binarization, or a segmentation network is trained using deep learning to segment the character area.

6. A printed text defect detection device based on contour structure analysis, Features: It includes an imaging device, a memory and a processor, wherein the imaging device is used to capture a printed image to be detected and store it in the memory, wherein the memory stores a printed text defect detection program designed using a printed text defect detection method based on contour structure analysis as described in any one of claims 1 to 5, the processor is communicatively connected to the memory, runs the printed text defect detection program, inputs the printed image to be detected into the printed text defect detection program, and outputs a detection result.

7. A printed text defect detection device based on contour structure analysis according to claim 6, Features: The system also includes a display, and the processor controls the display to display the detection result.

8. A memory, It is characterized in that A printed text defect detection program designed using a printed text defect detection method based on contour structure analysis as described in any one of claims 1 to 5 is stored.

Images