A method and device for detecting missing printed text on a card without a template image

Abstract

The present invention discloses a method and device for detecting missing printed text on a card without a template image, and relates to the technical field of detecting missing printed text on a card. First, target detection and alignment are performed on the image to be detected to obtain a normalized card image. Target detection and alignment of the card reduce the influence of affine transformation in the imaging process on the detection result. Card text information extraction and card font type detection are performed on the normalized image to generate a binary template text line image, which is convenient for matching with the text line of the normalized card image to calculate a difference map. The difference map can be processed to obtain the position of the missing text, thereby realizing the detection of missing printed text on the card without a template image. Automatic detection of missing printed text in the card is realized, with high detection efficiency and high detection accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of card and certificate printed text missing detection, and in particular to a card and certificate printed text missing detection method and device without a template image. Background Art

[0002] During the card production process, due to a print head failure, the printed text may be missing in a certain area; at the same time, during the card lamination process, the lamination may be damaged. When the damaged position is above the printed text, it may also cause some strokes or areas of the printed text to be missing.

[0003] Both of the above situations will result in unqualified cards. The traditional method to solve this problem often requires manual inspection, which is inefficient and prone to errors.

[0004] Therefore, it is necessary to develop an automated method and device to detect missing printed text in a card. Summary of the invention

[0005] In order to solve the technical problem of automatic detection of missing printed text in cards, the present invention provides a method and device for detecting missing printed text in cards without template images. The following technical solutions are adopted:

[0006] A method for detecting missing printed text on a card without a template image comprises the following steps:

[0007] Step 1: perform card target detection and alignment on the image to be detected to obtain a normalized card image;

[0008] Step 2, extracting the card text information, which includes text location information and text content information;

[0009] Step 3, obtaining the font type of the card text content information;

[0010] Step 4, generating a binary template text line image for each text line according to the card text information and the card font type;

[0011] Step 5, calculate the difference map between the binary template text line image and the normalized card image, and obtain the position of the incomplete text according to the difference map.

[0012] By adopting the above technical solution, the image to be detected is firstly subjected to target detection and alignment to obtain a normalized card image. Then the card text information and font type are extracted to generate a template text line image. Finally, the difference map between the template text line image and the normalized card image is calculated and the printed text missing detection area is obtained based on the difference map.

[0013] It can realize automatic detection of missing printed text in cards, with high detection efficiency and accuracy.

[0014] The main purpose of card target detection and alignment is to reduce the impact of affine transformation on the detection results during the imaging process. The purpose of card text information extraction and card font type detection on the normalized image is to generate a binary template text line image, which is easy to match with the text line of the normalized card image to calculate the difference map. The difference map can be processed to obtain the position of the missing text, thereby realizing the detection of missing text printed on the card without a template image.

[0015] Optionally, the method of step 1 is: use YOLO or SSD to train the card target detection model on the training set, then perform target detection on the test image to obtain the target position, and expand and crop the target image, use CornerNet or CenterNet to locate the four corner points of the card in the target image, use the perspective transformation algorithm to estimate the transformation matrix according to the corner point position, and use the transformation matrix to perform perspective transformation on the target image to obtain a normalized card image.

[0016] The perspective transformation algorithm can be getPerspectiveTransform and warpPerspective in the OpenCV environment.

[0017] Optionally, the method of step 2 is: for a card containing a chip, a card reader can be used to read the text information on the card surface and obtain the corresponding domains, and then a target detection algorithm is used to detect each domain on the normalized card image;

[0018] For cards without chips, CTPN or EAST is used to identify all text lines on the normalized card image, and the OCR algorithm is used to obtain the text information to obtain all text line positions and text information pairs.

[0019] The object detection algorithm can be YOLO; the OCR algorithm can be Tesseract OCR or CRNN.

[0020] In step 3, you can use prior knowledge to obtain card font information, or you can use font recognition algorithms to obtain font types. For some common cards, such as second-generation cards and driver's licenses, a fixed font is usually used. In addition, you can also train a classifier to identify the font type of the text line. Furthermore, you can also use multi-task learning methods to train a model to perform text detection and text type recognition at the same time.

[0021] Optionally, in step 4, the binary template text line image and the normalized card image text line have the same resolution.

[0022] Optionally, in step 4, the position of each word in the template text line is recorded.

[0023] By adopting the above technical solution, the card text information and card font type are obtained, and a template image can be automatically generated without a template image, and the printing missing of a single character can be automatically judged. If the card to be inspected has chip information that can be used, the chip information can be used to further automatically judge the printing missing of the entire character.

[0024] Optionally, step 5 includes the following sub-steps:

[0025] Step 51, extracting a grayscale image from the normalized card image;

[0026] Step 52, binarizing the grayscale image to obtain a binary card image;

[0027] Step 53, cutting the binary template text line image into single words to obtain single word binary image blocks;

[0028] Step 54, using the single-word binary image block as a template, searching at the position of the text line corresponding to the binary card image using a template matching method, finding the best match, performing pixel XOR operation on the single-word binary image block and the best match, and obtaining a difference map of the word;

[0029] Step 55, performing a morphological opening operation on the difference map. The morphological opening operation first performs an erosion operation on the binary image and then performs a dilation operation, which can eliminate small differences.

[0030] Step 56, extracting connected domains from the difference graph after the morphological opening operation, and determining whether there is a connected domain with an area greater than a threshold value T. If so, determining that the word is missing, and the missing position is the connected domain with a connected domain area greater than the threshold value T.

[0031] By adopting the above technical solution, misjudgment caused by errors in the normalization process can be reduced.

[0032] Optionally, in step 52, an adaptive threshold segmentation algorithm or a deep learning segmentation algorithm is adopted.

[0033] A card printed text missing detection device without template image comprises an imaging device, a memory and a processor. The imaging device is used to image a manufactured card to obtain an image to be detected, and to exchange the image to be detected with the memory. The memory stores a card printed text missing detection program designed by using a card printed text missing detection method without template image. The processor is communicatively connected with the memory to run the card printed text missing detection program to output a detection result.

[0034] In summary, the present invention includes at least the following beneficial technical effects:

[0035] The present invention can provide a method and device for detecting missing printed text on a card without a template image. First, target detection and alignment are performed on the image to be detected to obtain a normalized card image. Then, the card text information and font type are extracted to generate a template text line image. Finally, a difference map between the template text line image and the normalized card image is calculated and a printed text missing detection area is obtained according to the difference map. Automatic detection of missing printed text in the card is achieved with high detection efficiency and high detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of a method for detecting missing printed text on a card without a template image according to the present invention;

[0037] Figure 2 It is a schematic flow chart of the sub-steps of step 5 of a method for detecting missing printed text on a card without a template image according to the present invention. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below with reference to the accompanying drawings. The embodiment of the present invention discloses a method and device for detecting missing printed text on a card without a template image.

[0039] Reference Figure 1 and Figure 2 , a card printed text missing detection method without template image, comprising the following steps:

[0040] Step 1: perform card target detection and alignment on the image to be detected to obtain a normalized card image;

[0041] Step 2, extracting the card text information, which includes text location information and text content information;

[0042] Step 3, obtaining the font type of the card text content information;

[0043] Step 4, generating a binary template text line image for each text line according to the card text information and the card font type;

[0044] Step 5, calculate the difference map between the binary template text line image and the normalized card image, and obtain the position of the incomplete text according to the difference map.

[0045] First, the image to be detected is subjected to target detection and alignment to obtain a normalized card image. Then, the card text information and font type are extracted to generate a template text line image. Finally, the difference map between the template text line image and the normalized card image is calculated and the printed text missing detection area is obtained based on the difference map.

[0046] It can realize automatic detection of missing printed text in cards, with high detection efficiency and accuracy.

[0047] The main purpose of card target detection and alignment is to reduce the impact of affine transformation on the detection results during the imaging process. The purpose of card text information extraction and card font type detection on the normalized image is to generate a binary template text line image, which is easy to match with the text line of the normalized card image to calculate the difference map. The difference map can be processed to obtain the position of the missing text, thereby realizing the detection of missing text printed on the card without a template image.

[0048] The method of step 1 is: use YOLO or SSD to train the card target detection model on the training set, then perform target detection on the test image to obtain the target position, and expand and crop the target image, use CornerNet or CenterNet to locate the four corner points of the card in the target image, use the perspective transformation algorithm to estimate the transformation matrix according to the corner point position, and use the transformation matrix to perform perspective transformation on the target image to obtain the normalized card image.

[0049] The perspective transformation algorithm can be getPerspectiveTransform and warpPerspective in the OpenCV environment.

[0050] The method of step 2 is: for a card containing a chip, the card reader can be used to read the text information on the card surface and obtain the corresponding domain, and then the target detection algorithm is used to detect each domain on the normalized card image;

[0051] For cards without chips, CTPN or EAST is used to identify all text lines on the normalized card image, and the OCR algorithm is used to obtain the text information to obtain all text line positions and text information pairs.

[0052] The object detection algorithm can be YOLO; the OCR algorithm can be Tesseract OCR or CRNN.

[0053] In step 3, you can use prior knowledge to obtain card font information, or you can use font recognition algorithms to obtain font types. For some common cards, such as second-generation cards and driver's licenses, a fixed font is usually used. In addition, you can also train a classifier to identify the font type of the text line. Furthermore, you can also use multi-task learning methods to train a model to perform text detection and text type recognition at the same time.

[0054] In step 4, the binary template text line image and the normalized card image text line have the same resolution.

[0055] In step 4, the position of each word in the template text line is recorded.

[0056] By obtaining the card text information and card font type, the template image can be automatically generated without the template image, and the printing missing of a single character can be automatically judged. If the card to be inspected has chip information available, the chip information can be used to further automatically judge the printing missing of the entire character.

[0057] Step 5 includes the following sub-steps:

[0058] Step 51, extracting a grayscale image from the normalized card image;

[0059] Step 52, binarizing the grayscale image to obtain a binary card image;

[0060] Step 53, cutting the binary template text line image into single words to obtain single word binary image blocks;

[0061] Step 54, using the single-word binary image block as a template, searching at the position of the text line corresponding to the binary card image using a template matching method, finding the best match, performing pixel XOR operation on the single-word binary image block and the best match, and obtaining a difference map of the word;

[0062] Step 55, performing a morphological opening operation on the difference map. The morphological opening operation first performs an erosion operation on the binary image and then performs a dilation operation, which can eliminate small differences.

[0063] Step 56, extracting connected domains from the difference graph after the morphological opening operation, and determining whether there is a connected domain with an area greater than a threshold value T. If so, determining that the word is missing, and the missing position is the connected domain with a connected domain area greater than the threshold value T.

[0064] Processing the connected domain can reduce some false detections caused by scale or angle errors.

[0065] In step 52, an adaptive threshold segmentation algorithm or a deep learning segmentation algorithm is used.

[0066] A card printed text missing detection device without template image comprises an imaging device, a memory and a processor. The imaging device is used to image a manufactured card to obtain an image to be detected, and to exchange the image to be detected with the memory. The memory stores a card printed text missing detection program designed by using a card printed text missing detection method without template image. The processor is communicatively connected with the memory to run the card printed text missing detection program to output a detection result.

[0067] The following uses a specific implementation case to illustrate the implementation principle of a card printing and membership ... making and membership card printing and membership card making and membership card

[0068] 1. Perform card target detection and alignment on the image to be detected to obtain a normalized card image; use the YOLO algorithm to detect the target on the test image to obtain the target position, and expand and crop the target image. Use the CenterNet algorithm to locate the four corner points of the card in the target image. According to the position of the corner points, use the perspective transformation algorithm to estimate the transformation matrix, and use the transformation matrix to perform perspective transformation on the target image to obtain a normalized card image. The perspective transformation algorithm is getPerspectiveTransform and warpPerspective in the OpenCV environment.

[0069] 2. Extract the card text information, which includes text position information and text content information; use the EAST algorithm to detect all text lines on the normalized card image, and use the OCR algorithm to obtain text information to obtain all text line position and text information pairs.

[0070] 3. Get the font type of the card text content information; using prior knowledge, for some common cards, such as the second-generation card, driver's license, etc., the font is fixed, and the font type of the card can be directly obtained;

[0071] 4. Generate a binary template text line image for each text line according to the card text information and card font type;

[0072] 5. Calculate the difference map between the binary template text line image and the normalized card image, and obtain the position of the incomplete text based on the difference map. The specific method is as follows:

[0073] Extract the grayscale image from the normalized card image, formula G = 0.299 * R + 0.587 * G +0.114 * B;

[0074] G represents the pixel value of the grayscale image; R represents the pixel value of the red component of the color image; G represents the pixel value of the green component of the color image; B represents the pixel value of the blue component of the color image.

[0075] The grayscale image is binarized to obtain a binary card image. The binarization algorithm uses the Sauvola algorithm. The input of the Sauvola algorithm is a grayscale image. It takes the current pixel as the center and dynamically calculates the threshold of the pixel based on the grayscale mean and standard deviation in the neighborhood of the current pixel. The calculation of the Sauvola algorithm is as follows:

[0076] For the pixel with coordinates (x, y), calculate the gray value m(x, y) and standard deviation s(x, y) in the r*r neighborhood, and then find its threshold t(x, y)

[0077] ;

[0078] Cut the binary template text line image into single words to obtain single word binary image blocks;

[0079] Using the single-word binary image block as a template, the template matching method is used to search the corresponding text line position of the binary card image to find the best match, and the single-word binary image block and the best match are subjected to pixel XOR operation to obtain the difference map of the word;

[0080] Perform morphological opening operation on the difference map. The morphological opening operation first performs corrosion operation on the binary image and then performs dilation operation, which can eliminate small differences.

[0081] Connected domain extraction is performed on the difference graph after morphological opening operation to determine whether there is a connected domain with an area greater than the threshold T=30. If so, it is determined that the word is missing, and the missing position is the connected domain with a connected domain area greater than the threshold T.

[0082] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting missing printed text on a card without a template image, characterized in that: The following steps are involved: Step 1: perform card target detection and alignment on the image to be detected to obtain a normalized card image; Step 2, extracting the card text information, which includes text location information and text content information; Step 3, obtaining the font type of the card text content information; Step 4, generating a binary template text line image for each text line according to the card text information and the card font type; Step 5, calculating the difference map between the binary template text line image and the normalized card image, and obtaining the position of the incomplete text according to the difference map; The method of step 2 is: the card contains a chip, the card surface text information is read by a card reader, and the corresponding domain is obtained, and then the target detection algorithm is used to detect each domain on the normalized card image; Step 5 includes the following sub-steps: Step 51, extracting a grayscale image from the normalized card image; Step 52, binarizing the grayscale image to obtain a binary card image; Step 53, cutting the binary template text line image into single words to obtain single word binary image blocks; Step 54, using the single-word binary image block as a template, searching at the position of the text line corresponding to the binary card image using a template matching method, finding the best match, performing pixel XOR operation on the single-word binary image block and the best match, and obtaining a difference map of the word; Step 55, performing a morphological opening operation on the difference map, wherein the morphological opening operation first performs an erosion operation on the binary image and then performs an expansion operation to eliminate small differences; Step 56, extracting connected domains from the difference graph after the morphological opening operation, and determining whether there is a connected domain with an area greater than a threshold value T. If so, it is determined that the word is missing, and the missing position is a connected domain with an area greater than the threshold value T.

2. The method for detecting missing printed text on a card without a template image according to claim 1, characterized in that: The method of step 1 is: use YOLO or SSD to train the card target detection model on the training set, then perform target detection on the test image to obtain the target position, and expand and crop the target image, use CornerNet or CenterNet to locate the four corner points of the card in the target image, use the perspective transformation algorithm to estimate the transformation matrix according to the corner point position, and use the transformation matrix to perform perspective transformation on the target image to obtain the normalized card image.

3. The method for detecting missing printed text on a card without a template image according to claim 1, characterized in that: In step 4, the binary template text line image and the normalized card image text line have the same resolution.

4. The method for detecting missing printed text on a card without a template image according to claim 1, characterized in that: In step 4, the position of each word in the template text line is recorded.

5. The method for detecting missing printed text on a card without a template image according to claim 1, characterized in that: In step 52, an adaptive threshold segmentation algorithm or a deep learning segmentation algorithm is used.

6. A card printed text missing detection device without template image, characterized in that: The invention comprises an imaging device, a memory and a processor, wherein the imaging device is used to image a manufactured card to obtain an image to be detected and to exchange the image to be detected with the memory, the memory stores a card printed text missing detection program designed by using a card printed text missing detection method without a template image as described in any one of claims 1 to 5, and the processor is communicatively connected with the memory to run the card printed text missing detection program to output a detection result.

Images