A deep learning-based method and system for detecting printing defects of certificates in complex backgrounds

Abstract

The application discloses a complex background certificate printing defect detection method and system based on deep learning, and the method comprises the following steps: obtaining a target certificate image to be detected; detecting the target certificate image based on a trained target detection model, determining a target region image and edge position information of the target certificate image; determining the corner point position of the target region image based on a straight line detection algorithm; determining the straight line information of the text content of the target region image; matching the edge position information, the corner point position and the straight line information to obtain a matching result; and judging whether the target certificate image has a printing defect based on the matching result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of certificate image printing, and more particularly to a complex background certificate printing defect detection method and system based on deep learning. BACKGROUND

[0002] Various certificates and tickets such as ID cards, driver's licenses, Hong Kong and Macao passes, train tickets, and air tickets are gradually achieving full automation in printing and making. Through human-computer interaction between users and a certificate making system, automatic printing and making of user demand certificates or tickets are achieved. In the printing and making process of various certificates, the printing starting point and the printing direction of the certificate must be determined first to ensure accurate printing. In addition, after the printing of the certificate is completed, the certificate making system needs to verify again whether the printed certificate has defects to ensure the quality of the certificate output to the user.

[0003] In the prior art, various specific certificate image target detection algorithms are usually used to determine the certificate area, and then a text recognition algorithm is used to determine the position of the certificate keyword to achieve correct printing positioning of the certificate. Application (CN202011116509.4) attempts to use only a target detection algorithm to complete the area positioning of the certificate and the classification of the certificate type to detect whether the certificate has defects. Applications (CN202110238213.8) and (CN202011458177.8) are systems designed for the detection of ID cards and bank cards, respectively, and lack universality and generalization ability, and lose the ability to quickly detect when facing other certificates. In addition, the prior art assumes that the target detection accuracy is extremely high, but in the actual recognition process, the target detection model is difficult to completely match the certificate edge at 100%, and most cases are when the detection frame contains the target. Therefore, under the assumption that it is difficult to establish, the detection accuracy of the system for defective certificates cannot be guaranteed.

[0004] Therefore, a technology is needed to detect complex background certificate printing defects based on deep learning. SUMMARY

[0005] The technical solution of the present application provides a complex background certificate printing defect detection method and system based on deep learning to solve the problem of how to detect complex background certificate printing defects based on deep learning.

[0006] To solve the above problems, the present application provides a complex background certificate printing defect detection method based on deep learning, which comprises:

[0007] Obtaining a target certificate image to be detected;

[0008] Based on the trained target detection model, the target certificate image is detected to determine the target area image and edge position information of the target certificate image.

[0009] determine the corner point position of the target region image based on a straight line detection algorithm;

[0010] determine the straight line information of the text content of the target region image;

[0011] match the edge position information, the corner point position and the straight line information to obtain a matching result; based on the matching result, determine whether the target certificate image has a printing defect.

[0012] Preferably, further comprising:

[0013] establish a certificate image training set comprising a plurality of certificate images, and label the target position of the plurality of certificate images in the certificate image training set;

[0014] train the target detection model through the labeled certificate image training set.

[0015] Preferably, the corner point position of the target region is determined based on a straight line detection algorithm, comprising:

[0016] perform binaryzation processing on the target region image;

[0017] detect the straight line target in the target region image after binaryzation processing through a straight line detection algorithm, and determine a plurality of horizontal direction straight line clusters and a plurality of vertical direction straight line clusters of the straight line target respectively;

[0018] based on the plurality of horizontal direction straight line clusters and the plurality of vertical direction straight line clusters, form a pair of parallel lines, and determine four intersection points of the target region image as corner points based on the pair of parallel lines.

[0019] Preferably, the straight line information of the text content of the target region image is determined, comprising:

[0020] perform binaryzation processing on the target region image;

[0021] detect the straight line target in the target region image after binaryzation processing through a straight line detection algorithm, determine the straight line cluster of the straight line target, and determine the main direction of the straight line cluster through clustering statistical analysis.

[0022] Preferably, the target certificate includes: an ID card, a driver's license, a Hong Kong and Macao pass, a train ticket, and an air ticket.

[0023] Based on another aspect of the present application, the present application provides a complex background certificate printing defect detection system based on deep learning, comprising:

[0024] An input / output module is configured to acquire a target certificate image to be detected, match the edge position information, the corner point position, and the straight line information, and acquire a matching result; and determine whether the target certificate image has a printing defect based on the matching result.

[0025] A target binary classification module is configured to detect the target certificate image based on a trained target detection model, and determine a target region image and edge position information of the target certificate image.

[0026] A corner point detection module is configured to determine a corner point position of the target region image based on a straight line detection algorithm.

[0027] A straight line detection module is configured to determine straight line information of a text content in the target region image.

[0028] Preferably, the target binary classification module is further configured to:

[0029] A certificate image training set including a plurality of certificate images is established, and target positions of the plurality of certificate images in the certificate image training set are labeled.

[0030] The target detection model is trained based on the labeled certificate image training set.

[0031] Preferably, the corner point detection module is configured to determine a corner point position of the target region based on a straight line detection algorithm, and is further configured to:

[0032] The target region image is binarized.

[0033] The straight line targets in the binarized target region image are detected by a straight line detection algorithm, and a plurality of horizontal direction straight line clusters and a plurality of vertical direction straight line clusters of the straight line targets are determined respectively.

[0034] Parallel line pairs are formed based on the plurality of horizontal direction straight line clusters and the plurality of vertical direction straight line clusters, and four intersection points of the target region image are determined as corner points based on the parallel lines.

[0035] Preferably, the straight line detection module is configured to determine straight line information of a text content in the target region image, and is further configured to:

[0036] The target region image is binarized.

[0037] The straight line targets in the binarized target region image are detected by a straight line detection algorithm, a straight line cluster of the straight line targets is determined, and a main direction of the straight line cluster is determined by clustering statistical analysis.

[0038] Preferably, the target certificate image includes: an ID card image, a driver's license image, a Hong Kong and Macao pass image, a train ticket image, and an air ticket image.

[0039] The technical scheme of the present application provides a complex background certificate printing defect detection method and system based on deep learning, wherein the method comprises: obtaining a target certificate image to be detected; detecting the target certificate image based on a trained target detection model to determine a target region image and edge position information of the target certificate image; determining corner point positions of the target region image based on a straight line detection algorithm; determining straight line information of text content in the target region image; matching the edge position information, the corner point positions, and the straight line information to obtain a matching result; and judging whether the target certificate image has a printing defect based on the matching result. The technical scheme of the present application is aimed at the problem of certificate printing defect detection in a complex background, and a complex background certificate printing defect detection method and system based on deep learning are designed. By setting a target binary classification module, a corner point detection module, and a straight line detection module, the edges of a certificate are determined, the starting point of certificate printing is located, and whether the certificate printing has a defect is detected, thereby improving the detection efficiency of certificate image printing and ensuring the quality of the certificate delivered to the end user. BRIEF DESCRIPTION OF DRAWINGS

[0040] The exemplary embodiments of the present application can be more completely understood in reference to the following drawings:

[0041] Figure 1 A complex background certificate printing defect detection method based on deep learning according to a preferred embodiment of the present application is shown in the flowchart;

[0042] Figure 2 A processing flowchart of each module of the complex background certificate printing defect detection based on deep learning according to the preferred embodiment of the present application is shown;

[0043] Figure 3 A method flowchart of the target binary classification module according to the preferred embodiment of the present application is shown;

[0044] Figure 4 A method flowchart of the corner point detection module according to the preferred embodiment of the present application is shown;

[0045] Figure 5 A method flowchart of the straight line detection module according to the preferred embodiment of the present application is shown; and

[0046] Figure 6 A complex background certificate printing defect detection system structure diagram based on deep learning according to the preferred embodiment of the present application is shown. DETAILED DESCRIPTION

[0047] Reference will now be made to the drawings to describe the exemplary embodiments of the present application in greater detail. The present application can be variously embodied and is not limited to the embodiments described herein, which are provided for the purposes of disclosure and to fully and completely disclose the present application to those skilled in the art. The terms used in the exemplary embodiments of the present application, which are described with reference to the drawings, are terms defined by the present application and are not limited to the terms used in the exemplary embodiments. The same reference numerals are used throughout the drawings and the same elements are represented by the same reference numerals.

[0048] Unless otherwise defined, the terms (including technical terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. Also, it is to be understood that the terms defined by a generally used dictionary have meanings consistent with those in the context of the relevant art, and are not to be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0049] Figure 1 A flow chart of a complex background certificate printing defect detection method according to a preferred embodiment of the present application. The present application designs a target binary classification module to solve the problem of the applicability of the prior art detection method to a single certificate. The present application only detects the target edge and background of the certificate and no longer focuses on the accurate positioning of the printing area of the certificate. Since only the edge and background of the certificate are concerned, the present application can be applied to various types of certificates and can more accurately identify the background and certificate target. The present application proposes a corner point detection module to determine the printing start position based on the corner point position information of the certificate target to further improve the adaptability of the detection of different certificate images. The prior art uses a text target detection model to identify certificate printing defects such as tilting. The solution to this problem requires a high detection accuracy of the target detection model, thus increasing the difficulty of model training and samples. To solve this problem, the present application designs a straight line detection module to detect the straight line formed by the continuous text in the certificate and identify whether the certificate has defects in combination with the start printing position of the certificate.

[0050] Various certificates and bills such as ID cards, driver's licenses, Hong Kong and Macao passes, train tickets, and airplane tickets are completed in a certificate making machine. The output certificate needs to be detected for printing defects. To ensure that the certificate printed without defects reaches the user, the present application proposes a complex background certificate printing defect detection method based on deep learning to determine the printing start position of various certificates and whether there are defects, which mainly includes four modules, (1) an input and output module, (2) a certificate target binary classification module, (3) a corner point detection module, and (4) a straight line detection module, as shown in FIG. 1. Figure 2

[0051] As shown in FIG. 2, the input and output module receives the input image and outputs the result of the certificate target binary classification module. Figure 1 ​As shown, the present application provides a complex background certificate printing defect detection method based on deep learning, which comprises the following steps:

[0052] Step 101: obtaining a target certificate image to be detected; preferably, the target certificate includes: an ID card, a driver's license, a Hong Kong and Macao pass, a train ticket, and an air ticket.

[0053] The input and output module of the present application serves as the input and output of the entire system, inputs the certificate image to be detected, and outputs the certificate or whether the certificate has defects.

[0054] Step 102: detecting the target certificate image based on the trained target detection model to determine the target region image and edge position information of the target certificate image;

[0055] Preferably, it further comprises:

[0056] A certificate image training set comprising a plurality of certificate images is established, and the target positions of the plurality of certificate images in the certificate image training set are labeled;

[0057] The target detection model is trained through the labeled certificate image training set.

[0058] The target two-classification module provided by the present application takes the image of the printed certificate as input, uses the target detection model based on deep learning, only detects the target region of the certificate, and classifies the target region of the certificate into two parts: target region and background, ignores the specific category of the certificate, and focuses on the edge information of the target and background of the certificate.

[0059] The target two-classification module is used to extract the target region and background region information of the certificate image, and focuses on the acquisition of the edge of the target of the certificate, such as Figure 3 As shown, the steps are as follows:

[0060] The input and output module serves as the input and output of the entire system, and inputs the certificate printing image into the target two-classification module;

[0061] The target region and background region in the certificate printing image are detected and acquired by the target detection model based on deep learning, and the model is obtained by training a series of certificate images, wherein the training set only labels the target position, and the category is set as object, that is, all certificates are unified into one category, and only the position information of the certificate is focused on;

[0062] The position information obtained in the above step is used to cut the target region image from the certificate printing image as the input of the subsequent module.

[0063] Step 103: determining the corner point position of the target region image based on a straight line detection algorithm;

[0064] Preferably, the corner point position of the target region is determined based on a straight line detection algorithm, comprising:

[0065] binarizing the target region image;

[0066] detecting straight line targets in the binarized target region image by a straight line detection algorithm, and determining a plurality of horizontal straight line clusters and a plurality of vertical straight line clusters of the straight line targets respectively;

[0067] forming parallel line pairs based on the plurality of horizontal straight line clusters and the plurality of vertical straight line clusters, and determining four intersection points of the target region image as corner points based on the parallel lines.

[0068] The corner point detection module of the present application takes the target region image of the certificate detected in the above steps as input, determines the corner point position of the certificate target by combining gradient information and the straight line detection algorithm, and further determines the printing start position information of the certificate.

[0069] The corner point detection module of the present application is used to accurately locate the corner point position of the certificate target, extracts the certificate edge based on the straight line detection algorithm, adds right angle filtering and parallel filtering to locate the top-left corner intersection point of the certificate as shown in Figure 4 The steps are as follows:

[0070] binarizing the target region image output by the target binary classification module;

[0071] detecting straight line targets in the binarized image by the straight line detection algorithm, and determining a series of horizontal straight line clusters and vertical straight line clusters.

[0072] determining perpendicular line pairs and parallel line pairs from the straight line clusters, and filtering to obtain the final four intersection points of the certificate target.

[0073] Step 104: determining straight line information of the text content in the target region image;

[0074] Preferably, determining the straight line information of the text content in the target region image comprises:

[0075] binarizing the target region image;

[0076] detecting straight line targets in the binarized target region image by the straight line detection algorithm, determining straight line clusters of the straight line targets, and determining the main direction of the straight line clusters by clustering statistical analysis.

[0077] The straight line detection module of the present application takes the target region image of the certificate detected in the above steps as input, detects the straight line information of the text of the certificate, and matches the straight line information with the edge and corner point position of the certificate target, thereby determining whether the certificate printing has defects.

[0078] The straight line detection module of the present application is used to determine the main direction of the text of the certificate target, so as to judge whether the certificate printing has an inclination defect problemFigure 5 The specific steps are as follows:

[0079] The binary image of the target area of the input certificate is inputted;

[0080] A straight line detection algorithm is used to detect the straight line formed by the text sequence in the binary image, obtain a straight line cluster, and cluster and count the main direction of the straight line cluster.

[0081] It is judged whether the main direction of the straight line cluster matches the edge straight line, and whether the printed text of the certificate has an inclination defect is determined.

[0082] Step 105: The edge position information, corner point position and straight line information are matched to obtain a matching result; based on the matching result, it is judged whether the target certificate image has a printing defect.

[0083] The present application is aimed at detecting printing image defects of various types of certificates, and four processing modules are designed, including a target binary classification module, a corner point detection module and a straight line detection module.

[0084] The target binary classification module designed by the present application only detects the target edge and background of the certificate, focuses on the certificate edge, and is suitable for detecting various types of certificates.

[0085] The corner point detection module designed by the present application determines the printing starting position based on the corner point position information of the certificate target, further improving the adaptability of the present patent system to different certificate images.

[0086] The straight line detection module designed by the present application detects the straight line formed by the continuous text in the certificate, and identifies whether the certificate has a defect in combination with the starting printing position of the certificate, effectively solving the problems of high precision requirement and complex model training caused by the text target detection in the prior art.

[0087] The complex background certificate printing defect detection method based on deep learning proposed by the present application effectively solves the problem of multi-class certificate printing defect detection, and has the following advantages:

[0088] The target detection module in the prior art is replaced by the target binary classification module, which focuses on the edge acquisition of the certificate target, and the corner point detection module uses the straight line detection algorithm to determine the corner point of the certificate target, no longer relying on the certificate keyword recognition, improving the adaptability of the system to various certificate printing defects, so that the system becomes a general solution.

[0089] In the printing defect detection in the certificate target area, the text target detection in the prior art is replaced by the straight line detection module in the present application, the straight line detection module generates a straight line using continuous text, matches the straight line with the corner point position and edge position information, and thus judges whether the certificate printing has a defect, removes the text detection model training, reduces the influence of different certificate texts, and improves the certificate defect detection precision.

[0090] Figure 6 A structure diagram of a complex background certificate printing defect detection system based on deep learning according to a preferred embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, the present application provides a complex background certificate printing defect detection system based on deep learning, which comprises: Figure 6

[0091] An input and output module 601 is configured to acquire a target certificate image to be detected, match edge position information, corner point position and straight line information, acquire a matching result, and determine whether the target certificate image has a printing defect based on the matching result. Preferably, the target certificate image includes an ID card image, a driver's license image, a Hong Kong and Macao pass image, a train ticket image and an air ticket image.

[0092] A target binary classification module 602 is configured to detect the target certificate image based on a trained target detection model, and determine a target region image and edge position information of the target certificate image. Preferably, the target binary classification module 602 is further configured to:

[0093] establish a certificate image training set including a plurality of certificate images, and label target positions of the plurality of certificate images in the certificate image training set;

[0094] train the target detection model based on the labeled certificate image training set.

[0095] A corner point detection module 603 is configured to determine corner point positions of the target region image based on a straight line detection algorithm. Preferably, the corner point detection module is further configured to:

[0096] perform binary processing on the target region image;

[0097] detect straight line targets in the binary-processed target region image based on the straight line detection algorithm, and determine a plurality of horizontal straight line clusters and a plurality of vertical straight line clusters of the straight line targets, respectively;

[0098] determine four intersection points of the target region image as corner points based on parallel line pairs composed of the plurality of horizontal straight line clusters and the plurality of vertical straight line clusters.

[0099] A straight line detection module 604 is configured to determine straight line information of text content in the target region image.

[0100] Preferably, the straight line detection module 604 is further configured to:

[0101] perform binary processing on the target region image;

[0102] ​The straight line target in the target region image after the binarization processing is detected through a straight line detection algorithm, a straight line cluster of the straight line target is determined, and a main direction of the straight line cluster is determined through clustering statistical analysis.

[0103] A complex background certificate printing defect detection system 600 based on deep learning according to a preferred embodiment of the present application will not be described here again in detail.

[0104] The present application has been described by reference to a few embodiments. However, as known to those skilled in the art, other embodiments are equally within the scope of the present application, in addition to those disclosed above, as defined by the appended claims.

[0105] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a / an / the [device, component, etc.] are to be interpreted openly as referring to at least one instance of the device, component, etc., unless otherwise indicated. The steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.

Claims

1. A deep learning-based complex background certificate print defect detection method, the method comprising: obtaining a target certificate image to be detected; detecting the target certificate image based on a trained target detection model to determine a target region image and edge position information of the target certificate image; determining corner point positions of the target region image based on a straight line detection algorithm, comprising: performing binaryzation processing on the target region image; detecting straight line targets in the binaryzation-processed target region image by a straight line detection algorithm to determine a plurality of horizontal direction straight line clusters and a plurality of vertical direction straight line clusters of the straight line targets respectively; determining four intersection points of the target region image as corner points based on parallel lines formed by the plurality of horizontal direction straight line clusters and the plurality of vertical direction straight line clusters based on the parallel lines; determining straight line information of text content in the target region image, comprising: performing binaryzation processing on the target region image; detecting straight line targets in the binaryzation-processed target region image by a straight line detection algorithm to determine a straight line cluster of the straight line targets, and determining a main direction of the straight line cluster by clustering statistical analysis; matching the edge position information, the corner point positions, and the straight line information to obtain a matching result; and determining whether the target certificate image has a print defect based on the matching result.

2. The method of claim 1, further comprising: establishing a certificate image training set comprising a plurality of certificate images, and labeling target positions of the plurality of certificate images in the certificate image training set; training the target detection model by the labeled certificate image training set.

3. The method of claim 1, the target document comprising: Identity cards, driver's licenses, Macao travel permits, train tickets, and airplane tickets.

4. A deep learning-based complex background certificate print defect detection system, the system comprising: an input and output module configured to obtain a target certificate image to be detected; matching edge position information, corner point positions, and straight line information to obtain a matching result; determining whether the target certificate image has a print defect based on the matching result; a target binary classification module configured to detect the target certificate image based on a trained target detection model to determine a target region image and edge position information of the target certificate image; a corner point detection module configured to determine corner point positions of the target region image based on a straight line detection algorithm, and further configured to: perform binaryzation processing on the target region image; detect straight line targets in the binaryzation-processed target region image by a straight line detection algorithm to determine a plurality of horizontal direction straight line clusters and a plurality of vertical direction straight line clusters of the straight line targets respectively; determine four intersection points of the target region image as corner points based on parallel lines formed by the plurality of horizontal direction straight line clusters and the plurality of vertical direction straight line clusters based on the parallel lines; a straight line detection module configured to determine straight line information of text content in the target region image, and further configured to: perform binaryzation processing on the target region image; The straight line target in the target region image after the binarization processing is detected through a straight line detection algorithm, a straight line cluster of the straight line target is determined, and a main direction of the straight line cluster is determined through clustering statistical analysis. 5.The system of claim 4, wherein the target binary classification module is further configured to: establish a certificate image training set comprising a plurality of certificate images, mark target positions of the plurality of certificate images in the certificate image training set, and train the target detection model based on the marked certificate image training set.

6. The system of claim 4, the target document image comprising: An ID card image, a driver's license image, a Hong Kong and Macao pass image, a train ticket image, and an air ticket image.

Images