ID card recognition method based on compliance detection

Abstract

The present invention relates to an ID card recognition method based on compliance detection, which comprises: inputting an ID card image to be identified into a trained ID card type detection model, judging whether the type of the ID card image is compliant by the model; performing effective area detection on the ID card image of compliant type, locating the effective text area and portrait area on the ID card image; calculating the clarity score of each effective text area on the ID card image under a set resolution scale, taking the average of the clarity scores of all text areas on the ID card image as the overall clarity score of the image; segmenting the individual characters in the effective text area, calculating the ratio of the spacing between adjacent characters and the length of the two adjacent characters, and based on standard template matching, determining that there is a possibility of tampering for the ratio exceeding an acceptable range; inputting the effective portrait area into a trained portrait tampering detection model, and completing the tampering detection of the portrait area by the model. The present invention is applicable to the field of image recognition technology.

Description

Technical Field

[0001] The present invention relates to an identity card recognition method based on compliance detection, which is applicable to the technical field of image recognition. Background Art

[0002] The ID card is the most important proof of personal identity. Whether it is buying a ticket, applying for a new mobile phone number and bank card, bank credit, or all kinds of other life scenarios and personal business handling, the ID card is basically indispensable. Traditional ID card recognition uses the second-generation ID card reading and verification equipment to read the RFID chip inside the ID card and extract key personal information such as name, nationality, birthday, address, ID number, etc. This method has great limitations on usage scenarios.

[0003] With the development and popularization of mobile Internet, many industries have opened up online business scenarios. The traditional method of relying on ID card readers to read information is no longer applicable. Currently, many ID card information recognition and verification functions are based on online real-time completion. Relying on ID card OCR recognition can reduce the workload of manual input. There are also a large number of business scenarios that require taking photos of personal ID cards for data review and information retention, such as personal business on government service websites, bank credit, insurance business, etc.

[0004] At the same time, ID card information authentication and recognition also face some problems. The current ID card recognition OCR technology solutions generally focus on the accuracy of text content recognition, while ignoring the quality of the photo itself, the credibility of the content, and whether it complies with the specifications. Although it has been prompted to upload a clear and correct ID card when submitting the ID card for business processing, blurred images will still appear due to shooting equipment and personal reasons, or malicious upload of non-ID card photos, increasing the workload of manual review. After the RFID chip information cannot be extracted through the ID card reader, it is impossible to guarantee the authenticity of the submitted ID card, whether there is tampering with text or portraits, and forged false information. With the rapid development of deep learning technology, and the increasing power of various image beautification and tampering software, there are even forged ID cards that cannot be identified at a glance by humans. Summary of the invention

[0005] The technical problem to be solved by the present invention is: in view of the above-mentioned problems, an ID card recognition method based on compliance detection is provided to ensure the security of the entire ID card recognition process to a certain extent while extracting key information of the ID card.

[0006] The technical solution adopted by the present invention is: an identity card recognition method based on compliance detection, characterized in that:

[0007] Input the ID card image to be identified into the trained ID card type detection model, and use the model to determine whether the type of the ID card image is compliant;

[0008] Perform valid area detection on ID card images of compliant types and locate valid text areas and portrait areas on the ID card images;

[0009] Calculate the clarity score of each valid text area on the ID card image at the set resolution scale, take the average of the clarity scores of all text areas on the ID card image as the overall clarity score of the image, and judge it as a blurred sample when the overall clarity score is less than the set threshold;

[0010] Separate the individual characters in the valid text area, calculate the ratio of the distance between adjacent characters and the length of the two adjacent characters, and match them based on the standard template. If the ratio exceeds the acceptable range, it is determined that there is a possibility of tampering;

[0011] The effective portrait area is input into the trained portrait tampering detection model, and the tampering detection of the portrait area is completed through the model.

[0012] The ID card type detection model is constructed based on the residual network ResNet and adding the SE module.

[0013] The effective area detection on the ID card image of the conforming type includes: using the CenterNet target detection network.

[0014] The step of calculating the clarity score of each valid text area on the ID card image at the set resolution scale and taking the average of the clarity scores of all text areas on the ID card image as the overall clarity score of the image includes:

[0015] Use the Sobel operator to extract edge features in the valid text area:

[0016]

[0017] A represents the text area image, G X and G y is the convolution result in the X and Y directions using the Sobel convolution factor;

[0018] Perform a dot product operation on the results in the two directions, sum each element of the calculated dot product matrix result, and divide it by the height h of the area to obtain the clarity score of the area;

[0019] The clarity scores of all valid text areas are summed and averaged as the overall score:

[0020]

[0021] Among them, h is the height of the text area, sum represents the sum of matrix elements, and n represents the number of valid text areas.

[0022] The step of separating out individual characters in the valid text area includes:

[0023] For each valid text area, the Otsu method is used to obtain the segmentation threshold required for the binary image, and the projection method is used to segment the individual characters.

[0024] The step of inputting the effective portrait area into a trained portrait tampering detection model and completing tampering detection of the portrait area through the model includes:

[0025] The Efficientnet-b7 network is used as the basic feature extractor. At the same time, the idea of ​​two-stream Faster-RCNN with added noise stream is adopted to capture the edge features between tampered and non-tampered portraits and the deep background texture features. The idea of ​​feature pyramid is adopted to splice edge features and RGB stream features of different scales in the forward propagation of the first few layers of the network.

[0026] The input end of the edge feature stream is based on the traditional method canny operator to extract the edge feature map T. The input of the edge feature stream is downsampled and forward propagated in the network based on the convolution with a sliding step size of 2, and the edge feature convolution layer structure is established. The nth downsampled feature map, where K represents the convolution kernel:

[0027]

[0028] The RGB stream at the input of the network is sampled based on a convolution with a sliding stride of 2 to ensure that the feature map size is reduced to half of the original size each time during the first n downsampling processes. The feature map size is consistent with the edge feature stream downsampling. The RGB convolution layer is established, and the RGB stream convolution feature is D. After each sampling, the convolution feature layers of the two branches are spliced. For the RGB stream, the first n convolution downsampling processes all contain features obtained by convolution of the edge feature stream of another branch. The feature map calculation method after the nth downsampling is as follows:

[0029]

[0030] The last fully connected layer of the network is replaced with Global Average Pooling. The forward feature map of the subsequent efficientNet-b7 network is X. The real and fake training data sets in the ID card portrait scene are constructed. Finally, the tampering detection of the portrait area is completed through the trained softamx classifier. X represents the final feature map output by the network.

[0031] An identity card recognition device based on compliance detection, characterized by comprising:

[0032] The ID card type detection module is used to input the ID card image to be identified into the trained ID card type detection model, and use the model to determine whether the type of the ID card image is compliant;

[0033] The valid area detection module is used to detect the valid area of ​​the ID card image of the conforming type and locate the valid text area and portrait area on the ID card image;

[0034] A clarity judgment module is used to calculate the clarity score of each valid text area on the ID card image under a set resolution scale, take the average of the clarity scores of all text areas on the ID card image as the overall clarity score of the image, and judge it as a blurred sample when the overall clarity score is less than a set threshold;

[0035] The text tampering detection module is used to segment the individual characters in the valid text area, calculate the ratio of the distance between adjacent characters and the length of the two adjacent characters, and based on standard template matching, identify the possibility of tampering if the ratio exceeds the acceptable range;

[0036] The portrait tampering detection module is used to input the effective portrait area into the trained portrait tampering detection model and complete the tampering detection of the portrait area through the model.

[0037] A storage medium stores a computer program that can be executed by a processor, characterized in that when the computer program is executed, the steps of the identity card recognition method based on compliance detection are implemented.

[0038] A computer device has a memory and a processor, wherein the memory stores a computer program that can be executed by the processor, and is characterized in that when the computer program is executed, the steps of the identity card recognition method based on compliance detection are implemented.

[0039] The beneficial effects of the present invention are as follows: the present invention targets blank areas in identity card images and too many background areas, which bring strong interference to the judgment of clarity. First, the effective area is obtained, the scale of the effective area image is unified to several fixed ranges, the edge features of the effective area are extracted, and after the features of multiple areas are processed, the results of all areas are averaged to determine whether they are clear enough to meet the upload requirements.

[0040] The present invention uses a deep learning method to construct training samples, build a residual network, and introduce the attention concept of computer vision as a feature extractor to classify features in a high-dimensional space to determine whether the photo is an ID card picture, whether it is the national emblem side or the portrait side, the original or the photocopy, and whether it is a screen copy. Through this step of ID card quality compliance detection, a large number of non-compliant pictures can be filtered out, reducing the ineffective use of machines and human resources.

[0041] The present invention uses a combination of traditional and deep learning methods, and performs traditional logical rule matching based on a standard ID card template. In the ID card image quality compliance detection step, an effective text area has been obtained, and adaptive threshold setting is performed to perform precise text cutting, calculate the spacing ratio of characters, and compare it with the standard template data to determine whether the text is forged or tampered with.

[0042] The present invention is based on a deep learning method and utilizes traditional edge feature extraction methods, focusing on the edge feature information of the portrait face itself and the area connected to the ID card. Based on the concept of feature pyramid, the edge information is downsampled multiple times, and the dual-stream splicing features of the RGB stream and the edge information stream are constructed at the input end and in the forward process of the network. The global average pooling layer is used to replace the fully connected layer, and the classifier is trained to determine whether the portrait has been tampered with or replaced. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of an embodiment. DETAILED DESCRIPTION

[0044] This embodiment is a method for identity card recognition based on compliance detection, which specifically includes the following steps:

[0045] S1. Input the ID card image to be identified into the trained ID card type detection model, and use the model to determine whether the type of the ID card image is compliant.

[0046] In this example, the ID card type detection model is based on the residual network ResNet, and based on this, the SE (Squeeze-and-Excitation Networks) module is added to introduce the attention concept in machine vision. The ID card images taken are divided into six categories: original portrait side, original national emblem side, photocopy portrait side, photocopy national emblem side, screen copy, and non-ID card. A training data set of six categories of images is constructed. After obtaining the probability of each category based on the trained softmax classifier, according to the actual upload requirements, ID card photos that do not meet the category requirements will not be processed subsequently.

[0047] S2. Perform valid area detection on the ID card image of the specified type, and locate the valid text area and portrait area on the ID card image.

[0048] The effective area of ​​the identity card that complies with the category is detected. Based on the idea of ​​target detection, the task is defined as two types of target detection: text and portrait. Different key information fields such as name, gender, nationality, date of birth, address, identity card number, issuing authority, and validity date are classified into 8 targets belonging to the same text category. If the address field has multiple text lines, they are merged into one target.

[0049] In this embodiment, the CenterNet target detection network is used to locate multiple valid text areas and portrait areas on the ID card image.

[0050] S3. Calculate the clarity score of each valid text area on the ID card image under the set resolution scale, take the average of the clarity scores of all text areas on the ID card image as the overall clarity score of the image, and judge it as a blurred sample when the overall clarity score is less than the set threshold.

[0051] Due to reasons such as shooting equipment and parameters, the resolution of ID card images varies greatly. It is necessary to scale the different effective areas of the detected ID card and fix each area to a suitable ratio to ensure that the effective areas of different ID card images are judged for clarity under the same set resolution scale.

[0052] For the multiple valid text regions obtained in the above steps, the Sobel operator is used to extract edge features. This operator can obtain the first-order gradient of the text region and has a certain inhibitory effect on noise.

[0053]

[0054] A represents the text area image, G X and G y It is the convolution result in the X and Y directions using the Sobel convolution factor.

[0055] Perform a dot product operation on the results in both directions, sum each element of the calculated dot product matrix result, and divide it by the height h of the area to obtain the clarity score of the area. Then sum the clarity scores of all text areas and take the average as the overall score.

[0056]

[0057] Among them, h is the height of the text area, sum represents the sum of matrix elements, and n represents the number of valid text areas on the ID card image.

[0058] According to the prior knowledge of blurred samples, an appropriate clarity judgment threshold is adjusted and set, and samples below the threshold are considered blurred samples.

[0059] S4. Segment the individual characters in the valid text area, calculate the ratio of the distance between adjacent characters and the length of the two adjacent characters, and match based on the standard template. If the ratio exceeds the acceptable range, it is determined that there is a possibility of tampering.

[0060] For each valid text area that has been detected, the Otsu method (OTSU) is used to obtain the segmentation threshold required for the binary image. For fields such as name and nationality, projection is performed in the vertical direction. For fields such as addresses that have multiple lines of text, projection is required in both the horizontal and vertical directions, and the projection method is used to segment individual characters.

[0061] For the character sets segmented from different text areas, the ratio of the spacing between adjacent characters and the length of two adjacent characters is calculated, and based on standard template matching, if the ratio is abnormal and exceeds the acceptable range, it is determined that there is a possibility of tampering.

[0062] S5. Input the effective portrait area into the trained portrait tampering detection model, and complete the tampering detection of the portrait area through the model.

[0063] The Efficientnet-b7 network is used as the basic feature extractor. At the same time, the idea of ​​two-stream Faster-RCNN with added noise stream is adopted to capture the edge features between tampered and non-tampered portraits as well as deep background texture and other features. The idea of ​​feature pyramid is adopted to splice edge features and RGB stream features of different scales in the forward propagation of the first few layers of the network.

[0064] The input end of the edge feature stream is based on the traditional method canny operator to extract the edge feature map T. The input of the edge feature stream is downsampled and forward propagated in the network based on the convolution with a sliding step size of 2. The edge feature convolution layer structure is established. The nth downsampled feature map, where K represents the convolution kernel, is extracted by the traditional method:

[0065]

[0066] The RGB stream at the input end of the network is also based on convolution sampling with a sliding step size of 2 to ensure that the feature map size is reduced to half of the original size each time during the first n downsampling processes. The feature map size is consistent with the edge feature stream downsampling, and an RGB convolution layer is established. The RGB stream convolution feature is D. After each sampling, the convolution feature layers of the two branches are spliced. That is, for the RGB stream, the first n convolution downsampling processes include the features obtained by convolution of the edge feature stream of another branch. The feature map calculation method after the nth downsampling is as follows. In this method, sampling is performed three times, and the edge feature stream information is discarded after the third sampling.

[0067]

[0068] This embodiment replaces the last fully connected layer of the network with a Global Average Pooling layer to reduce overfitting, reduce the amount of calculation, retain spatial information, and enable the classifier to use global information. The feature map of the subsequent efficienNet-b7 network forward is X, and the real and fake training data sets in the ID card portrait scene are constructed. Finally, the tampering detection of the portrait area is completed through the trained softamx classifier. X represents the final feature map output by the network.

[0069]

[0070] After the compliance detection of the ID card image, this embodiment performs accurate OCR recognition of the ID card content information, and extracts the key field information of the ID card through preprocessing, text detection, text recognition, and post-processing.

[0071] This embodiment uses OCR technology to extract information content from ID cards, reduces manual work, and at the same time, performs dual compliance testing of quality and content on ID card images. Based on a combination of traditional and deep learning methods, it deeply mines the features of ID cards, ID card text, and ID card portraits, and to a certain extent solves the problems of low efficiency of repeated review caused by low quality of ID card images, and false information caused by malicious forgery of content, and obtains ID card OCR recognition results with guaranteed quality and content.

[0072] This embodiment also provides an ID card recognition device based on compliance detection, including: an ID card type detection module, an effective area detection module, a clarity judgment module, a text tampering detection module and a portrait tampering detection module.

[0073] In this example, the ID card type detection module is used to input the ID card image to be identified into the trained ID card type detection model, and determine whether the type of the ID card image is compliant through the model; the effective area detection module is used to perform effective area detection on the ID card image of compliant type, and locate the effective text area and portrait area on the ID card image; the clarity judgment module is used to calculate the clarity score of each effective text area on the ID card image under the set resolution scale, take the average of the clarity scores of all text areas on the ID card image as the overall clarity score of the image, and judge it as a blurred sample when the overall clarity score is less than the set threshold; the text tampering detection module is used to segment the individual characters in the effective text area, calculate the ratio of the distance between adjacent characters and the length of the two adjacent characters, and based on standard template matching, determine that there is a possibility of tampering if the ratio exceeds the acceptable range; the portrait tampering detection module is used to input the effective portrait area into the trained portrait tampering detection model, and complete the tampering detection of the portrait area through the model.

[0074] This embodiment also provides a storage medium on which a computer program that can be executed by a processor is stored. When the computer program is executed, the steps of the identity card recognition method based on compliance detection in this example are implemented.

[0075] This embodiment also provides a computer device having a memory and a processor, wherein the memory stores a computer program that can be executed by the processor, and when the computer program is executed, the steps of the identity card recognition method based on compliance detection in this example are implemented.

Claims

1. An identity card recognition method based on compliance detection, characterized in that: Input the ID card image to be identified into the trained ID card type detection model, and use the model to determine whether the type of the ID card image is compliant; Perform valid area detection on ID card images of compliant types and locate valid text areas and portrait areas on the ID card images; Calculate the clarity score of each valid text area on the ID card image at the set resolution scale, take the average of the clarity scores of all text areas on the ID card image as the overall clarity score of the image, and judge it as a blurred sample when the overall clarity score is less than the set threshold; Separate the individual characters in the valid text area, calculate the ratio of the distance between adjacent characters and the length of the two adjacent characters, and match them based on the standard template. If the ratio exceeds the acceptable range, it is determined that there is a possibility of tampering; Input the effective portrait area into the trained portrait tampering detection model, and complete the tampering detection of the portrait area through the model; The step of inputting the effective portrait area into a trained portrait tampering detection model and completing tampering detection of the portrait area through the model includes: The Efficientnet-b7 network is used as the basic feature extractor. The idea of ​​two-stream Faster-RCNN with added noise stream is adopted to capture the edge features between tampered and non-tampered portraits and the deep background texture features. The idea of ​​feature pyramid is adopted to splice edge features and RGB stream features of different scales in the first few layers of forward propagation of the network. The input end of the edge feature is based on the traditional method canny operator to extract the edge feature map T, and the input of the edge feature is downsampled and forward propagated in the network based on the convolution with a sliding step size of 2, and the edge feature convolution layer structure is established. The nth downsampled feature map, where K represents the convolution kernel: The RGB stream at the input of the network is sampled based on a convolution with a sliding stride of 2 to ensure that the feature map size is reduced to half of the original size each time during the first n downsampling processes. The feature map size is consistent with the edge feature downsampling. The RGB convolution layer is established, and the RGB stream convolution feature is D. After each sampling, the convolution feature layers of the two branches are spliced. For the RGB stream, the first n convolution downsampling processes all contain features obtained by convolution of the edge features of another branch. The feature map calculation method after the nth downsampling is as follows: The last fully connected layer of the network is replaced with Global Average Pooling. The forward feature map of the subsequent efficientNet-b7 network is X. The real and fake training data sets in the ID card portrait scene are constructed. Finally, the tampering detection of the portrait area is completed through the trained softamx classifier. X represents the final feature map output by the network.

2. The ID card recognition method based on compliance detection according to claim 1 is characterized in that: The ID card type detection model is constructed based on the residual network ResNet and adding the SE module.

3. The ID card recognition method based on compliance detection according to claim 1 is characterized in that: The effective area detection on the ID card image of the conforming type includes: using the CenterNet target detection network.

4. The ID card recognition method based on compliance detection according to claim 1 is characterized in that: The step of calculating the clarity score of each valid text area on the ID card image at the set resolution scale and taking the average of the clarity scores of all text areas on the ID card image as the overall clarity score of the image includes: Use the Sobel operator to extract edge features in the valid text area: A represents the text area image, G X and G y is the convolution result in the X and Y directions using the Sobel convolution factor; Perform a dot product operation on the results in the two directions, sum each element of the calculated dot product matrix result, and divide it by the height h of the area to obtain the clarity score of the area; The clarity scores of all valid text areas are summed and averaged as the overall score: Among them, h is the height of the text area, sum represents the sum of matrix elements, and n represents the number of valid text areas.

5. The ID card recognition method based on compliance detection according to claim 1 is characterized in that: The step of separating out individual characters in the valid text area includes: For each valid text area, the Otsu method is used to obtain the segmentation threshold required for the binary image, and the projection method is used to segment the individual characters.

6. An identity card recognition device based on compliance detection, using the identity card recognition method based on compliance detection as described in any one of claims 1 to 5, characterized in that: include: The ID card type detection module is used to input the ID card image to be identified into the trained ID card type detection model, and use the model to determine whether the type of the ID card image is compliant; The valid area detection module is used to detect the valid area of ​​the ID card image of the conforming type and locate the valid text area and portrait area on the ID card image; A clarity judgment module is used to calculate the clarity score of each valid text area on the ID card image under a set resolution scale, take the average of the clarity scores of all text areas on the ID card image as the overall clarity score of the image, and judge it as a blurred sample when the overall clarity score is less than a set threshold; The text tampering detection module is used to segment the individual characters in the valid text area, calculate the ratio of the distance between adjacent characters and the length of the two adjacent characters, and based on standard template matching, identify the possibility of tampering if the ratio exceeds the acceptable range; The portrait tampering detection module is used to input the effective portrait area into the trained portrait tampering detection model and complete the tampering detection of the portrait area through the model.

7. A storage medium having stored thereon a computer program executable by a processor, characterized in that: When the computer program is executed, the steps of the identity card recognition method based on compliance detection described in any one of claims 1 to 5 are implemented.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program executable by the processor, wherein: When the computer program is executed, the steps of the identity card recognition method based on compliance detection described in any one of claims 1 to 5 are implemented.

Images