Certificate identification method, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention provides a certificate identification method, electronic equipment and a computer readable storage medium, in the certificate identification method, after the electronic equipment obtains a first image of a certificate used by a user, anti-counterfeiting identification is carried out on the certificate according to the first image, and the probability that the certificate is a legal certificate is obtained. When the probability is smaller than a first threshold value and larger than or equal to a second threshold value, the electronic device obtains a second image stored in the certificate and obtains a third image of the fingerprint of the user, and then fingerprint comparison is conducted according to the third image and the second image. And when the fingerprint comparison result is that the fingerprint of the third image is matched with the fingerprint of the second image, determining that the holder and the user of the certificate are the same person, so that the user can be considered to pass verification, the certificate verification safety can be effectively improved, and a large-amount transfer scene in a financial scene is taken as an example, so that the verification efficiency is improved. The security of certificate verification is improved, and the fund loss can be effectively reduced.

Description

Technical field

[0001] The embodiments of this specification relate to the field of Internet technology, and in particular to a document recognition method, an electronic device, and a computer-readable storage medium. [Background Technology]

[0002] In some scenarios, such as finance, user authentication is crucial, requiring both facial and ID verification. ID verification involves comparing the user's face with the ID's facial image. However, counterfeit IDs and / or facial masks are becoming increasingly realistic. Therefore, a document recognition method is needed to verify the authenticity of user IDs. [Summary of the invention]

[0003] The embodiments of this specification provide a document recognition method, an electronic device, and a computer-readable storage medium to improve the security of document authentication.

[0004] In a first aspect, an embodiment of the present specification provides a method for document recognition, comprising: obtaining a first image of a document used by a user; wherein the first image includes images of the front and / or back of the document; performing anti-counterfeiting identification on the document based on the first image, and obtaining a probability that the document is a legal document; when the probability is less than a first threshold value and greater than or equal to a second threshold value, obtaining a second image stored in the document, and obtaining a third image of the user's fingerprint; wherein the first threshold value is greater than the second threshold value, the second image includes a fingerprint image stored in the document, and the third image includes a fingerprint image collected in a contactless manner; performing fingerprint comparison based on the third image and the second image; when the result of the fingerprint comparison is that the fingerprint of the third image matches the fingerprint of the second image, it is determined that the holder and the user of the document are the same person.

[0005] In the above-mentioned certificate identification method, after the electronic device obtains the first image of the certificate used by the user, it performs anti-counterfeiting identification on the certificate based on the first image to obtain the probability that the certificate is a legal certificate. When the above-mentioned probability is less than the first threshold value and greater than or equal to the second threshold value, the electronic device obtains the second image stored in the certificate and obtains the third image of the user's fingerprint, and then performs fingerprint comparison based on the third image and the second image. When the result of the above-mentioned fingerprint comparison is that the fingerprint of the third image matches the fingerprint of the second image, it is determined that the holder of the above-mentioned certificate and the user are the same person. In this way, it can be considered that the user authentication has passed, thereby effectively improving the security of the certificate authentication. Taking the large-value transfer scenario in the financial scenario as an example, the improvement of the security of the certificate authentication can effectively reduce financial losses.

[0006] In one possible implementation, after obtaining the probability that the certificate is a legal certificate, the method further includes: when the probability is less than a first threshold and greater than or equal to a second threshold, obtaining the user information stored in the certificate, and performing optical character recognition on the first image to obtain the user information in the first image; when the user information stored in the certificate is consistent with the user information in the first image, determining that the certificate is a legal certificate.

[0007] In one possible implementation, obtaining the second image stored in the certificate includes: when the first image includes a fingerprint image, detecting the position of the fingerprint image in the first image, and based on the position, intercepting the fingerprint image from the first image, and the intercepted fingerprint image is the second image; or, obtaining the fingerprint information stored in the certificate, and obtaining the second image based on the fingerprint information.

[0008] In one possible implementation, obtaining the third image of the user's fingerprint includes: capturing a hand image of one side of the user's palm; performing finger rotation frame detection on the hand image; and performing foreground segmentation on the hand image based on the result of the finger rotation frame detection to obtain a third image including the user's fingerprint.

[0009] In one possible implementation, before performing fingerprint comparison based on the third image and the second image, the method further includes: performing finger liveness detection based on the third image; when the result of the liveness detection is that the third image is a live fingerprint image, scoring the fingerprint quality of the third image; when the fingerprint quality score of the third image is greater than or equal to a third threshold, performing an expansion transformation on the third image; aligning the second image and the third image after the expansion transformation respectively; and performing binarization ridge extraction on the fingerprint image obtained after the alignment of the second image and the third image.

[0010] In one possible implementation, performing fingerprint comparison based on the third image and the second image includes performing fingerprint comparison based on a fingerprint image obtained by performing binarization ridge extraction on the second image and the third image.

[0011] In one possible implementation, the fingerprint comparison of the fingerprint image obtained after binarization ridge extraction based on the second image and the third image includes: comparing the fingerprint images obtained after binarization ridge extraction based on the second image and the third image through at least two comparison networks respectively to obtain at least two scores; and fusing the at least two scores to obtain a fingerprint comparison score.

[0012] In one possible implementation, the fingerprint comparison result that the fingerprint of the third image matches the fingerprint of the second image includes: the fingerprint comparison score is greater than or equal to a fourth threshold.

[0013] In a second aspect, an embodiment of this specification provides an electronic device comprising: at least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the method provided in the first aspect.

[0014] In a third aspect, an embodiment of this specification provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the method provided in the first aspect.

[0015] It should be understood that the second to third aspects of the embodiments of this specification are consistent with the technical solutions of the first aspect of the embodiments of this specification, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here.

Brief Description of the Drawings

[0016] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 A flowchart of a document recognition method provided in one embodiment of this specification;

[0018] Figure 2 A schematic diagram of an anti-counterfeiting algorithm provided in one embodiment of this specification;

[0019] Figure 3 A schematic diagram of a certificate provided for one embodiment of this specification;

[0020] Figure 4 A schematic diagram of finger rotation frame detection and foreground segmentation provided in one embodiment of this specification;

[0021] Figure 5 A flowchart of a document recognition method provided in another embodiment of this specification;

[0022] Figure 6 A flowchart of a document recognition method provided in yet another embodiment of this specification;

[0023] Figure 7 A schematic diagram of finger liveness detection provided in one embodiment of this specification;

[0024] Figure 8 A schematic diagram of binary ridge extraction provided in one embodiment of this specification;

[0025] Figure 9 A schematic diagram of a fingerprint comparison network provided in one embodiment of this specification;

[0026] Figure 10 A flowchart of a document recognition method provided in yet another embodiment of this specification;

[0027] Figure 11 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this specification. [Specific implementation method]

[0028] In order to better understand the technical solutions of this specification, the embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0029] It should be clear that the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this specification.

[0030] The terms used in the embodiments of this specification are only for the purpose of describing specific embodiments and are not intended to limit this specification. The singular forms "a", "an", "the" and "the" used in the embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0031] In the existing related technologies, forged documents and / or forged face masks are becoming more and more realistic, so the security of document fitting is greatly challenged. Based on the above problems, the embodiment of this specification proposes a document identification method. When the document anti-counterfeiting identification determines that the current document has a suspicious risk, the fingerprint image stored in the document and the fingerprint image of the user collected contactlessly are compared. If the fingerprint comparison is consistent, it can be determined that the holder and user of the current document are the same person. In this way, the user authentication can be considered to have passed, thereby effectively improving the security of document authentication. Taking the large-value transfer scenario in the financial scenario as an example, the improvement of the security of document authentication can effectively reduce financial losses.

[0032] Figure 1 This is a flowchart of a document recognition method provided in one embodiment of this specification. The document recognition method provided in this embodiment can be executed by an electronic device, wherein the electronic device can be a terminal device such as a smartphone, a tablet computer or a personal computer (PC), or a server, which can be set up in the cloud.

[0033] like Figure 1 As shown, the above-mentioned document recognition method may include:

[0034] Step 102: Acquire a first image of the ID card used by the user, wherein the first image includes an image of the front and / or back of the ID card.

[0035] Specifically, in this step, if the electronic device is a terminal device, the electronic device can capture the first image of the certificate used by the user through a camera; if the electronic device is a server, the electronic device can receive the first image of the certificate used by the user sent by the terminal device, that is, the terminal device can capture the first image of the certificate used by the user through a camera, and then the terminal device sends the first image to the electronic device, and the electronic device obtains the first image.

[0036] Step 104: perform anti-counterfeiting identification on the certificate based on the first image to obtain a probability that the certificate is a legitimate certificate.

[0037] In this step, when the electronic device performs anti-counterfeiting identification on the above-mentioned certificate, the anti-counterfeiting algorithm used may include modules such as material anti-counterfeiting, tampering detection, and deepfake detection. Figure 2 As shown, Figure 2 A schematic diagram of an anti-counterfeiting algorithm provided in one embodiment of this specification. Deepfake is a technology that uses deep learning technology to create fake video or audio content. Deepfake realistically replaces the image (e.g., face and / or voice) of one person with that of another, making the forged content appear very real.

[0038] from Figure 2 It can be seen that the material anti-counterfeiting module includes branches such as copying, single-frame color printing, card sleeve, active light color printing and active light high imitation, tampering detection includes branches such as physical tampering, digital tampering, foreign body occlusion and security feature detection, and deepfake detection module includes branches such as single-frame deepfake, multi-vision deepfake and behavioral feature detection. After the electronic device obtains the first image of the document used by the user, it uses Figure 2 The anti-counterfeiting algorithm performs anti-counterfeiting identification on the aforementioned document and obtains a probability that the aforementioned document is legitimate. If the probability is greater than or equal to a first threshold, the document used by the user is legitimate. If the probability is less than a second threshold, the document used by the user is forged. In either case, step 106 need not be executed, and the process can be terminated. If the probability is less than the first threshold and greater than or equal to the second threshold, the document is suspicious and step 106 must be executed.

[0039] Among them, the first threshold is greater than the second threshold. The sizes of the first threshold and the second threshold can be set by yourself during the specific implementation. This embodiment does not limit the sizes of the first threshold and the second threshold. For example, the first threshold can be 80% and the second threshold can be 20%.

[0040] Step 106: When the probability is less than the first threshold and greater than or equal to the second threshold, obtain the second image stored in the certificate and obtain a third image of the user's fingerprint.

[0041] The second image may be a fingerprint image stored in the above-mentioned certificate, and the third image may be a fingerprint image collected in a contactless manner.

[0042] In this step, if the above probability is less than the first threshold and greater than or equal to the second threshold, it means that the ID used by the user is at suspicious risk. Therefore, it is necessary to obtain the fingerprint image stored in the ID and the user's fingerprint image for fingerprint comparison to improve the security of ID authentication.

[0043] Similarly, in this step, if the electronic device is a terminal device, the electronic device can obtain the second image stored in the above-mentioned certificate and the third image of the user's fingerprint collected through the camera.

[0044] If the electronic device is a server, then when acquiring the second image, the electronic device may independently acquire the second image stored in the aforementioned certificate, or may receive the second image sent by the terminal device. When acquiring the third image of the user's fingerprint, the electronic device may receive the third image of the user's fingerprint sent by the terminal device. In other words, when acquiring the second image, the electronic device may acquire the second image independently, or the terminal device may acquire the second image and then send the second image to the electronic device, which then receives the second image sent by the terminal device. When acquiring the third image, the terminal device may capture the third image of the user's fingerprint using a camera, and then the terminal device may send the third image to the electronic device, which then receives the third image sent by the terminal device.

[0045] In some examples, obtaining the second image stored in the above-mentioned certificate may be: when the above-mentioned first image includes a fingerprint image, detecting the position of the above-mentioned fingerprint image in the first image, and based on the above-mentioned position, intercepting the above-mentioned fingerprint image from the first image, and the intercepted above-mentioned fingerprint image is the second image; or, obtaining the fingerprint information stored in the above-mentioned certificate, and obtaining the second image based on the above-mentioned fingerprint information.

[0046] About 20% of the world's ID cards have chips or fingerprint images. For example, a certain country's ID card has a chip on the front and a fingerprint image on the back. Figure 3 As shown, Figure 3This is a schematic diagram of a certificate provided in one embodiment of this specification. Figure 3 Taking the certificate shown in the figure as an example, when obtaining the second image stored in the certificate, since the first image includes the reverse image of the certificate, Figure 3 The back image of the certificate shown includes a fingerprint image, so the electronic device can first detect the position of the fingerprint image in the first image, and then intercept the fingerprint image from the back image of the certificate based on the position. In this way, the intercepted fingerprint image is the second image.

[0047] In addition, due to Figure 3 The front of the certificate shown is provided with a chip. Therefore, when obtaining the second image stored in the certificate, the electronic device can also read the fingerprint information stored in the chip, and then obtain the second image based on the fingerprint information.

[0048] Furthermore, in some countries, the items for certificate registration include fingerprint information, which is stored in the form of digitized fingerprint feature points in the server that manages the above-mentioned certificate. Therefore, the electronic device can also obtain the fingerprint information retained when the above-mentioned certificate is registered from the server that manages the above-mentioned certificate, and then obtain a second image based on the above-mentioned fingerprint information.

[0049] In some further examples, obtaining the third image of the user's fingerprint can be: photographing a hand image of the palm side of the user, performing finger rotation frame detection on the hand image, and performing foreground segmentation on the hand image based on the result of the finger rotation frame detection to obtain a third image including the user's fingerprint. Figure 4 This is a schematic diagram of finger rotation frame detection and foreground segmentation provided in one embodiment of this specification, as shown in FIG. Figure 4 As shown, after capturing a hand image on one side of the user's palm, the electronic device performs finger rotation frame detection on the hand image. The finger rotation frame detection detects whether each finger is on the left hand or the right hand, and frames the area of ​​each finger including the fingerprint in the image. Then, based on the result of the finger rotation frame detection, the electronic device can perform foreground segmentation on the hand image to obtain a third image including the user's fingerprint.

[0050] Step 108: Perform fingerprint comparison based on the third image and the second image.

[0051] Step 110: When the result of the fingerprint comparison is that the fingerprint in the third image matches the fingerprint in the second image, it is determined that the holder and the user of the certificate are the same person.

[0052] In the above-mentioned certificate identification method, after the electronic device obtains the first image of the certificate used by the user, it performs anti-counterfeiting identification on the certificate based on the first image to obtain the probability that the certificate is a legal certificate. When the above-mentioned probability is less than the first threshold value and greater than or equal to the second threshold value, the electronic device obtains the second image stored in the certificate and obtains the third image of the user's fingerprint, and then performs fingerprint comparison based on the third image and the second image. When the result of the above-mentioned fingerprint comparison is that the fingerprint of the third image matches the fingerprint of the second image, it is determined that the holder of the above-mentioned certificate and the user are the same person. In this way, it can be considered that the user authentication has passed, thereby effectively improving the security of the certificate authentication. Taking the large-value transfer scenario in the financial scenario as an example, the improvement of the security of the certificate authentication can effectively reduce financial losses.

[0053] Figure 5 A flowchart of a document recognition method provided in another embodiment of this specification is shown in FIG. Figure 5 As shown in this manual Figure 1 After step 104 in the illustrated embodiment, the following steps may also be included:

[0054] Step 502: When the probability is less than the first threshold and greater than or equal to the second threshold, the user information stored in the certificate is obtained, and optical character recognition is performed on the first image to obtain the user information in the first image.

[0055] Step 504: When the user information stored in the certificate is consistent with the user information in the first image, the certificate is determined to be a legal certificate.

[0056] Specifically, if the ID card is provided with a chip and / or an electronic information code, the chip and / or electronic information code stores user information, which may include name, ID number, and / or date of birth. The electronic information code may be a QR code or a barcode, or other forms of information codes. This embodiment does not limit the form of the electronic information code, as long as the electronic information code can store information.

[0057] Thus, when the probability that the above-mentioned document is a legitimate document is less than a first threshold and greater than or equal to a second threshold, the electronic device can obtain the user information stored in the above-mentioned chip and / or electronic information code, and perform optical character recognition (OCR) on the first image of the document to obtain the user information in the first image. The user information stored in the above-mentioned document is then compared with the user information in the first image. When the user information stored in the above-mentioned document is consistent with the user information in the first image, the above-mentioned document can be determined to be a legitimate document, thereby preventing the information on the document from being tampered with.

[0058] In specific implementation, the electronic device can obtain the user information stored in the above chip through near field communication (NFC); ​​the electronic device can obtain the user information stored in the above electronic information code by identifying the electronic information code.

[0059] In this embodiment, after obtaining the first image of the certificate used by the user, the certificate is firstly subjected to anti-counterfeiting identification based on the first image. If there is a suspicious risk in the certificate, on the one hand, a fingerprint comparison can be performed to determine that the holder and the user of the certificate are the same person. On the other hand, the user information stored in the certificate can be compared with the user information obtained by OCR to prevent the information on the certificate from being tampered with, thereby further improving the security of the certificate authentication.

[0060] Figure 6 A flowchart of a document recognition method provided in another embodiment of this specification is as follows: Figure 6 As shown in this manual Figure 1 Before step 108 in the illustrated embodiment, the following steps may also be included:

[0061] Step 602: Perform finger liveness detection based on the third image.

[0062] Specifically, see Figure 7 , Figure 7 This is a schematic diagram of finger liveness detection provided in one embodiment of this specification, as shown in FIG. Figure 7 As shown, after obtaining the third image of the user's fingerprint, the third image may be processed as follows:

[0063] Through lightweight branch 1, the original fingerprint image is processed to obtain spatial domain feature 1;

[0064] After performing a fast Fourier transform (FFT) frequency domain transformation on the third image, the image is processed by the lightweight branch 2 to obtain a frequency domain feature 2;

[0065] After high-pass filtering, the third image is processed by lightweight branch 3 to obtain high-frequency features 3;

[0066] After the third image is processed by the contour operator, it is processed by the lightweight branch 4 to obtain the contour feature 4;

[0067] After the constrained convolution calculation is performed on the third image, it is processed by the lightweight branch 5 to obtain the constrained feature 5.

[0068] Then, spatial feature 1, frequency feature 2, high-frequency feature 3, contour feature 4, and constraint feature 5 are subjected to multimodal feature fusion. The fused features are processed using a multilayer perceptron (MLP) to obtain the probability that the third image is a live fingerprint image. Thus, when the probability that the third image is a live fingerprint image is greater than or equal to a certain threshold, for example, 85%, the third image can be determined to be a live fingerprint image.

[0069] Step 604: When the liveness detection result indicates that the third image is a live fingerprint image, a fingerprint quality score is performed on the third image.

[0070] Step 606: When the fingerprint quality score of the third image is greater than or equal to a third threshold, perform an expansion transformation on the third image.

[0071] The third threshold can be set arbitrarily during implementation, and this embodiment does not limit the value of the third threshold. For example, scoring the fingerprint quality of the third image can include grading the fingerprint quality of the third image and determining the level corresponding to the fingerprint quality of the third image. Assuming that fingerprint quality is divided into three levels: level 1, level 2, and level 3, with higher levels indicating higher fingerprint quality, the third threshold can be level 2. That is, when the level corresponding to the fingerprint quality of the third image is greater than or equal to level 2, the third image is subjected to an unfolding transformation.

[0072] Specifically, in terms of morphological differences, since fingers are very soft and have a certain volume and thickness, when taking pictures, different areas of the fingers will produce slight perspective distortion due to the different distances from the lens. Similarly, in the process of recording fingerprints, they will also produce distortion due to squeezing. These deformations will bring challenges to the consistency comparison of fingerprints. Therefore, before performing fingerprint comparison, the three-dimensional third image obtained by taking pictures can be subjected to a three-dimensional expansion transformation. Among them, the three-dimensional expansion transformation is to perform perspective expansion and affine transformation on the third image to align it with the two-dimensional feature space of the second image (i.e., the fingerprint).

[0073] Step 608: perform alignment processing on the second image and the third image after the expansion transformation.

[0074] Specifically, as mentioned above, the third image has perspective distortion and the second image has extrusion distortion. These deformations will bring challenges to the consistency comparison of fingerprints. Therefore, before performing fingerprint comparison, the second image and the third image after the expansion transformation can be aligned through a pre-trained alignment correction network, so as to map the second image and the third image to the standard morphological space and eliminate the differences in scale, angle, center and / or distortion between the two modal fingerprint images.

[0075] Step 610: Binarize and extract ridges on the fingerprint image obtained by aligning the second image and the third image.

[0076] Specifically, in terms of visual differences, fingerprints in the hand-printing mode have relatively simple and clear black and white ridge information, but there are issues with discontinuous lines. Fingerprints in the photo-taking mode are generally less noticeable, with lines hidden beneath the richness of skin tones and lighting variations. Therefore, before performing fingerprint comparison, a series of classical operators can be used to map fingerprints from both modalities into a clear and unambiguous binary representation, alleviating the underlying visual differences between the two modalities.

[0077] Figure 8 A schematic diagram of binary ridge extraction provided in one embodiment of this specification is provided. Figure 8 Each line in is the binarization process of a fingerprint. Figure 8 The first column of images from the left is the fingerprint image in the fingerprint mode (i.e., the second image), the second column of images from the left is the image after the center point detection of the fingerprint image in the fingerprint mode, the third column of images from the left is the image after alignment processing of the fingerprint image in the fingerprint mode based on the result of the center point detection, thereby mapping the second image to the standard morphological space, and the fourth column of images from the left is the image after mapping the third column of images from the left to the binary representation consistent space, that is, the fingerprint image obtained after the binary ridge line extraction of the second image. Specifically, when mapping the third column of images from the left to the binary representation consistent space, the third column of images from the left can be filtered and edge detected to obtain valid identity information (i.e., ridge line patterns), and then the obtained valid identity information is mapped to the binary representation consistent space to obtain the fourth column of images from the left.

[0078] Figure 8The first column of images from the right is the fingerprint image in the photo mode (i.e., the third image), the second column of images from the right is the image after the center point detection of the fingerprint image in the photo mode, the third column of images from the right is the image after the fingerprint image in the photo mode is expanded and aligned according to the results of the center point detection, thereby mapping the third image to the standard morphological space, and the fourth column of images from the right is the image after mapping the image in the third column from the right to the binary representation consistent space, that is, the fingerprint image obtained after the binarization ridge line extraction of the third image. Specifically, when mapping the image in the third column from the right to the binary representation consistent space, the image in the third column from the right can be filtered and edge detected to obtain valid identity information (i.e., ridge line patterns), and then the obtained valid identity information is mapped to the binary representation consistent space to obtain the fourth column of images from the right.

[0079] Thus, step 108 may be:

[0080] Step 612: perform fingerprint comparison based on the fingerprint image obtained after performing binarization ridge extraction on the second image and the third image.

[0081] See also Figure 8 The fingerprint image obtained after the second image is binarized and ridge line extraction is the fourth column image from the left, and the fingerprint image obtained after the third image is binarized and ridge line extraction is the fourth column image from the right.

[0082] Specifically, performing fingerprint comparison based on the fingerprint image obtained after binarizing the second image and the third image with ridge extraction can include comparing the fingerprint images obtained after binarizing the second image and the third image with ridge extraction using at least two comparison networks, respectively, to obtain at least two scores. The at least two scores are then fused to obtain a fingerprint comparison score.

[0083] Thus, in step 110 , the fingerprint comparison result that the fingerprint of the third image matches the fingerprint of the second image may be: the fingerprint comparison score is greater than or equal to the fourth threshold.

[0084] The size of the fourth threshold can be set in a specific implementation. This embodiment does not limit the size of the fourth threshold. For example, the fourth threshold can be 80%.

[0085] Specifically, when performing fingerprint comparison, considering that a single comparison network cannot meet the security requirements of document authentication, this embodiment uses at least two comparison networks to perform fingerprint comparison, thereby achieving performance gains and improving the security of document authentication.

[0086] In some examples, after weighing the time and performance, such as Figure 9As shown in FIG, three comparison networks can be used for fingerprint comparison, and the three comparison networks can be respectively: the additive angular margin loss for deep face recognition (Arcface) network, the momentum contrast (Moco) network and the minutiae network. Figure 9 A schematic diagram of a fingerprint comparison network provided in one embodiment of this specification.

[0087] Among them, the Arcface network is a very classic and effective method in face matching. It uses millions of synthetic fingerprint data to learn hyperspheres and establish independent representations for each identifier.

[0088] The MoCo network uses contrastive learning training loss, focusing on learning subtle differences between difficult samples that are difficult to distinguish between positive and negative;

[0089] The minutiae network draws on traditional fingerprint recognition technology, focusing on detail point information such as bifurcations to capture the local consistency of fingerprints.

[0090] In these examples, the electronic device can use the three comparison networks above to identify the fingerprint images obtained after performing binarization ridge extraction on the second and third images, obtaining three scores. The three scores are then combined to obtain a fingerprint comparison score.

[0091] The fingerprint comparison score is then compared with a fourth threshold. If the fingerprint comparison score is greater than or equal to the fourth threshold, it can be determined that the result of the fingerprint comparison is that the fingerprint in the third image matches the fingerprint in the second image. In this way, the electronic device can determine that the holder and user of the certificate are the same person.

[0092] In this embodiment, the electronic device uses at least two comparison networks to perform fingerprint comparison, which can significantly reduce the error rate of fingerprint comparison and improve the accuracy and security of document authentication.

[0093] Figure 10 A flowchart of a document recognition method provided in another embodiment of this specification is as follows: Figure 10 As shown, the above-mentioned document recognition method may include:

[0094] Step 1002: Acquire a first image of the ID card used by the user, wherein the first image includes an image of the front and / or back of the ID card.

[0095] Step 1004: perform anti-counterfeiting identification on the certificate based on the first image to obtain a probability that the certificate is a legitimate certificate.

[0096] Step 1006: Determine whether the probability is less than a first threshold and greater than or equal to a second threshold. If yes, proceed to step 1008, step 1010, step 1016, or step 1018.

[0097] If the probability is greater than or equal to the first threshold, the document can be determined to be a legitimate document; if the probability is less than the second threshold, the document can be determined to be a forged document and the process can be terminated.

[0098] Step 1008: Acquire the second image stored in the above-mentioned certificate, wherein the second image may be the fingerprint image stored in the above-mentioned certificate. Then, execute step 1012.

[0099] The specific implementation method of obtaining the second image stored in the above document can be found in Figure 1 The description of step 106 in the illustrated embodiment will not be repeated here.

[0100] Step 1010: Acquire a third image of the user's fingerprint. Then, execute step 1012.

[0101] The specific implementation method of obtaining the third image of the user's fingerprint can be found in Figure 1 The description of step 106 in the illustrated embodiment will not be repeated here.

[0102] Step 1012: perform fingerprint comparison based on the third image and the second image.

[0103] The specific implementation of fingerprint comparison based on the third image and the second image can be found in Figure 6 The description of the illustrated embodiment will not be repeated here.

[0104] Step 1014: When the fingerprint comparison result shows that the fingerprint in the third image matches the fingerprint in the second image, it is determined that the holder and the user of the certificate are the same person. This process ends.

[0105] Step 1016: Obtain the user information stored in the above-mentioned certificate. Then, execute step 1020.

[0106] The specific implementation method of obtaining the user information stored in the above documents can be found in Figure 5 The description of step 504 in the illustrated embodiment will not be repeated here.

[0107] Step 1018: Perform OCR on the first image to obtain user information in the first image. Then, execute step 1020.

[0108] Step 1020: When the user information stored in the certificate is consistent with the user information in the first image, the certificate is determined to be a legitimate certificate. This process ends.

[0109] In this embodiment, after obtaining the first image of the certificate used by the user, the certificate is firstly subjected to anti-counterfeiting identification based on the first image. If there is a suspicious risk in the certificate, on the one hand, a fingerprint comparison can be performed to determine that the holder and the user of the certificate are the same person. On the other hand, the user information stored in the certificate can be compared with the user information obtained by OCR to prevent the information on the certificate from being tampered with, thereby further improving the security of the certificate authentication.

[0110] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0111] The embodiment of this specification also provides an electronic device, which may include at least one processor; and at least one memory in communication with the processor, wherein: the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute Figures 1 to 10 The method provided by the illustrated embodiment.

[0112] The electronic device may be a terminal device such as a smart phone, a tablet computer or a PC; the electronic device may also be a server, for example, a server with a document recognition function, which may be set up in the cloud.

[0113] Figure 11 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this specification. Figure 11 A block diagram is shown of an exemplary electronic device suitable for implementing embodiments of the present description. Figure 11 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of this specification.

[0114] like Figure 11 As shown, the electronic device is in the form of a general-purpose computing device. Components of the electronic device may include, but are not limited to, one or more processors 1110, a communication interface 1120, a memory 1130, and a communication bus 1140 connecting the various components (including the memory 1130, the communication interface 1120, and the processor 1110).

[0115] The communication bus 1140 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, or a local bus using any of a variety of bus architectures. For example, the communication bus 1140 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnection (PCI) bus.

[0116] Electronic devices typically include a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0117] The memory 1130 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The memory 1130 may include at least one program product having a set (e.g., at least one) program modules configured to execute the instructions of the present invention. Figures 1 to 10 Functionality of the illustrated embodiment.

[0118] A program / utility having a set (at least one) of program modules may be stored in the memory 1130. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment. The program modules generally implement the instructions in this specification. Figures 1 to 10 The functions and / or methods of the described embodiments.

[0119] The processor 1110 executes various functional applications and data processing by running the programs stored in the memory 1130, such as implementing the present specification. Figures 1 to 10 The illustrated embodiment provides a method for document recognition.

[0120] The embodiment of this specification provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to execute the embodiment of this specification. Figures 1 to 10 The illustrated embodiment provides a method for document recognition.

[0121] The above-mentioned non-transitory computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by an instruction execution system, device or device or used in combination with it.

[0122] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0123] Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, radio frequency (RF), etc., or any suitable combination of the foregoing.

[0124] Computer program code for performing the operations of this specification may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0125] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0126] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0127] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout this specification, "plurality" means at least two, such as two or three, unless otherwise specifically defined.

[0128] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of this specification includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of this specification belong.

[0129] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0130] It should be noted that the terminals involved in the embodiments of this specification may include but are not limited to personal computers (PCs), personal digital assistants (PDAs), wireless handheld devices, tablet computers, mobile phones, MP3 players, MP4 players, etc.

[0131] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interface, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0132] In addition, the functional units in the various embodiments of this specification may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional units.

[0133] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform some steps of the method described in various embodiments of this specification. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0134] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A document recognition method, comprising: Acquire a first image of an ID card used by a user; wherein the first image includes an image of the front and / or back of the ID card; performing anti-counterfeiting identification on the document based on the first image to obtain a probability that the document is a legitimate document; When the probability is less than a first threshold and greater than or equal to a second threshold, obtaining a second image stored in the certificate and obtaining a third image of the user's fingerprint; wherein the first threshold is greater than the second threshold, the second image includes the fingerprint image stored in the certificate, and the third image includes the fingerprint image collected in a contactless manner; performing fingerprint comparison based on the third image and the second image; When the result of the fingerprint comparison is that the fingerprint of the third image matches the fingerprint of the second image, it is determined that the holder and the user of the certificate are the same person.

2. The method according to claim 1, wherein After obtaining the probability that the certificate is a legal certificate, the method further includes: When the probability is less than a first threshold and greater than or equal to a second threshold, obtaining user information stored in the certificate, and performing optical character recognition on the first image to obtain user information in the first image; When the user information stored in the certificate is consistent with the user information in the first image, the certificate is determined to be a legal certificate.

3. The method according to claim 1, wherein The obtaining of the second image stored in the certificate includes: When the first image includes a fingerprint image, detecting the position of the fingerprint image in the first image, and intercepting the fingerprint image from the first image based on the position, the intercepted fingerprint image being the second image; or The fingerprint information stored in the certificate is obtained, and the second image is obtained according to the fingerprint information.

4. The method according to claim 1, wherein The obtaining of the third image of the user's fingerprint includes: capturing a hand image of one side of the user's palm; Performing finger rotation frame detection on the hand image; According to the result of the finger rotation frame detection, foreground segmentation is performed on the hand image to obtain a third image including the user's fingerprint.

5. The method according to claim 1, wherein Before performing fingerprint comparison based on the third image and the second image, the method further includes: performing finger liveness detection according to the third image; When the result of the liveness detection is that the third image is a live fingerprint image, performing a fingerprint quality score on the third image; When the fingerprint quality score of the third image is greater than or equal to a third threshold, performing an expansion transformation on the third image; performing alignment processing on the second image and the third image after the expansion transformation; Binarization ridge extraction is performed on the fingerprint image obtained after the second image and the third image are aligned.

6. The method according to claim 5, wherein: The performing fingerprint comparison based on the third image and the second image includes: Fingerprint comparison is performed based on the fingerprint image obtained after binarization ridge extraction of the second image and the third image.

7. The method according to claim 6, wherein: The fingerprint comparison of the fingerprint image obtained after performing binarization ridge extraction on the second image and the third image comprises: Comparing the fingerprint images obtained after performing binarization ridge extraction on the second image and the third image using at least two comparison networks, respectively, to obtain at least two scores; The at least two scores are merged to obtain a fingerprint comparison score.

8. The method according to claim 7, wherein: The result of the fingerprint comparison being that the fingerprint of the third image matches the fingerprint of the second image includes: the fingerprint comparison score being greater than or equal to a fourth threshold.

9. An electronic device comprising: at least one processor; as well as at least one memory in communication with the processor, wherein: The memory stores program instructions that can be executed by the processor, and the processor can execute the method according to any one of claims 1 to 8 by calling the program instructions. 10 . A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions enable the computer to execute the method according to claim 1 .

Images