Multi-modal feature-based identity document spoofing detection

Abstract

Disclosed in embodiments of the present disclosure are a multi-modal feature-based identity document spoofing detection method and apparatus, a medium, and a device. The method comprises: first, determining multi-modal feature data corresponding to an identity document image to be detected, the multi-modal feature data comprising multiple pieces of basic identity document attribute data; determining at least one identity document spoofing risk type corresponding to said identity document image; matching combined identity document attribute data corresponding to the identity document spoofing risk type from the multi-modal feature data; and on the basis of the combined identity document attribute data, determining an identity document spoofing risk detection result corresponding to said identity document image. Compared with the current visual model-based detection solution, the present technical solution can respond to a new spoofing risk only by means of model iterative update, thereby effectively improving the response speed of identity document spoofing risk detection, achieving rapid emergency response to new spoofing risks, and improving the security of private data and user accounts.

Description

Certificate forgery detection based on multi-modal features TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to certificate forgery detection based on multi-modal features. BACKGROUND

[0002] The trusted identity authentication technology refers to a process of remotely completing automatic verification of user identity and registration and authentication of personal information by using an AI (Artificial Intelligence) technology, helping enterprises to understand the real identity of users and reduce risks caused by identity fraud. Compared with a traditional offline identity authentication process, an online verification method is simpler, more efficient and better in user experience. For example, an electronic identity authentication platform eKYC and a digital identity verification platform ADVANCE.AI use the trusted identity authentication technology.

[0003] At present, the trusted identity authentication technology mainly adopts a method of realizing certificate forgery risk detection based on a visual model. However, when a new certificate forgery risk occurs, the detection method based on the visual model needs to be iteratively updated, and even a large amount of sample data needs to be collected to complete the model iterative update. The method cannot realize emergency response to the new certificate forgery risk, and the security of privacy data and user accounts is low. SUMMARY

[0004] In a first aspect of the embodiments of the present disclosure, a certificate forgery detection method based on multi-modal features is provided. The method can effectively improve the response speed of certificate forgery risk detection, realize rapid emergency response to new forgery risks, and improve the security of privacy data and user accounts. The method comprises: acquiring a to-be-detected certificate image, and determining multi-modal feature data corresponding to the to-be-detected certificate image, wherein the multi-modal feature data comprises different types of basic certificate attribute data; determining at least one certificate forgery risk type corresponding to the to-be-detected certificate image; matching combined certificate attribute data corresponding to the certificate forgery risk type from the multi-modal feature data; and determining a certificate forgery risk detection result corresponding to the to-be-detected certificate image by using the combined certificate attribute data.

[0005] Further, in some embodiments, the determination of the multi-modal feature data corresponding to the to-be-detected certificate image comprises: performing attribute feature extraction on the to-be-detected certificate image by using each certificate attribute extraction algorithm in a preset certificate attribute extraction algorithm pool to obtain different types of basic certificate attribute data; and taking each basic certificate attribute data as the multi-modal feature data corresponding to the to-be-detected certificate image.

[0006] Further, in some embodiments, the certificate attribute extraction algorithm pool includes any one or a combination of an optical character recognition algorithm cluster, a visual and information comparison algorithm cluster, a traditional image feature processing algorithm cluster, and a feature visual representation algorithm cluster; the attribute feature extraction of the to-be-detected certificate image is performed by each certificate attribute extraction algorithm in the preset certificate attribute extraction algorithm pool, to obtain different types of basic certificate attribute data, including: the optical character recognition of the to-be-detected certificate image is performed by each optical character recognition algorithm in the optical character recognition algorithm cluster, to obtain at least one basic certificate attribute data containing certificate text features; and / or the certificate information comparison of the to-be-detected certificate image is performed by each information comparison algorithm in the visual and information comparison algorithm cluster, to obtain at least one basic certificate attribute data containing certificate information features; and / or the image feature extraction of the to-be-detected certificate image is performed by each image feature extraction algorithm in the traditional image feature processing algorithm cluster, to obtain at least one basic certificate attribute data containing certificate image features; and / or the visual feature extraction of the to-be-detected certificate image is performed by each visual feature extraction algorithm in the feature visual representation algorithm cluster, to obtain at least one basic certificate attribute data containing certificate visual features.

[0007] Further, in some embodiments, the optical character recognition algorithm includes a text confidence output algorithm, and the optical character recognition of the to-be-detected certificate image by each optical character recognition algorithm in the optical character recognition algorithm cluster to obtain at least one basic certificate attribute data containing certificate text features includes: inputting the to-be-detected certificate image into the text confidence output algorithm, to perform the optical character recognition of the to-be-detected certificate image by the text confidence output algorithm, determine text lines and text single characters, and output text line confidences corresponding to the text lines and single character confidences corresponding to the text single characters; and the text line confidences and the single character confidences are taken as at least one basic certificate attribute data containing certificate text features.

[0008] Further, in some embodiments, the optical character recognition algorithm includes a text box detection algorithm, and the optical character recognition of the to-be-detected certificate image by each optical character recognition algorithm in the optical character recognition algorithm cluster to obtain at least one basic certificate attribute data containing certificate text features includes: inputting the to-be-detected certificate image into the text box detection algorithm, to perform the optical character recognition of the to-be-detected certificate image by the text box detection algorithm, determine text content, and output a text bounding box corresponding to the text content; and the text bounding box is taken as at least one basic certificate attribute data containing certificate text features.

[0009] Further, in some embodiments, the optical character recognition type algorithm includes a text line space detection algorithm, and the obtaining, by each of the optical character recognition type algorithms in the optical character recognition algorithm cluster, at least one basic certificate attribute data containing certificate text features by performing optical character recognition on the to-be-detected certificate image, comprises: inputting the to-be-detected certificate image into the text line space detection algorithm, to perform optical character recognition on the to-be-detected certificate image by the text line space detection algorithm, determine text content and a text bounding box corresponding to the text content, and determine a number of spaces between the text bounding boxes; and taking the number of spaces as the at least one basic certificate attribute data containing certificate text features.

[0010] Further, in some embodiments, the optical character recognition type algorithm includes a text self-checking algorithm, and the obtaining, by each of the optical character recognition type algorithms in the optical character recognition algorithm cluster, at least one basic certificate attribute data containing certificate text features by performing optical character recognition on the to-be-detected certificate image, comprises: inputting the to-be-detected certificate image into the text self-checking algorithm, to perform optical character recognition on the to-be-detected certificate image by the text self-checking algorithm, determine at least two certificate text information, and perform information self-checking on the certificate text information to obtain a first self-checking result; and taking the first self-checking result as the at least one basic certificate attribute data containing certificate text features.

[0011] Further, in some embodiments, the optical character recognition type algorithm includes a text self-checking algorithm, and the obtaining, by each of the optical character recognition type algorithms in the optical character recognition algorithm cluster, at least one basic certificate attribute data containing certificate text features by performing optical character recognition on the to-be-detected certificate image, comprises: inputting the to-be-detected certificate image into the text self-checking algorithm, to perform optical character recognition on the to-be-detected certificate image by the text self-checking algorithm, determine at least two certificate text information, and perform information self-checking on the certificate text information to obtain a first self-checking result; and taking the first self-checking result as the at least one basic certificate attribute data containing certificate text features.

[0012] Further, in some embodiments, the optical character recognition algorithm includes a single word coordinate detection algorithm, the at least one basic certificate attribute data including certificate text features is obtained by performing optical character recognition on the to-be-detected certificate image by each information comparison algorithm in the visual and information comparison algorithm cluster, including: inputting the to-be-detected certificate image into the single word coordinate detection algorithm, to perform optical character recognition on the to-be-detected certificate image by the single word coordinate detection algorithm, determine the single word boundary box corresponding to each text character, and determine the position coordinates corresponding to each single word boundary box; the position coordinates are taken as the at least one basic certificate attribute data including certificate text features.

[0013] Further, in some embodiments, the information comparison algorithm includes a face comparison algorithm, the at least one basic certificate attribute data including certificate information features is obtained by performing certificate information comparison on the to-be-detected certificate image by each information comparison algorithm in the visual and information comparison algorithm cluster, including: inputting the to-be-detected certificate image into the face comparison algorithm, to identify the target face image in the to-be-detected certificate image by the face comparison algorithm, and perform pixel-level alignment comparison between the target face image and the collected face image, to determine the face recognition result; the face recognition result is taken as the at least one basic certificate attribute data including certificate information features.

[0014] Further, in some embodiments, the information comparison algorithm includes a gender comparison algorithm, the at least one basic certificate attribute data including certificate information features is obtained by performing certificate information comparison on the to-be-detected certificate image by each information comparison algorithm in the visual and information comparison algorithm cluster, including: inputting the to-be-detected certificate image into the gender comparison algorithm, to determine the detection gender information corresponding to the face image in the to-be-detected certificate image by the gender comparison algorithm, extract the record gender information in the to-be-detected certificate image, and determine the gender comparison result according to the detection gender information and the record gender information; the gender comparison result is taken as the at least one basic certificate attribute data including certificate information features.

[0015] Further, in some embodiments, the information comparison algorithm includes an age comparison algorithm, and the performing of the document information comparison on the to-be-detected document image by each information comparison algorithm in the visual and information comparison algorithm cluster respectively to obtain at least one basic document attribute data containing document information features includes: inputting the to-be-detected document image into the age comparison algorithm, determining detection age information corresponding to a face image in the to-be-detected document image by the age comparison algorithm, extracting record age information in the to-be-detected document image, and determining an age comparison result according to the detection age information and the record age information; and taking the age comparison result as the at least one basic document attribute data containing document information features.

[0016] Further, in some embodiments, the image feature extraction algorithm includes an edge detection algorithm, and the performing of the image feature extraction on the to-be-detected document image by each image feature extraction algorithm in the traditional image feature processing algorithm cluster respectively to obtain at least one basic document attribute data containing document image features includes: inputting the to-be-detected document image into the edge detection algorithm, determining all edge lines in the to-be-detected document image by the edge detection algorithm; and taking the edge lines as the at least one basic document attribute data containing document image features.

[0017] Further, in some embodiments, the image feature extraction algorithm includes a local image feature classification algorithm, and the performing of the image feature extraction on the to-be-detected document image by each image feature extraction algorithm in the traditional image feature processing algorithm cluster respectively to obtain at least one basic document attribute data containing document image features includes: inputting the to-be-detected document image into the local image feature classification algorithm, extracting image features corresponding to the to-be-detected document image by the local image feature classification algorithm, classifying local region features in the image features, and determining a foreign matter and occlusion detection result in the to-be-detected document image; and taking the foreign matter and occlusion detection result as the at least one basic document attribute data containing document image features.

[0018] Further, in some embodiments, the image feature extraction algorithm includes a background color feature detection algorithm, the image feature extraction by each image feature extraction algorithm in the cluster of conventional image feature processing algorithms on the to-be-detected certificate image to obtain at least one basic certificate attribute data containing certificate image features, comprising: inputting the to-be-detected certificate image into the background color feature detection algorithm, to perform pixel color statistics on the to-be-detected certificate image by the background color feature detection algorithm, to obtain a color histogram, and to determine the background color of each region of interest in the to-be-detected certificate image by the color histogram; and taking the background color of the region of interest as at least one basic certificate attribute data containing certificate image features.

[0019] Further, in some embodiments, the visual feature extraction algorithm includes an image style consistency detection algorithm, the visual feature extraction by each visual feature extraction algorithm in the cluster of feature visual representation algorithms on the to-be-detected certificate image to obtain at least one basic certificate attribute data containing certificate visual features, comprising: inputting the to-be-detected certificate image into the image style consistency detection algorithm, to identify at least two local image style features of each region of interest in the to-be-detected certificate image by the image style consistency detection algorithm, and to determine the image style consistency between the local image style features belonging to the same region of interest; and taking the image style consistency as at least one basic certificate attribute data containing certificate visual features.

[0020] Further, in some embodiments, the visual feature extraction algorithm includes a background extraction algorithm, the visual feature extraction by each visual feature extraction algorithm in the cluster of feature visual representation algorithms on the to-be-detected certificate image to obtain at least one basic certificate attribute data containing certificate visual features, comprising: inputting the to-be-detected certificate image into the background extraction algorithm, to extract background similar features in the to-be-detected certificate image by the background extraction algorithm, and to compare the background similar features with reference background features corresponding to the certificate type of the to-be-detected certificate image, to determine a background comparison result; and taking the background comparison result as at least one basic certificate attribute data containing certificate visual features.

[0021] Further, in some embodiments, the visual feature extraction algorithm includes a foreground extraction algorithm, and the visual feature extraction by each visual feature extraction algorithm in the visual feature extraction algorithm cluster on the to-be-detected certificate image respectively to obtain at least one basic certificate attribute data containing certificate visual features includes: inputting the to-be-detected certificate image into the foreground extraction algorithm, extracting the foreground similar features in the to-be-detected certificate image by the foreground extraction algorithm, and comparing the foreground similar features with reference foreground features corresponding to the certificate type of the to-be-detected certificate image to determine a foreground comparison result; and taking the foreground comparison result as the at least one basic certificate attribute data containing certificate visual features.

[0022] Further, in some embodiments, the determination of the certificate forgery detection result corresponding to the to-be-detected certificate image based on the combined certificate attribute data includes: obtaining a certificate forgery interception parameter corresponding to the to-be-verified certificate attribute; and determining the certificate forgery detection result corresponding to the to-be-detected certificate image based on the matching degree between the combined certificate attribute data and the certificate forgery interception parameter.

[0023] Further, in some embodiments, the determination of the at least one certificate forgery risk type corresponding to the to-be-detected certificate image includes: extracting certificate identification information corresponding to the to-be-detected certificate image, and determining a first certificate type corresponding to the to-be-detected certificate image through the certificate identification information; and determining the at least one certificate forgery risk type corresponding to the to-be-detected certificate image according to the first certificate type.

[0024] Further, in some embodiments, the determination of the at least one certificate forgery risk type corresponding to the to-be-detected certificate image includes: determining a second certificate type specified by the to-be-detected certificate image; and determining the at least one certificate forgery risk type corresponding to the to-be-detected certificate image according to an interception risk event corresponding to the second certificate type.

[0025] Further, in some embodiments, the matching of the combined certificate attribute data corresponding to the certificate forgery risk type from the multi-modal feature data includes: determining at least one to-be-verified certificate attribute corresponding to the certificate forgery risk type; and matching the combined certificate attribute data corresponding to the certificate forgery risk type from the multi-modal feature data based on the to-be-verified certificate attribute.

[0026] In a second aspect of the embodiments of the present disclosure, a certificate forgery detection device based on multi-modal features is further provided, comprising: a multi-modal feature extraction module, configured to acquire a to-be-detected certificate image, and determine multi-modal feature data corresponding to the to-be-detected certificate image, wherein the multi-modal feature data comprises different types of basic certificate attribute data; a risk type determination module, configured to determine at least one certificate forgery risk type corresponding to the to-be-detected certificate image; a certificate attribute data combination module, configured to match combined certificate attribute data corresponding to the certificate forgery risk type from the multi-modal feature data; and a detection result output module, configured to determine a certificate forgery detection result corresponding to the to-be-detected certificate image through the combined certificate attribute data.

[0027] In a third aspect of the embodiments of the present disclosure, a computer program product is further provided, and the computer program product stores at least one instruction, the at least one instruction being adapted to be loaded by a processor and execute the method steps in the first aspect.

[0028] In a fourth aspect of the embodiments of the present disclosure, a storage medium is further provided, and the storage medium stores a computer program, the computer program being adapted to be loaded by a processor and execute the method steps in the first aspect.

[0029] In a fifth aspect of the embodiments of the present disclosure, an electronic device is further provided, comprising: a processor and a memory; wherein the memory stores a computer program, the computer program being adapted to be loaded by the processor and execute the method steps in the first aspect.

[0030] In the embodiments of the present disclosure, by obtaining a to-be-detected certificate image, and determining multi-modal feature data corresponding to the to-be-detected certificate image, the multi-modal feature data includes different types of basic certificate attribute data; at least one certificate forgery risk type corresponding to the to-be-detected certificate image is determined; the combined certificate attribute data corresponding to the certificate forgery risk type is matched from the multi-modal feature data; and the certificate forgery risk detection result corresponding to the to-be-detected certificate image is determined through the combined certificate attribute data. On the one hand, by extracting different types of multi-modal feature data corresponding to the to-be-detected certificate image, and by screening and combining the multi-modal feature data to obtain the combined certificate attribute data, the detection of the certificate forgery risk type is realized. Compared with the current detection method based on a visual model, the model iteration process is not required. Only when a new certificate forgery risk type appears, the multi-modal feature data for detecting the certificate forgery risk type needs to be configured, which effectively improves the response speed, enables quick emergency response to the new certificate forgery risk type, and improves the security of private data and user accounts. On the other hand, by configuring the combined basic certificate attribute data, the detection of the certificate forgery risk type of all types of certificate images can be completed. When a new certificate forgery risk type appears, the basic certificate attribute data for detecting the new certificate forgery risk type needs to be configured, which improves the flexibility of the certificate forgery risk detection system, reduces the operation difficulty of the certificate forgery risk detection, and can cover most of the certificate detection scenarios on the market with low deployment difficulty. BRIEF DESCRIPTION OF DRAWINGS

[0031] FIG. 1 shows a schematic diagram of a system architecture of an exemplary application environment of a certificate forgery detection method and device based on multi-modal features, to which the embodiments of the present disclosure can be applied.

[0032] FIG. 2 is a flowchart of a certificate forgery detection method based on multi-modal features according to an embodiment of the present disclosure.

[0033] FIG. 3 is a flowchart of a process of extracting multi-modal feature data according to an embodiment of the present disclosure.

[0034] FIG. 4 is a flowchart of a process of extracting basic certificate attribute data based on a certificate attribute extraction algorithm pool according to an embodiment of the present disclosure.

[0035] FIG. 5 is a flowchart of a process of determining a certificate forgery detection result according to an embodiment of the present disclosure.

[0036] FIG. 6 is a structural diagram of configuring a certificate forgery risk type corresponding to a to-be-detected certificate image according to an embodiment of the present disclosure.

[0037] FIG. 7 is another structural diagram of configuring a certificate forgery risk type corresponding to a to-be-detected certificate image according to an embodiment of the present disclosure.

[0038] FIG. 8 is a structural schematic diagram of a certificate camouflage detection device based on multi-modal features according to an embodiment of the present disclosure.

[0039] FIG. 9 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0040] For the purposes of the present disclosure, technical solutions and advantages, the following will combine specific embodiments of the present disclosure and corresponding drawings to clearly and completely describe the technical solutions of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, not all embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present disclosure.

[0041] FIG. 1 shows a schematic diagram of a system architecture of an exemplary application environment of a certificate camouflage detection method and device based on multi-modal features according to an embodiment of the present disclosure.

[0042] As shown in FIG. 1, the system architecture 100 can include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a communication link medium between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or optical fiber cables, etc. The terminal devices 101, 102, 103 can be various electronic devices with artificial intelligence (AI) computing capabilities, including but not limited to desktop computers, portable computers, smartphones, and tablet computers, etc. It should be understood that the number of terminal devices, networks and servers in FIG. 1 is only illustrative. According to the needs of implementation, there can be any number of terminal devices, networks and servers. For example, the server 105 can be a server cluster composed of multiple servers, etc.

[0043] The certificate camouflage detection method based on multi-modal features provided by the embodiments of the present disclosure is generally executed by the terminal devices 101, 102, 103, and accordingly, the certificate camouflage detection device based on multi-modal features is generally provided in the terminal devices 101, 102, 103. However, it is easily understood by those skilled in the art that the certificate camouflage detection method based on multi-modal features provided by the embodiments of the present disclosure can also be executed by the server 105, and accordingly, the certificate camouflage detection device based on multi-modal features can also be provided in the server 105, which is not specially limited in this exemplary embodiment.

[0044] Please refer to FIG. 2, which is a flowchart of a certificate forgery detection method based on multi-modal features according to an embodiment of the present disclosure. In the embodiment of the present disclosure, the privacy data protection method for large language models can be applied to a terminal device or a server, and the present example embodiment does not make special limitations on this. In the following, the method is taken as an example of being executed by a server, and the flowchart shown in FIG. 2 is described in detail. The privacy data protection method for large language models in the embodiment of the present disclosure can specifically include the following steps: step S210, obtaining a to-be-detected certificate image and determining multi-modal feature data corresponding to the to-be-detected certificate image, wherein the multi-modal feature data includes different types of basic certificate attribute data; step S220, determining at least one certificate forgery risk type corresponding to the to-be-detected certificate image; step S230, matching combined certificate attribute data corresponding to the certificate forgery risk type from the multi-modal feature data; and step S240, determining a certificate forgery risk detection result corresponding to the to-be-detected certificate image through the combined certificate attribute data.

[0045] According to the certificate forgery detection method based on multi-modal features in the embodiment of the present disclosure, on the one hand, by extracting different types of multi-modal basic certificate attribute data corresponding to the to-be-detected certificate image, and by screening and combining the multi-modal basic certificate attribute data to obtain combined certificate attribute data, the detection of the certificate forgery risk type is realized. Compared with the current detection method based on a visual model, the method does not require a model iteration process. Only when a new certificate forgery risk type appears, the multi-modal basic certificate attribute data for detecting the certificate forgery risk type needs to be configured, which effectively improves the response speed, enables a quick emergency response to the newly added certificate forgery risk type, and improves the security of privacy data and user accounts. On the other hand, by configuring the data types of the combined basic certificate attribute data, the detection of the certificate forgery risk type for all types of certificate images can be completed. When a new certificate forgery risk type appears, the basic certificate attribute data for detecting the new certificate forgery risk type can be configured, which improves the flexibility of the certificate forgery risk detection system, reduces the operation difficulty of the certificate forgery risk detection, and can be compatible with most certificate detection scenarios on the market with low deployment difficulty.

[0046] In the following, the certificate forgery detection method based on multi-modal features in the embodiment of the present disclosure is described.

[0047] In step S210, a to-be-detected certificate image is obtained, and multi-modal feature data corresponding to the to-be-detected certificate image is determined, wherein the multi-modal feature data includes different types of basic certificate attribute data.

[0048] In an example embodiment of the present disclosure, the to-be-detected certificate image refers to an image corresponding to a to-be-verified certificate collected, for example, the to-be-verified certificate can be a carrier bearing personal information and privacy information of a customer, such as a personal identity certificate, a passport certificate, a driver's license, etc.; accordingly, the to-be-detected certificate image can be a personal identity certificate image, or a passport certificate image, a driver's license image; of course, the to-be-detected certificate image can also be an image of other proof certificates with fixed formats, such as a bank card image, an enterprise employee identity certificate image, etc. The example embodiment does not specially limit the type of to-be-detected certificate card in the to-be-detected certificate image.

[0049] The multi-modal feature data refers to various types of basic certificate attribute data extracted from the to-be-detected certificate image for detecting certificate forgery risks, for example, the basic certificate attribute data can be text line confidence corresponding to a text line in the to-be-detected certificate image and single-character confidence corresponding to a single character in the text, a text boundary box corresponding to text content, the number of spaces between the text boundary boxes, information self-checking results in the extracted certificate text information, and position coordinates corresponding to single-character boundary boxes, etc. It can also be a face recognition result, a gender comparison result, and an age comparison result corresponding to the to-be-detected certificate image, etc. It can also be an edge line in the to-be-detected certificate image, a foreign object occlusion detection result, a background color of a region of interest, image style consistency, a background comparison result, and a foreground comparison result, etc. Of course, it can also be other various types of basic certificate attribute data that can be used to detect certificate forgery risks. The specific basic certificate attribute data in the multi-modal feature data can be configured according to actual conditions. The example embodiment does not specially limit the type of basic certificate attribute data contained in the multi-modal feature data.

[0050] It can be understood that an algorithm pool for extracting basic certificate attribute data can be constructed. Each algorithm in the algorithm pool is specially used to extract a single basic certificate attribute data corresponding to the to-be-detected certificate image. The to-be-detected certificate image is processed through each algorithm in the algorithm pool to obtain a plurality of different types of basic certificate attribute data, thereby constructing the multi-modal feature data corresponding to the to-be-detected certificate image. The to-be-detected certificate image can also be recognized and analyzed by an artificial intelligence model to obtain multi-modal feature data containing a plurality of different types of basic certificate attribute data, such as a multi-layer perception machine model for recognizing and analyzing the to-be-detected certificate image to obtain multi-modal feature data, or a multi-gate mixture-of-experts (MMoE) model for recognizing and analyzing the to-be-detected certificate image to obtain multi-modal feature data. Of course, the extraction of the multi-modal feature data corresponding to the to-be-detected certificate image can also be realized by other ways, which are not specially limited by the example embodiment.

[0051] In step S220, it is determined that at least one certificate forgery risk type corresponding to the certificate image to be detected.

[0052] In an example embodiment of the present disclosure, the certificate forgery risk type refers to a type of forgery behavior in the certificate image to be detected by modifying the certificate information and bypassing the detection and identification risk, for example, the certificate forgery risk type can be a handwriting tampering risk type, a foreign object shielding risk type, etc., or a portrait replacement risk type, a text covering risk type, etc., and the specific certificate forgery risk type can be customized according to the actual situation, and the example embodiment does not make special limitations on the risk form or tampering behavior of the certificate forgery risk type of the certificate image to be detected.

[0053] The at least one certificate forgery risk type to be detected by the current certificate image to be detected can be specified by the system, or the at least one certificate forgery risk type to be detected corresponding to the certificate type in the certificate image to be detected can be determined, and the certificate forgery risk type of different certificate types can also be pre-configured by the system. Of course, the certificate forgery risk type of the certificate image to be detected can also be configured in other ways, for example, it can be understood that the certificate forgery risk type corresponding to the certificate image to be detected can also be all the certificate forgery risk types that the system can detect, and the example embodiment does not make special limitations on the determination mode of the certificate forgery risk type to be detected of the certificate image to be detected.

[0054] In step S230, the combined certificate attribute data corresponding to the certificate forgery risk type is matched from the multi-modal feature data.

[0055] In an example embodiment of the present disclosure, the combined certificate attribute data refers to a set of basic certificate attribute data composed of multiple basic certificate attribute data for detecting the certificate forgery risk type, which can be used to detect and identify the certificate forgery risk type in the certificate image to be detected, for example, the certificate forgery risk type can be a foreign object shielding risk type, and then the combined certificate attribute data can be a set composed of basic certificate attribute data such as edge lines in the certificate image to be detected, foreign object shielding detection results, background color of the region of interest, etc., which is used to detect whether the certificate forgery risk type exists in the certificate image to be detected.

[0056] The combined certificate attribute data corresponding to the certificate forgery risk type can be matched from the multi-modal feature data. For example, the to-be-verified attribute corresponding to each certificate forgery risk type can be pre-configured, and the corresponding basic certificate attribute data is matched from the multi-modal feature data as the combined certificate attribute data through the to-be-verified attribute. Of course, the tamper-evident region of the certificate forgery risk type in the to-be-detected certificate image can be determined, and the basic certificate attribute data associated with the tamper-evident region is matched from the multi-modal feature data. The manner of matching the combined certificate attribute data from the multi-modal feature data is not specially limited in this example embodiment.

[0057] In step S240, the certificate forgery risk detection result corresponding to the to-be-detected certificate image is determined through the combined certificate attribute data.

[0058] In an example embodiment of the present disclosure, the certificate forgery risk detection result refers to a result obtained after the to-be-detected certificate image is verified based on the combined certificate attribute data. For example, the certificate forgery risk detection result can be that there is a corresponding certificate forgery risk type or there is no corresponding certificate forgery risk type, or the certificate verification passes or the certificate verification fails. The manifestation manner of the certificate forgery risk detection result is not specially limited in this example embodiment.

[0059] The pre-set certificate forgery interception parameter (which can be simply understood as a risk interception rule) can be obtained, so as to compare the certificate forgery interception parameter with the combined certificate attribute data, thereby determining the certificate forgery risk detection result corresponding to the to-be-detected certificate image. The certificate forgery interception parameter can determine whether the combined certificate attribute data can reflect the corresponding certificate forgery risk type. The specific certificate forgery interception parameter can correspond to each basic certificate attribute data in the combined certificate attribute data.

[0060] For example, assuming that the combined certificate attribute data can include the basic certificate attribute data of the text line confidence corresponding to the text line, then correspondingly, the certificate forgery interception parameter can be a text line confidence threshold. When the text line confidence corresponding to the text line is greater than or equal to the text line confidence threshold, it can be considered that the basic certificate attribute data does not belong to the attribute feature corresponding to the certificate forgery risk type, so that it can be determined that the combined certificate attribute data does not belong to the attribute feature corresponding to the certificate forgery risk type, and the certificate forgery risk detection result of the to-be-detected certificate image is that there is no corresponding certificate forgery risk type or the certificate verification passes. Conversely, it can be considered that the combined certificate attribute data belongs to the attribute feature corresponding to the certificate forgery risk type, and the certificate forgery risk detection result of the to-be-detected certificate image is that there is a corresponding certificate forgery risk type or the certificate verification fails.

[0061] It can be understood that only one basic certificate attribute data is included in the combined certificate attribute data as an example here. In actual application, the combined certificate attribute data can generally include multiple basic certificate attribute data. At this time, it can be selected that when any basic certificate attribute data does not meet the certificate disguise interception parameter, it is considered that the to-be-detected certificate image exists a certificate disguise risk type, otherwise it does not exist a certificate disguise risk type. It can also be selected that when the proportion of the number of basic certificate attribute data meeting the certificate disguise interception parameter to the total number of basic certificate attribute data in the combined certificate attribute data is less than a preset proportion threshold, it is considered that the to-be-detected certificate image exists a certificate disguise risk type, otherwise it does not exist a certificate disguise risk type. The above is only illustrative and should not cause any special limitation to the example embodiment.

[0062] By extracting multiple basic certificate attribute data of different types corresponding to the to-be-detected certificate image, and by screening and combining multiple basic certificate attribute data to obtain combined certificate attribute data, the detection of the certificate disguise risk type is realized. Compared with the current detection method based on the visual model, the model iteration process is not needed. Only when a new disguise risk type appears, multiple basic certificate attribute data for detecting the disguise risk type are configured, the response speed is effectively improved, the rapid emergency response to the newly added disguise risk type can be realized, and the security of the privacy data and the user account is improved. By configuring the data type of the combined basic certificate attribute data, the disguise risk detection of all types of certificate images can be completed. When a new disguise risk type appears, the basic certificate attribute data for detecting the newly added disguise risk type is configured, the flexibility of the disguise risk detection system is improved, the operation difficulty of the disguise risk detection is reduced, most of the certificate detection scenes on the market can be compatible and covered, and the deployment difficulty is low.

[0063] Next, steps S210 to S240 are described in detail.

[0064] In an example embodiment of the present disclosure, the determination of the multi-modal feature data corresponding to the to-be-detected certificate image can be implemented by steps in FIG. 3. Referring to FIG. 3, the steps can specifically include: step S310, performing attribute feature extraction on the to-be-detected certificate image by each certificate attribute extraction algorithm in a preset certificate attribute extraction algorithm pool to obtain different types of basic certificate attribute data; and step S320, taking each basic certificate attribute data as the multi-modal feature data corresponding to the to-be-detected certificate image.

[0065] The certificate attribute extraction algorithm pool refers to a set of algorithms pre-configured for extracting attribute features of the to-be-detected certificate image. Each algorithm in the certificate attribute extraction algorithm pool is dedicated to extracting a certain basic certificate attribute data in the to-be-detected certificate image, so as to realize detection and identification of one or more types of disguise risks. For example, the certificate attribute extraction algorithm pool can include a text confidence output algorithm dedicated to extracting text line confidence corresponding to a text line and single-character confidence corresponding to a single character in the to-be-detected certificate image. The certificate attribute extraction algorithm pool can also include a face comparison algorithm dedicated to determining a face recognition result of a face image in the to-be-detected certificate image. Of course, the certificate attribute extraction algorithm pool can also include other types of basic algorithms for extracting basic certificate attribute data in the to-be-detected certificate image. The certificate attribute extraction algorithm pool can be customized according to the basic certificate attribute data required in the actual application process, and the present example embodiment does not make special limitations thereto.

[0066] The certificate attribute extraction algorithm pool can effectively improve the richness of the extracted basic certificate attribute data, thereby improving the number of detectable and identifiable certificate disguise risk types and ensuring the coverage range of the detection. At the same time, since the certificate attribute extraction algorithm pool can be flexibly configured, when a new certificate disguise risk type appears, the corresponding attribute extraction algorithm required for extracting the basic certificate attribute data of the certificate disguise risk type can be added to the certificate attribute extraction algorithm pool to complete the response to the new certificate disguise risk type, which is fast in response speed. Compared with the visual model iterative updating method in the related art, the workload of extracting a single basic certificate attribute data is smaller, and the setting is more rapid, thereby effectively ensuring the emergency response speed and improving the security of the privacy data and the user account.

[0067] Optionally, the certificate attribute extraction algorithm pool can include any one or a combination of the following: an optical character recognition algorithm cluster, a visual and information comparison algorithm cluster, a traditional image feature processing algorithm cluster, and a feature visual representation algorithm cluster. The optical character recognition algorithm cluster refers to a set of algorithms including optical character recognition algorithms, such as a text confidence output algorithm, a text box detection algorithm, a text line space detection algorithm, a text self-checking algorithm, an interesting text checking algorithm, and a single word coordinate detection algorithm. The visual and information comparison algorithm cluster refers to a set of algorithms including information comparison algorithms, such as a face comparison algorithm, a gender comparison algorithm, and an age comparison algorithm. The traditional image feature processing algorithm cluster refers to a set of algorithms including image feature extraction algorithms, such as an edge detection algorithm, a local image feature classification algorithm, and a background color feature detection algorithm. The feature visual representation algorithm cluster refers to a set of algorithms including visual feature extraction algorithms, such as an image style consistency detection algorithm, a background extraction algorithm, and a foreground extraction algorithm. Of course, the attribute feature extraction algorithms in the above algorithm clusters are only illustrative examples, and the present embodiment is not limited thereto.

[0068] It can be understood that different types of algorithms can be set in the certificate attribute extraction algorithm pool according to different application scenarios. For example, the certificate attribute extraction algorithm pool can be set with any one of the following: an optical character recognition algorithm cluster, a visual and information comparison algorithm cluster, a traditional image feature processing algorithm cluster, and a feature visual representation algorithm cluster. It can also be set with any two of the following: an optical character recognition algorithm cluster, a visual and information comparison algorithm cluster, a traditional image feature processing algorithm cluster, and a feature visual representation algorithm cluster. It can also be set with any three of the following: an optical character recognition algorithm cluster, a visual and information comparison algorithm cluster, a traditional image feature processing algorithm cluster, and a feature visual representation algorithm cluster. Of course, it can also be set with an optical character recognition algorithm cluster, a visual and information comparison algorithm cluster, a traditional image feature processing algorithm cluster, and a feature visual representation algorithm cluster. The present example embodiment does not limit the type and number of algorithms set in the certificate attribute extraction algorithm pool.

[0069] The steps shown in Figure 4 can be used to extract attribute features from the document image to be detected using various document attribute extraction algorithms in the preset document attribute extraction algorithm pool, thereby obtaining basic document attribute data of different types. Referring to Figure 4, this can specifically include: step S410, performing optical character recognition on the document image to be detected using various optical character recognition algorithms in the optical character recognition algorithm cluster to obtain at least one basic document attribute data containing document text features; and / or step S420, performing optical character recognition on the document image to be detected using various information comparison algorithms in the visual and information comparison algorithm cluster to obtain at least one basic document attribute data containing document text features; and / or step S420, performing optical character recognition on the document image to be detected using various information comparison algorithms in the visual and information comparison algorithm cluster to obtain at least one basic document attribute data containing document text features. The image of the document to be detected is compared with document information to obtain at least one basic document attribute data containing document information features; and / or in step S430, the image feature extraction algorithms in the traditional image feature processing algorithm cluster are used to extract image features from the image of the document to be detected to obtain at least one basic document attribute data containing document image features; and / or in step S440, the visual feature extraction algorithms in the feature visual representation algorithm cluster are used to extract visual features from the image of the document to be detected to obtain at least one basic document attribute data containing document visual features.

[0070] Understandably, steps S410 to S420 can be selectively executed depending on the different attribute feature extraction algorithms set in the document attribute extraction algorithm pool. For example, when the document attribute extraction algorithm pool includes an optical character recognition algorithm cluster, a visual and information comparison algorithm cluster, a traditional image feature processing algorithm cluster, or a feature visual representation algorithm cluster, steps S410, S420, S430, or S440 can be executed accordingly. Similarly, when the document attribute extraction algorithm pool includes any two, three, or four combinations of the optical character recognition algorithm cluster, visual and information comparison algorithm cluster, traditional image feature processing algorithm cluster, and feature visual representation algorithm cluster, the corresponding steps can be selected and executed accordingly. This example embodiment does not impose any special limitations on this.

[0071] In an optional implementation, the optical character recognition algorithm may include a text confidence output algorithm, which can determine at least one basic document attribute data containing document text features through the following steps. Specifically, it may include: inputting the document image to be detected into the text confidence output algorithm to perform optical character recognition on the document image to be detected, determining text lines and text characters, and outputting the text line confidence corresponding to the text line and the text character confidence corresponding to the text character. Then, the text line confidence and the text character confidence can be used as at least one basic document attribute data containing document text features.

[0072] The text line confidence refers to an estimated confidence degree of an output text line, which is a comprehensive score based on factors such as text line layout rationality. The single character confidence refers to an estimated confidence degree of an output single character, which is a comprehensive score based on factors such as character recognition accuracy. The text line confidence and the single character confidence can be used to detect and identify the risk types of note tampering (coarse), foreign matter shielding, handwritten text and the like.

[0073] For example, when detecting interception, a text line confidence threshold and a single character confidence threshold can be set. When the text line confidence is greater than or equal to the text line confidence threshold, and / or the single character confidence is greater than or equal to the single character confidence threshold, it can be determined that the risk types of note tampering (coarse), foreign matter shielding, handwritten text and the like do not exist in the to-be-detected certificate image. Of course, it can be understood that the detection and identification result of a single basic certificate attribute data is only a detection evaluation item for a certain type of certificate forgery risk, and the corresponding detection and identification result cannot be directly used as the certificate forgery risk detection result of the to-be-detected certificate image. Instead, it is used as a judgment standard for the certificate forgery risk detection result. The specific certificate forgery risk detection result needs to be determined in combination with the detection and evaluation results of other basic certificate attribute data in the combined certificate attribute data, which will not be described in detail here.

[0074] By using the text line confidence and the single character confidence as basic certificate attribute data, the detection and identification of the risk types of note tampering (coarse), foreign matter shielding, handwritten text and the like can be effectively realized.

[0075] In an optional embodiment, the optical character recognition algorithm can include a text box detection algorithm. At least one basic certificate attribute data containing certificate text features can be determined by the following steps, which can specifically include: inputting the to-be-detected certificate image into the text box detection algorithm to perform optical character recognition on the to-be-detected certificate image by the text box detection algorithm, determining the text content, and outputting the text boundary box corresponding to the text content; and taking the text boundary box as the at least one basic certificate attribute data containing certificate text features.

[0076] The text boundary box refers to a bounding box defined for each line of text in the OCR recognition process, which contains the position, size and direction of the text line. Since the position, size and direction of the text boundary box in the to-be-detected certificate image of the same type of certificate are basically fixed and unchanged, the text boundary box can be used to detect and identify the risk types of note tampering (coarse), adding notes in front of the text line or shielding foreign matter at the beginning of the text line and the like.

[0077] In an optional implementation, the optical character recognition algorithm can include a text line space detection algorithm, and the at least one basic certificate attribute data containing the certificate text feature can be determined by the following steps, which can specifically include: inputting the to-be-detected certificate image into the text line space detection algorithm to perform optical character recognition on the to-be-detected certificate image by the text line space detection algorithm, determining the text content and the text boundary box corresponding to the text content, and determining the number of spaces between the text boundary boxes; and taking the number of spaces as the at least one basic certificate attribute data containing the certificate text feature.

[0078] The number of spaces between the text boundary boxes refers to the number of space states between the text boundary boxes corresponding to the text content in the to-be-detected certificate image. Generally, the text lines or text contents in the same certificate type conform to a certain form, and the intervals between the text lines or text contents are fixed. By detecting the number of spaces between the text boundary boxes, the detection and recognition of the certificate forgery risk types such as small foreign matter occlusion in the text and text occlusion and covering can be effectively implemented.

[0079] In an optional implementation, the optical character recognition algorithm can include a text self-checking algorithm, and the at least one basic certificate attribute data containing the certificate text feature can be determined by the following steps, which can specifically include: inputting the to-be-detected certificate image into the text self-checking algorithm to perform optical character recognition on the to-be-detected certificate image by the text self-checking algorithm, determining at least two certificate text information, and performing information self-checking on the certificate text information to obtain a first self-checking result, and then taking the first self-checking result as the at least one basic certificate attribute data containing the certificate text feature.

[0080] The certificate text information refers to the text information obtained by performing optical character recognition on the to-be-detected certificate image, for example, taking the certificate type in the to-be-detected certificate image as a personal identification card, the certificate text information can be the birth date and the certificate number, the recognized birth date is checked with the birth date contained in the certificate number to determine whether they match; of course, the certificate text information can be gender information and a certificate number, the recognized gender information is checked with the gender identifier contained in the certificate number to determine whether they match, of course, the type of the certificate text information extracted from the to-be-detected certificate image is not specially limited in this embodiment. The first self-checking result refers to the checking result obtained by performing information self-checking on the recognized certificate text information.

[0081] The certificate text information can effectively implement the detection and recognition of the certificate forgery risk types such as certificate number tampering, gender tampering, and name compliance.

[0082] In an optional implementation, the optical character recognition algorithm can include an interested text verification algorithm, and the at least one basic certificate attribute data containing the certificate text feature can be determined by the following steps, which can specifically include: the image of the certificate to be detected can be input into the interested text verification algorithm, so as to determine the interested visual reading area of the certificate type corresponding to the image of the certificate to be detected through the interested text verification algorithm, and the optical character recognition is performed on the interested visual reading area, at least two interested certificate information is determined, and the information self-checking is performed on the interested certificate information to obtain a second self-checking result; and then the second self-checking result can be taken as the at least one basic certificate attribute data containing the certificate text feature.

[0083] The interested visual reading area refers to an area containing key identification information in the image of the certificate to be detected, for example, when the image of the certificate to be detected is a passport certificate, the interested visual reading area can be the name area, the certificate number area, the machine readable area (MRZ), the validity period area and the birth date area in the passport certificate, etc. The second self-checking result refers to a verification result obtained by performing information self-checking on the interested certificate information in the interested visual reading area.

[0084] The second self-checking result obtained by performing information self-checking on the interested certificate information in the interested visual reading area can effectively realize the detection and identification of the certificate disguise risk type such as the tampering of the entire row of content.

[0085] In an optional implementation, the optical character recognition algorithm can include a single word coordinate detection algorithm, and the at least one basic certificate attribute data containing the certificate text feature can be determined by the following steps, which can specifically include: the image of the certificate to be detected can be input into the single word coordinate detection algorithm, so as to perform the optical character recognition on the image of the certificate to be detected through the single word coordinate detection algorithm, determine the single word boundary box corresponding to each text character, and determine the position coordinates corresponding to each single word boundary box; and then the position coordinates can be taken as the at least one basic certificate attribute data containing the certificate text feature.

[0086] The single word boundary box refers to a boundary box determined for the recognized text character, for example, the single word boundary box can be determined by a text line detection method based on single word detection, such as Craft, or the single word boundary box can be determined by an image recognition processing method based on the text line boundary box, and the embodiment is not specially limited to the method for determining the single word boundary box.

[0087] After the single-character bounding boxes are determined, the position coordinates corresponding to the single-character bounding boxes can be determined, and through the position coordinates corresponding to the single-character bounding boxes, the detection and identification of the certificate forgery risk types such as single-character tampering, large foreign object covering and shielding and the like can be effectively realized. For example, whether a certain single character is tampered with can be determined by detecting whether the up, down, left and right of the single-character bounding boxes in the same text line are aligned through the position coordinates.

[0088] In an optional embodiment, the information comparison algorithm can include a face comparison algorithm, and the at least one basic certificate attribute data containing the certificate information features can be determined by the following steps, which can specifically include: the face comparison algorithm can be input into the to-be-detected certificate image to identify the target face image in the to-be-detected certificate image through the face comparison algorithm, and the target face image and the collected face image are pixel-level aligned and compared to determine the face recognition result; and then the face recognition result can be used as the at least one basic certificate attribute data containing the certificate information features.

[0089] The target face image refers to the face image detected and identified from the to-be-detected certificate image, and the collected face image refers to the face image corresponding to the certificate user in the to-be-detected certificate image and pre-collected. The face recognition result is determined by pixel-level alignment and comparison of the target face image and the collected face image, and then the face recognition result can be used to effectively realize the detection and identification of the certificate forgery risk types such as portrait covering tampering, large face tampering and the like.

[0090] In an optional embodiment, the information comparison algorithm can include a gender comparison algorithm, and the at least one basic certificate attribute data containing the certificate information features can be determined by the following steps, which can specifically include: the gender comparison algorithm can be input into the to-be-detected certificate image to determine the detection gender information corresponding to the face image in the to-be-detected certificate image through the gender comparison algorithm, and extract the recorded gender information in the to-be-detected certificate image, and determine the gender comparison result according to the detection gender information and the recorded gender information; and then the gender comparison result can be used as the at least one basic certificate attribute data containing the certificate information features.

[0091] The detection gender information refers to the gender information inferred and predicted based on the artificial intelligence recognition of the face image in the to-be-detected certificate image, and the recorded gender information refers to the gender information obtained by performing optical character recognition on the to-be-detected certificate image. Through the gender comparison result, the detection and identification of the certificate forgery risk types such as portrait replacement tampering, gender information tampering, and certificate number partial tampering can be effectively realized.

[0092] In an optional implementation, the information comparison algorithm can include an age comparison algorithm, and the determination of the at least one basic certificate attribute data containing the certificate information features can be implemented by the following steps, which can specifically include: inputting the to-be-detected certificate image into the age comparison algorithm to determine the detection age information corresponding to the face image in the to-be-detected certificate image through the age comparison algorithm, extracting the recorded age information in the to-be-detected certificate image, and determining an age comparison result according to the detection age information and the recorded age information; and then the age comparison result can be taken as the at least one basic certificate attribute data containing the certificate information features.

[0093] The detection age information refers to the age information inferred and predicted based on the artificial intelligence recognition of the face image in the to-be-detected certificate image, and the recorded age information refers to the age information obtained by performing optical character recognition on the to-be-detected certificate image. The age comparison result can effectively implement the detection and recognition of the certificate forgery risk types such as portrait replacement tampering, birth date tampering, and certificate number partial tampering.

[0094] In an optional implementation, the image feature extraction algorithm can include an edge detection algorithm, and the determination of the at least one basic certificate attribute data containing the certificate information features can be implemented by the following steps, which can specifically include: inputting the to-be-detected certificate image into the edge detection algorithm to determine all edge lines in the to-be-detected certificate image through the edge detection algorithm; and then the edge lines can be taken as the at least one basic certificate attribute data containing the certificate image features.

[0095] The edge lines refer to the edge features extracted from the to-be-detected certificate image through the edge detection algorithm, and the edge lines can effectively identify the features that have obvious differences from real certificates in tampering attacks, for example, the edge lines can be extracted from the to-be-detected certificate image through the Canny algorithm, Sobel algorithm, etc. based on edge detection.

[0096] The edge lines can effectively implement the detection and recognition of the certificate forgery risk types such as the covering tampering of the text lines with obvious edges (such as the forgery behavior of directly covering the corresponding text lines with a paper strip written with tampering information), the single-character or single-word covering tampering, and the rectangular frame covering face tampering.

[0097] In an optional implementation, the image feature extraction algorithm can include a local image feature classification algorithm, and the determination of the at least one basic certificate attribute data containing the certificate information feature can be implemented by the following steps, which can specifically include: inputting the to-be-detected certificate image into the local image feature classification algorithm, extracting the image feature corresponding to the to-be-detected certificate image through the local image feature classification algorithm, classifying the local region features in the image feature, and determining the foreign matter shielding detection result in the to-be-detected certificate image; and then the foreign matter shielding detection result can be taken as the at least one basic certificate attribute data containing the certificate image feature.

[0098] The foreign matter shielding detection result refers to the identification result of foreign matters that do not belong to the certificate in the to-be-detected certificate image. For example, the LBP, HOG or other algorithm can be used to extract the image feature corresponding to the to-be-detected certificate image, the local features of the foreign matter region can exist in the image feature, and the foreign matter condition that can exist in the to-be-detected certificate image can be detected by classifying the local region features in the image feature, and the detected foreign matter condition is taken as the foreign matter shielding detection result in the to-be-detected certificate image.

[0099] The foreign matter shielding detection result can effectively realize the detection and identification of the certificate disguise risk type with foreign matter shielding, covering and tampering.

[0100] In an optional implementation, the image feature extraction algorithm can include a background color feature detection algorithm, and the determination of the at least one basic certificate attribute data containing the certificate information feature can be implemented by the following steps, which can specifically include: inputting the to-be-detected certificate image into the background color feature detection algorithm, performing pixel color statistics on the to-be-detected certificate image through the background color feature detection algorithm to obtain a color histogram, and determining the background color of each region of interest in the to-be-detected certificate image through the color histogram; and then the background color of the region of interest can be taken as the at least one basic certificate attribute data containing the certificate image feature.

[0101] The color histogram refers to a statistical histogram used to describe the proportion of different colors in the to-be-detected certificate image. The background color of each region of interest can be represented by the color feature corresponding to the color histogram. Since different certificates are provided with anti-fake background colors or identification background colors, the background color of each region of interest in the to-be-detected certificate image can be determined by the color distribution ratio, which can effectively distinguish the difference between the tampering attack and the real certificate. For example, when there is a handwriting tampering, covering tampering or foreign matter shielding, the distribution ratio of the background color of the region of interest can be changed.

[0102] The background color (such as the color distribution ratio) of the region of interest can effectively realize detection and identification of the certificate forgery risk types such as handwriting tampering, overlay tampering, or foreign matter shielding.

[0103] In an optional embodiment, the visual feature extraction algorithm can include an image style consistency detection algorithm, and the at least one basic certificate attribute data containing the visual features of the certificate can be determined by the following steps: the image to be detected can be input into the image style consistency detection algorithm, so as to identify at least two local image style features of each region of interest in the image to be detected by the image style consistency detection algorithm, and determine the image style consistency between the local image style features belonging to the same region of interest; and then the image style consistency can be taken as the at least one basic certificate attribute data containing the visual features of the certificate.

[0104] The local image style feature refers to a feature reflecting the image content style in the region of interest, for example, the local image style feature can be the background color, font handwriting, font size, font color, and the like of each part in the region of interest; generally, the local image style features in the same region of interest should be consistent, and if there is a tampering attack behavior such as handwriting tampering, overlay tampering, or foreign matter shielding, the local image style features in the region of interest can be inconsistent, for example, the font handwriting, font size, font color, and the like cannot be consistent due to handwriting tampering attack.

[0105] The image style consistency between the local image style features belonging to the same region of interest can effectively realize detection and identification of the certificate forgery risk types such as handwriting tampering, overlay tampering, or foreign matter shielding.

[0106] In an optional embodiment, the visual feature extraction algorithm can include a background extraction algorithm, and the at least one basic certificate attribute data containing the visual features of the certificate can be determined by the following steps: the image to be detected can be input into the background extraction algorithm, so as to extract the background similar features in the image to be detected by the background extraction algorithm, compare the background similar features with the reference background features corresponding to the certificate type of the image to be detected, and determine the background comparison result; and then the background comparison result can be taken as the at least one basic certificate attribute data containing the visual features of the certificate.

[0107] The background similar feature refers to a feature shared by the background regions of the same certificate in the image to be detected, for example, the background similar feature can include the background color distribution, background pattern texture, background group shape, and the like; these features are usually irrelevant to the foreground image or text content in the image to be detected. The reference background feature refers to a feature corresponding to the real background of the same certificate in the image to be detected.

[0108] The background comparison result of the extracted background similar features and the reference background features can effectively realize detection and identification of various possible certificate background shielding certificate forgery risk types, such as handwriting tampering, overlay tampering, foreign matter shielding, portrait replacement, and text line covering.

[0109] In an optional embodiment, the visual feature extraction algorithm can include a foreground extraction algorithm, and the determination of the at least one basic certificate attribute data containing the certificate visual features can be realized by the following steps: the to-be-detected certificate image can be input into the foreground extraction algorithm to extract the foreground similar features in the to-be-detected certificate image by the foreground extraction algorithm, and the foreground similar features are compared with the reference foreground features corresponding to the certificate type of the to-be-detected certificate image to determine a foreground comparison result; and then the foreground comparison result can be taken as the at least one basic certificate attribute data containing the certificate visual features.

[0110] The foreground similar features refer to features shared by the foreground of the to-be-detected certificate in the to-be-detected certificate image, for example, the foreground similar features can include contour or edge features, significant corner features, geometric shape features, color and texture features of the foreground text or image in the to-be-detected certificate image. The reference foreground features refer to features corresponding to the real foreground of the same certificate in the to-be-detected certificate image.

[0111] The background comparison result of the extracted foreground similar features and the reference foreground features can effectively realize detection and identification of various possible certificate foreground shielding or modifying certificate forgery risk types, such as handwriting tampering, overlay tampering, foreign matter shielding, portrait replacement, and text line covering.

[0112] The certificate attribute extraction algorithms in the above certificate attribute extraction algorithm pool are only illustrative examples, and the present example embodiment is not limited thereto. By configuring the certificate attribute extraction algorithms in the certificate attribute extraction algorithm pool, detection and identification of any certificate forgery risk type can be realized, that is, by extracting multiple basic certificate attribute data in the to-be-detected certificate image through the certificate attribute extraction algorithm pool, and matching the combined basic certificate attribute data for detecting and identifying the certificate forgery risk type, detection of the forgery risk of all types of certificate images can be completed. When a new forgery risk type appears, by configuring the basic certificate attribute data corresponding to the detection of the new forgery risk type, the flexibility of the forgery risk detection system is improved, the operation difficulty of the forgery risk detection is reduced, most of the certificate detection scenarios on the market can be compatible and covered, and the deployment difficulty is low.

[0113] In an example embodiment of the present disclosure, matching the combined certificate attribute data corresponding to the certificate forgery risk type from the multi-modal feature data can be achieved through the following steps, which can specifically include: at least one to-be-verified certificate attribute corresponding to the certificate forgery risk type can be determined, and the combined certificate attribute data corresponding to the certificate forgery risk type is matched from the multi-modal feature data based on the to-be-verified certificate attribute.

[0114] The to-be-verified certificate attribute refers to an attribute item pre-configured for detecting and identifying the certificate forgery risk type, for example, the to-be-verified certificate attribute can be a text line confidence attribute item, a single word confidence attribute item, a text boundary box attribute item, etc. Each to-be-verified certificate attribute corresponds to a corresponding basic certificate attribute data one by one, for example, if the to-be-verified certificate attribute is a text line confidence attribute item, the text line confidence is matched from the multi-modal feature data through the to-be-verified certificate attribute; similarly, each certificate forgery risk type corresponds to multiple to-be-verified certificate attributes, and then the multiple basic certificate attribute data for detecting and identifying the certificate forgery risk type corresponding to the certificate forgery risk type can be matched from the multi-modal feature data according to the to-be-verified certificate attribute, and the combined certificate attribute data is formed by the multiple basic certificate attribute data.

[0115] Through at least one to-be-verified certificate attribute corresponding to the certificate forgery risk type, the combined certificate attribute data for detecting and identifying the certificate forgery risk type in the to-be-detected certificate image can be quickly matched from the multi-modal feature data, thereby improving the certificate forgery detection efficiency; and by configuring the to-be-verified certificate attribute corresponding to the certificate forgery risk type, the configuration of the certificate forgery risk type can be completed, the detection and identification difficulty of the certificate forgery risk is reduced, and the configuration efficiency is improved.

[0116] Optionally, the combined certificate attribute data can be used to determine the certificate forgery detection result corresponding to the to-be-detected certificate image through the steps in FIG. 5, as shown in FIG. 5, which can specifically include: step S510, obtaining the certificate forgery interception parameter corresponding to the to-be-verified certificate attribute; step S520, determining the certificate forgery detection result corresponding to the to-be-detected certificate image based on the matching degree between the combined certificate attribute data and the certificate forgery interception parameter.

[0117] The certificate forgery interception parameter refers to a parameter for setting the interception rule corresponding to each basic certificate attribute data, for example, the certificate forgery interception parameter can be set with a text line confidence threshold and a single word confidence threshold, so that when the text line confidence is greater than or equal to the text line confidence threshold, and / or the single word confidence is greater than or equal to the single word confidence threshold, it can be determined that there is no certificate forgery risk type such as note tampering (coarse), foreign matter shielding, and handwritten text in the to-be-detected certificate image.

[0118] The matching degree refers to the ratio of the number of basic certificate attribute data verified by comparison and verification of the certificate disguise interception parameters in the combined certificate attribute data to the total number of basic certificate attribute data in the combined certificate attribute data. For example, if the matching degree is greater than or equal to the matching degree threshold value, it can be considered that the to-be-detected certificate image basically does not exist the certificate disguise risk type, and at this time, the certificate disguise detection result corresponding to the to-be-detected certificate image is that there is no certificate disguise risk type or the detection and recognition pass.

[0119] Of course, in some embodiments, the matching degree is considered to be completely matched only when the to-be-detected certificate image basically does not exist the certificate disguise risk type, that is, the matching degree threshold value is 100%; when the comparison and verification of any basic certificate attribute data and the corresponding certificate disguise interception parameter fails, it can be considered that the certificate disguise detection result corresponding to the to-be-detected certificate image is that there is a certificate disguise risk type or the detection and recognition fail.

[0120] By configuring the certificate disguise interception parameters corresponding to each to-be-verified certificate attribute, the detection and verification of each basic certificate attribute data in the combined certificate attribute data can be effectively realized, and the accuracy of the certificate disguise detection result can be improved; and the detection and recognition of various certificate disguise risk types can be quickly realized, the detection efficiency of the certificate disguise detection result is improved, and the emergency response efficiency for various certificate disguise risk types is improved.

[0121] In an example embodiment of the present disclosure, the determination of at least one certificate disguise risk type corresponding to the to-be-detected certificate image can be realized by the steps in FIG. 6. Referring to FIG. 6, the specific process can include: step S610, extracting certificate identification information corresponding to the to-be-detected certificate image, and determining a first certificate type corresponding to the to-be-detected certificate image through the certificate identification information; and step S620, determining at least one certificate disguise risk type corresponding to the to-be-detected certificate image according to the first certificate type.

[0122] The certificate identification information refers to identification information for identifying the certificate type in the to-be-detected certificate image. For example, the certificate identification information can be a certificate number coding mode in the to-be-detected certificate image, or a certificate style in the to-be-detected certificate image. The type of the certificate identification information is not specially limited in the present example embodiment.

[0123] The first certificate type corresponding to the to-be-detected certificate image can be determined through the certificate identification information. For example, the function type of the certificate in the to-be-detected certificate image can be determined through the certificate identification information, such as a personal identity certificate, a passport certificate, and a driver's license certificate. The nationality category of the certificate in the to-be-detected certificate image can also be determined through the certificate identification information, such as an A country certificate and a B country certificate.

[0124] The certificate forgery risk types corresponding to different certificate types can be pre-configured. For example, for the personal identification certificate of country A, according to historical interception information, it can be determined that the certificate forgery risk of the personal identification certificate type of country A is mostly handwriting tampering and pasting foreign matter covering and shielding tampering. Therefore, for the first certificate type corresponding to the to-be-detected certificate image, the personal identification certificate of country A, the certificate forgery risk types that need to be detected and recognized for the to-be-detected certificate image can be configured as handwriting tampering and pasting foreign matter covering and shielding tampering. Of course, this is only an illustrative example and should not impose any special limitations on the present example embodiment.

[0125] By detecting the first certificate type corresponding to the to-be-detected certificate image, the certificate forgery risk type corresponding to the to-be-detected certificate image is automatically configured in combination with historical interception information for the first certificate type. This can effectively improve the configuration efficiency of the certificate forgery risk type. Moreover, by setting the corresponding certificate forgery risk type for different certificate types, the detection and interception efficiency of the certificate forgery risk can be improved, and the success rate of detecting the certificate forgery risk type and the accuracy of the certificate forgery risk result can be improved.

[0126] In an example embodiment of the present disclosure, the determination of the at least one certificate forgery risk type corresponding to the to-be-detected certificate image can be implemented through the steps in FIG. 7. Referring to FIG. 7, the determination can specifically include: step S710, determining a second certificate type specified by the to-be-detected certificate image; and step S720, determining the at least one certificate forgery risk type corresponding to the to-be-detected certificate image according to an interception risk event corresponding to the second certificate type.

[0127] The second certificate type refers to a pre-configured certificate type with different interception priorities. For example, the second certificate type can be a serious forgery risk certificate type, a moderate forgery risk certificate type, and a slight forgery risk certificate type, and different combinations of certificate forgery risk types can be set for different certificate types with different interception priorities. For example, for the personal identification certificate of country A and the passport certificate of country B, the probability of certificate forgery risk is relatively high. Therefore, the second certificate type of the personal identification certificate of country A and the passport certificate of country B can be set as the serious forgery risk certificate type, and the corresponding certificate forgery risk type can be configured for the serious forgery risk certificate type and the corresponding certificate function.

[0128] By setting the second certificate type, the configuration flexibility of the certificate forgery risk type corresponding to the to-be-detected certificate image can be further improved, the accuracy of the configured certificate forgery risk type can be improved, and thus the success rate of detecting the certificate forgery risk type and the accuracy of the certificate forgery risk result can be improved.

[0129] In summary, the method for detecting certificate forgery based on multi-modal features provided in the embodiments of the present disclosure includes obtaining a to-be-detected certificate image and determining multi-modal feature data corresponding to the to-be-detected certificate image, the multi-modal feature data including different types of basic certificate attribute data; determining at least one certificate forgery risk type corresponding to the to-be-detected certificate image; matching combined certificate attribute data corresponding to the certificate forgery risk type from the multi-modal feature data; and determining a certificate forgery risk detection result corresponding to the to-be-detected certificate image through the combined certificate attribute data. On the one hand, by extracting different types of multi-modal basic certificate attribute data corresponding to the to-be-detected certificate image, and by screening and combining the multi-modal basic certificate attribute data to obtain combined certificate attribute data, the detection of the certificate forgery risk type is realized. Compared with the current detection method based on a visual model, the method does not require a model iteration process. Only when a new certificate forgery risk type appears, the multi-modal basic certificate attribute data for detecting the certificate forgery risk type needs to be configured, which effectively improves the response speed, enables a quick emergency response to the new certificate forgery risk type, and improves the security of private data and user accounts. On the other hand, by configuring the data types of the combined basic certificate attribute data, the detection of the certificate forgery risk type for all types of certificate images can be completed. When a new certificate forgery risk type appears, the basic certificate attribute data for detecting the new certificate forgery risk type can be configured, which improves the flexibility of the certificate forgery risk detection system, reduces the operation difficulty of the certificate forgery risk detection, and can be compatible with most certificate detection scenarios on the market with low deployment difficulty.

[0130] Referring to FIG. 8, the structure of a device for detecting certificate forgery based on multi-modal features is shown. As shown in FIG. 8, the device for detecting certificate forgery based on multi-modal features 800 can be implemented by software, hardware, or a combination of both to become all or part of an electronic device. According to some embodiments, the device for detecting certificate forgery based on multi-modal features 800 includes a multi-modal feature extraction module 810, a risk type determination module 820, a certificate attribute data combination module 830, and a detection result output module 840. Specifically, the multi-modal feature extraction module 810 can be configured to obtain a to-be-detected certificate image and determine multi-modal feature data corresponding to the to-be-detected certificate image, the multi-modal feature data including different types of basic certificate attribute data. The risk type determination module 820 can be configured to determine at least one certificate forgery risk type corresponding to the to-be-detected certificate image. The certificate attribute data combination module 830 can be configured to match combined certificate attribute data corresponding to the certificate forgery risk type from the multi-modal feature data. The detection result output module 840 can be configured to determine a certificate forgery detection result corresponding to the to-be-detected certificate image through the combined certificate attribute data.

[0131] Further, in some embodiments, the multi-modal feature extraction module 810 is configured to: perform attribute feature extraction on the to-be-detected certificate image by each certificate attribute extraction algorithm in the preset certificate attribute extraction algorithm pool respectively, to obtain different types of basic certificate attribute data; and take each of the basic certificate attribute data as multi-modal feature data corresponding to the to-be-detected certificate image.

[0132] Further, in some embodiments, the certificate attribute extraction algorithm pool includes any one or a combination of multiple of an optical character recognition algorithm cluster, a visual and information comparison algorithm cluster, a traditional image feature processing algorithm cluster, and a feature visual representation algorithm cluster; the multi-modal feature extraction module 810 can include: an optical character recognition unit, configured to perform optical character recognition on the to-be-detected certificate image by each optical character recognition algorithm in the optical character recognition algorithm cluster respectively, to obtain at least one basic certificate attribute data containing certificate text features; and / or a visual and information comparison unit, configured to perform certificate information comparison on the to-be-detected certificate image by each information comparison algorithm in the visual and information comparison algorithm cluster respectively, to obtain at least one basic certificate attribute data containing certificate information features; and / or a traditional image feature processing unit, configured to perform image feature extraction on the to-be-detected certificate image by each image feature extraction algorithm in the traditional image feature processing algorithm cluster respectively, to obtain at least one basic certificate attribute data containing certificate image features; and / or a feature visual representation unit, configured to perform visual feature extraction on the to-be-detected certificate image by each visual feature extraction algorithm in the feature visual representation algorithm cluster respectively, to obtain at least one basic certificate attribute data containing certificate visual features.

[0133] Further, in some embodiments, the optical character recognition algorithm includes a text confidence output algorithm, and the optical character recognition unit is configured to: input the to-be-detected certificate image into the text confidence output algorithm, to perform optical character recognition on the to-be-detected certificate image by the text confidence output algorithm, to determine text lines and text single characters, and to output text line confidences corresponding to the text lines and single character confidences corresponding to the text single characters; and take the text line confidences and the single character confidences as at least one basic certificate attribute data containing certificate text features.

[0134] Further, in some embodiments, the optical character recognition algorithm includes a text box detection algorithm, and the optical character recognition unit is configured to: input the to-be-detected certificate image into the text box detection algorithm, to perform optical character recognition on the to-be-detected certificate image by the text box detection algorithm, to determine text content, and to output a text bounding box corresponding to the text content; and take the text bounding box as at least one basic certificate attribute data containing certificate text features.

[0135] Further, in some embodiments, the optical character recognition algorithm includes a text line space detection algorithm, and the optical character recognition unit is configured to: input the to-be-detected certificate image into the text line space detection algorithm, perform optical character recognition on the to-be-detected certificate image by the text line space detection algorithm, determine text content and a text boundary box corresponding to the text content, and determine a number of spaces between the text boundary boxes; and take the number of spaces as at least one basic certificate attribute data containing a certificate text feature.

[0136] Further, in some embodiments, the optical character recognition algorithm includes a text self-checking algorithm, and the optical character recognition unit is configured to: input the to-be-detected certificate image into the text self-checking algorithm, perform optical character recognition on the to-be-detected certificate image by the text self-checking algorithm, determine at least two certificate text information, and perform information self-checking on the certificate text information to obtain a first self-checking result; and take the first self-checking result as at least one basic certificate attribute data containing a certificate text feature.

[0137] Further, in some embodiments, the optical character recognition algorithm includes a text self-checking algorithm, and the optical character recognition unit is configured to: input the to-be-detected certificate image into the text self-checking algorithm, perform optical character recognition on the to-be-detected certificate image by the text self-checking algorithm, determine at least two certificate text information, and perform information self-checking on the certificate text information to obtain a first self-checking result; and take the first self-checking result as at least one basic certificate attribute data containing a certificate text feature.

[0138] Further, in some embodiments, the optical character recognition algorithm includes a text self-checking algorithm, and the optical character recognition unit is configured to: input the to-be-detected certificate image into the text self-checking algorithm, perform optical character recognition on the to-be-detected certificate image by the text self-checking algorithm, determine at least two certificate text information, and perform information self-checking on the certificate text information to obtain a first self-checking result; and take the first self-checking result as at least one basic certificate attribute data containing a certificate text feature.

[0139] Further, in some embodiments, the information comparison algorithm includes a face comparison algorithm, and the visual and information comparison unit is configured to: input the to-be-detected certificate image into the face comparison algorithm, to identify a target face image in the to-be-detected certificate image by the face comparison algorithm, and perform pixel-level alignment comparison between the target face image and the collected face image to determine a face recognition result; and take the face recognition result as at least one basic certificate attribute data containing certificate information features.

[0140] Further, in some embodiments, the information comparison algorithm includes a gender comparison algorithm, and the visual and information comparison unit is configured to: input the to-be-detected certificate image into the gender comparison algorithm, to determine detected gender information corresponding to a face image in the to-be-detected certificate image by the gender comparison algorithm, and extract recorded gender information in the to-be-detected certificate image, and determine a gender comparison result according to the detected gender information and the recorded gender information; and take the gender comparison result as at least one basic certificate attribute data containing certificate information features.

[0141] Further, in some embodiments, the information comparison algorithm includes an age comparison algorithm, and the visual and information comparison unit is configured to: input the to-be-detected certificate image into the age comparison algorithm, to determine detected age information corresponding to a face image in the to-be-detected certificate image by the age comparison algorithm, and extract recorded age information in the to-be-detected certificate image, and determine an age comparison result according to the detected age information and the recorded age information; and take the age comparison result as at least one basic certificate attribute data containing certificate information features.

[0142] Further, in some embodiments, the image feature extraction algorithm includes an edge detection algorithm, and the traditional image feature processing unit is configured to: input the to-be-detected certificate image into the edge detection algorithm, to determine all edge lines in the to-be-detected certificate image by the edge detection algorithm; and take the edge lines as at least one basic certificate attribute data containing certificate image features.

[0143] Further, in some embodiments, the image feature extraction algorithm includes a local image feature classification algorithm, and the traditional image feature processing unit is configured to: input the to-be-detected certificate image into the local image feature classification algorithm, to extract image features corresponding to the to-be-detected certificate image by the local image feature classification algorithm, and classify local region features in the image features to determine a foreign matter and occlusion detection result in the to-be-detected certificate image; and take the foreign matter and occlusion detection result as at least one basic certificate attribute data containing certificate image features.

[0144] Further, in some embodiments, the image feature extraction algorithm includes a background color feature detection algorithm, and the conventional image feature processing unit is configured to: input the to-be-detected certificate image into the background color feature detection algorithm, perform pixel color statistics on the to-be-detected certificate image by the background color feature detection algorithm, obtain a color histogram, and determine the background color of each region of interest in the to-be-detected certificate image by the color histogram; and take the background color of the region of interest as at least one basic certificate attribute data containing certificate image features.

[0145] Further, in some embodiments, the visual feature extraction algorithm includes an image style consistency detection algorithm, and the feature visual representation unit is configured to: input the to-be-detected certificate image into the image style consistency detection algorithm, identify at least two local image style features of each region of interest in the to-be-detected certificate image by the image style consistency detection algorithm, and determine the image style consistency between the local image style features belonging to the same region of interest; and take the image style consistency as at least one basic certificate attribute data containing certificate visual features.

[0146] Further, in some embodiments, the visual feature extraction algorithm includes a background extraction algorithm, and the feature visual representation unit is configured to: input the to-be-detected certificate image into the background extraction algorithm, extract background similar features in the to-be-detected certificate image by the background extraction algorithm, and compare the background similar features with reference background features corresponding to the certificate type of the to-be-detected certificate image to determine a background comparison result; and take the background comparison result as at least one basic certificate attribute data containing certificate visual features.

[0147] Further, in some embodiments, the visual feature extraction algorithm includes a foreground extraction algorithm, and the feature visual representation unit is configured to: input the to-be-detected certificate image into the foreground extraction algorithm, extract foreground similar features in the to-be-detected certificate image by the foreground extraction algorithm, and compare the foreground similar features with reference foreground features corresponding to the certificate type of the to-be-detected certificate image to determine a foreground comparison result; and take the foreground comparison result as at least one basic certificate attribute data containing certificate visual features.

[0148] Further, in some embodiments, the detection result output module 840 is configured to: obtain certificate disguise interception parameters corresponding to the to-be-verified certificate attributes; and determine a certificate disguise detection result corresponding to the to-be-detected certificate image based on a matching degree between the combined certificate attribute data and the certificate disguise interception parameters.

[0149] Further, in some embodiments, the risk type determination module 820 is configured to: extract certificate identification information corresponding to the to-be-detected certificate image, and determine a first certificate type corresponding to the to-be-detected certificate image through the certificate identification information; and determine at least one certificate forgery risk type corresponding to the to-be-detected certificate image according to the first certificate type.

[0150] Further, in some embodiments, the risk type determination module 820 is configured to: determine a second certificate type specified by the to-be-detected certificate image; and determine at least one certificate forgery risk type corresponding to the to-be-detected certificate image according to an interception risk event corresponding to the second certificate type.

[0151] Further, in some embodiments, the certificate attribute data combination module 830 is configured to: determine at least one to-be-verified certificate attribute corresponding to the certificate forgery risk type; and match combined certificate attribute data corresponding to the certificate forgery risk type from the multi-modal feature data based on the to-be-verified certificate attribute.

[0152] The device embodiments described above correspond to the method embodiments, and specific descriptions can be referred to the descriptions of the method embodiments, which will not be repeated here. The device embodiments are based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. Specific descriptions can be referred to the corresponding method embodiments.

[0153] The disclosure also provides a computer storage medium, which can store a plurality of instructions. The instructions are suitable for being loaded and executed by a processor to implement the method in the embodiments of the disclosure. The specific implementation process can be referred to the specific descriptions of the embodiments of the disclosure, which will not be repeated here.

[0154] The disclosure also provides a computer program product, which stores at least one instruction. The at least one instruction is loaded and executed by the processor to implement the method in the embodiments of the disclosure. The specific implementation process can be referred to the specific descriptions of the embodiments of the disclosure, which will not be repeated here.

[0155] The disclosure also provides a structure diagram of an electronic device shown in FIG. 9. As shown in FIG. 9, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and can also include other hardware required by a business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the voice activity detection method described above.

[0156] Of course, in addition to the software implementation, the present disclosure does not exclude other implementations, such as a logic device or a combination of software and hardware, and so on, that is, the execution subject of the following processing flow is not limited to each logic unit, and can also be hardware or a logic device.

[0157] For a technical improvement, it can be obvious whether the improvement is in hardware (e.g., improvement of circuit structures of diodes, transistors, switches, etc.) or in software (e.g., improvement of method flow). However, with the development of technology, many improvements of method flow today can be considered as direct improvements of hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement of method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A designer programs a digital system "integrated" on a PLD by himself / herself, without having to ask a chip manufacturer to design and manufacture a special integrated circuit chip. Moreover, instead of manually manufacturing an integrated circuit chip, such programming is now mostly implemented by "logic compiler" software, which is similar to a software compiler used when a program is developed and written, and the original code before compilation also needs to be written in a specific programming language, which is called a hardware description language (HDL), and there are many kinds of HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It should be clear to those skilled in the art that only a little logical programming of the method flow in the above-mentioned hardware description languages and programming into an integrated circuit can easily obtain a hardware circuit that implements the logical method flow.

[0158] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.

[0159] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0160] For the sake of description, the above apparatuses are described in functional division and are described respectively. Of course, the functions of the units can be implemented in the same or multiple software and / or hardware when implementing the present disclosure.

[0161] Those skilled in the art will understand that the embodiments of the present disclosure can be provided as a method, a system or a computer program product. Therefore, the present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0163] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks.

[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0165] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0166] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer readable media.

[0167] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0168] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0169] Those skilled in the art will appreciate that embodiments of the present disclosure can be provided as a method, system or computer program product. Accordingly, the present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0170] The present disclosure can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The present disclosure can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including storage devices.

[0171] The various embodiments in the present disclosure are described in a progressive manner, and the same or similar parts among the various embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the system embodiments are described simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments.

[0172] The above only describes the embodiments of the present disclosure and is not intended to limit the present disclosure. The present disclosure can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present disclosure shall be included in the scope of the claims of the present disclosure.

Claims

1. A multi-modal feature-based certificate forgery detection method, the method comprising: obtaining a to-be-detected certificate image, and determining multi-modal feature data corresponding to the to-be-detected certificate image, the multi-modal feature data comprising different types of basic certificate attribute data; determining at least one certificate forgery risk type corresponding to the to-be-detected certificate image; matching, from the multi-modal feature data, combined certificate attribute data corresponding to the certificate forgery risk type; determining a certificate forgery risk detection result corresponding to the to-be-detected certificate image by means of the combined certificate attribute data.

2. The multi-modal feature-based certificate forgery detection method according to claim 1, wherein the multi-modal feature data corresponding to the to-be-detected certificate image is determined by: respectively extracting attribute features of the to-be-detected certificate image by means of each certificate attribute extraction algorithm in a preset certificate attribute extraction algorithm pool, to obtain different types of basic certificate attribute data; and regarding each of the basic certificate attribute data as the multi-modal feature data corresponding to the to-be-detected certificate image.

3. The multi-modal feature-based certificate forgery detection method according to claim 2, wherein the certificate attribute extraction algorithm pool comprises any one or a combination of multiple of an optical character recognition algorithm cluster, a visual and information comparison algorithm cluster, a traditional image feature processing algorithm cluster, and a feature visual representation algorithm cluster; the attribute features of the to-be-detected certificate image are respectively extracted by means of each certificate attribute extraction algorithm in the preset certificate attribute extraction algorithm pool to obtain different types of basic certificate attribute data, comprising: respectively performing optical character recognition on the to-be-detected certificate image by means of each optical character recognition algorithm in the optical character recognition algorithm cluster, to obtain at least one basic certificate attribute data comprising certificate text features; and / or respectively performing certificate information comparison on the to-be-detected certificate image by means of each information comparison algorithm in the visual and information comparison algorithm cluster, to obtain at least one basic certificate attribute data comprising certificate information features; and / or respectively extracting image features of the to-be-detected certificate image by means of each image feature extraction algorithm in the traditional image feature processing algorithm cluster, to obtain at least one basic certificate attribute data comprising certificate image features; and / or respectively extracting visual features of the to-be-detected certificate image by means of each visual feature extraction algorithm in the feature visual representation algorithm cluster, to obtain at least one basic certificate attribute data comprising certificate visual features.

4. The multi-modal feature-based certificate forgery detection method according to claim 3, wherein the optical character recognition algorithm comprises a text confidence output algorithm, and the optical character recognition on the to-be-detected certificate image by means of each optical character recognition algorithm in the optical character recognition algorithm cluster to obtain at least one basic certificate attribute data comprising certificate text features comprises: inputting the to-be-detected certificate image into the text confidence output algorithm to perform optical character recognition on the to-be-detected certificate image by the text confidence output algorithm, determining text lines and text single characters, and outputting text line confidences corresponding to the text lines and single character confidences corresponding to the text single characters; taking the text line confidences and the single character confidences as at least one basic certificate attribute data containing certificate text features.

5. The certificate forgery detection method based on multi-modal features according to claim 3, the optical character recognition type algorithm comprising a text box detection algorithm, and the performing optical character recognition on the to-be-detected certificate image by each optical character recognition type algorithm in the optical character recognition algorithm cluster to obtain at least one basic certificate attribute data containing certificate text features comprising: inputting the to-be-detected certificate image into the text box detection algorithm to perform optical character recognition on the to-be-detected certificate image by the text box detection algorithm, determining text content, and outputting a text bounding box corresponding to the text content; taking the text bounding box as at least one basic certificate attribute data containing certificate text features.

6. The certificate forgery detection method based on multi-modal features according to claim 3, the optical character recognition type algorithm comprising a text line space detection algorithm, and the performing optical character recognition on the to-be-detected certificate image by each optical character recognition type algorithm in the optical character recognition algorithm cluster to obtain at least one basic certificate attribute data containing certificate text features comprising: inputting the to-be-detected certificate image into the text line space detection algorithm to perform optical character recognition on the to-be-detected certificate image by the text line space detection algorithm, determining text content and a text bounding box corresponding to the text content, and determining a number of spaces between the text bounding boxes; taking the number of spaces as at least one basic certificate attribute data containing certificate text features.

7. The certificate forgery detection method based on multi-modal features according to claim 3, the optical character recognition type algorithm comprising a text self-checking algorithm, and the performing optical character recognition on the to-be-detected certificate image by each optical character recognition type algorithm in the optical character recognition algorithm cluster to obtain at least one basic certificate attribute data containing certificate text features comprising: inputting the to-be-detected certificate image into the text self-checking algorithm to perform optical character recognition on the to-be-detected certificate image by the text self-checking algorithm, determining at least two certificate text information, and performing information self-checking on the certificate text information to obtain a first self-checking result; taking the first self-checking result as at least one basic certificate attribute data containing certificate text features.

8. The multi-modal feature-based certificate forgery detection method of claim 3, wherein the OCRE-like algorithm comprises an interested text verification algorithm, and the performing optical character recognition on the to-be-detected certificate image by each OCRE-like algorithm in the OCRE algorithm cluster to obtain at least one basic certificate attribute data comprising certificate text features comprises: inputting the to-be-detected certificate image into the interested text verification algorithm to determine an interested visual reading area of a certificate type corresponding to the to-be-detected certificate image by the interested text verification algorithm, performing optical character recognition on the interested visual reading area to determine at least two interested certificate information, and performing information self-verification on the interested certificate information to obtain a second self-verification result; and taking the second self-verification result as the at least one basic certificate attribute data comprising certificate text features.

9. The multi-modal feature-based certificate forgery detection method of claim 3, wherein the OCRE-like algorithm comprises a single word coordinate detection algorithm, and the performing optical character recognition on the to-be-detected certificate image by each OCRE-like algorithm in the OCRE algorithm cluster to obtain at least one basic certificate attribute data comprising certificate text features comprises: inputting the to-be-detected certificate image into the single word coordinate detection algorithm to perform optical character recognition on the to-be-detected certificate image by the single word coordinate detection algorithm, determine a single word bounding box corresponding to each text character, and determine a position coordinate corresponding to each single word bounding box; and taking the position coordinate as the at least one basic certificate attribute data comprising certificate text features.

10. The multi-modal feature-based certificate forgery detection method of claim 3, wherein the information comparison algorithm comprises a face comparison algorithm, and the performing certificate information comparison on the to-be-detected certificate image by each information comparison algorithm in the visual and information comparison algorithm cluster to obtain at least one basic certificate attribute data comprising certificate information features comprises: inputting the to-be-detected certificate image into the face comparison algorithm to identify a target face image in the to-be-detected certificate image by the face comparison algorithm, and perform pixel-level alignment comparison between the target face image and a collected face image to determine a face recognition result; and taking the face recognition result as the at least one basic certificate attribute data comprising certificate information features.

11. The multi-modal feature-based certificate forgery detection method of claim 3, wherein the information comparison algorithm comprises a gender comparison algorithm, and the performing certificate information comparison on the to-be-detected certificate image by each information comparison algorithm in the visual and information comparison algorithm cluster to obtain at least one basic certificate attribute data comprising certificate information features comprises: inputting the to-be-detected certificate image into the gender comparison algorithm to determine a gender of a person in the to-be-detected certificate image by the gender comparison algorithm; and taking the gender as the at least one basic certificate attribute data comprising certificate information features. ​ ​ ​ ​ ​ ​ inputting the to-be-detected certificate image into the gender comparison algorithm to determine detection gender information corresponding to a face image in the to-be-detected certificate image by the gender comparison algorithm, extracting record gender information in the to-be-detected certificate image, and determining a gender comparison result according to the detection gender information and the record gender information; taking the gender comparison result as at least one basic certificate attribute data containing certificate information features.

12. The certificate disguise detection method based on multi-modal features according to claim 3, the information comparison algorithm comprises an age comparison algorithm, each information comparison algorithm in the visual and information comparison algorithm cluster respectively performs certificate information comparison on the to-be-detected certificate image to obtain at least one basic certificate attribute data containing certificate information features, comprising: inputting the to-be-detected certificate image into the age comparison algorithm to determine detection age information corresponding to a face image in the to-be-detected certificate image by the age comparison algorithm, extracting record age information in the to-be-detected certificate image, and determining an age comparison result according to the detection age information and the record age information; taking the age comparison result as at least one basic certificate attribute data containing certificate information features.

13. The certificate disguise detection method based on multi-modal features according to claim 3, the image feature extraction algorithm comprises an edge detection algorithm, each image feature extraction algorithm in the traditional image feature processing algorithm cluster respectively performs image feature extraction on the to-be-detected certificate image to obtain at least one basic certificate attribute data containing certificate image features, comprising: inputting the to-be-detected certificate image into the edge detection algorithm to determine all edge lines in the to-be-detected certificate image by the edge detection algorithm; taking the edge lines as at least one basic certificate attribute data containing certificate image features.

14. The certificate disguise detection method based on multi-modal features according to claim 3, the image feature extraction algorithm comprises a local image feature classification algorithm, each image feature extraction algorithm in the traditional image feature processing algorithm cluster respectively performs image feature extraction on the to-be-detected certificate image to obtain at least one basic certificate attribute data containing certificate image features, comprising: inputting the to-be-detected certificate image into the local image feature classification algorithm to extract image features corresponding to the to-be-detected certificate image by the local image feature classification algorithm, and classify local region features in the image features to determine a foreign matter and occlusion detection result in the to-be-detected certificate image; taking the foreign matter and occlusion detection result as at least one basic certificate attribute data containing certificate image features.

15. The multi-modal feature based certificate forgery detection method according to claim 3, wherein the image feature extraction algorithm comprises a background color feature detection algorithm, and the image feature extraction by each of the image feature extraction algorithms in the cluster of traditional image feature processing algorithms on the image to be detected to obtain at least one basic certificate attribute data comprising certificate image features comprises: inputting the image to be detected into the background color feature detection algorithm to perform pixel color statistics on the image to be detected by the background color feature detection algorithm to obtain a color histogram, and determining the background color of each region of interest in the image to be detected by the color histogram; and taking the background color of the region of interest as the at least one basic certificate attribute data comprising certificate image features.

16. The multi-modal feature based certificate forgery detection method according to claim 3, wherein the visual feature extraction algorithm comprises an image style consistency detection algorithm, and the visual feature extraction by each of the visual feature extraction algorithms in the cluster of feature visual representation algorithms on the image to be detected to obtain at least one basic certificate attribute data comprising certificate visual features comprises: inputting the image to be detected into the image style consistency detection algorithm to identify at least two local image style features of each region of interest in the image to be detected by the image style consistency detection algorithm, and determine the image style consistency between the local image style features belonging to the same region of interest; and taking the image style consistency as the at least one basic certificate attribute data comprising certificate visual features.

17. The multi-modal feature based certificate forgery detection method according to claim 3, wherein the visual feature extraction algorithm comprises a background extraction algorithm, and the visual feature extraction by each of the visual feature extraction algorithms in the cluster of feature visual representation algorithms on the image to be detected to obtain at least one basic certificate attribute data comprising certificate visual features comprises: inputting the image to be detected into the background extraction algorithm to extract background similar features in the image to be detected by the background extraction algorithm, and comparing the background similar features with reference background features corresponding to the certificate type of the image to be detected to determine a background comparison result; and taking the background comparison result as the at least one basic certificate attribute data comprising certificate visual features.

18. The multi-modal feature based certificate forgery detection method according to claim 3, wherein the visual feature extraction algorithm comprises a foreground extraction algorithm, and the visual feature extraction by each of the visual feature extraction algorithms in the cluster of feature visual representation algorithms on the image to be detected to obtain at least one basic certificate attribute data comprising certificate visual features comprises: ​ ​ ​ ​ ​ ​ inputting the to-be-detected certificate image into the foreground extraction algorithm to extract foreground similar features in the to-be-detected certificate image by the foreground extraction algorithm, and comparing the foreground similar features with reference foreground features corresponding to a certificate type of the to-be-detected certificate image to determine a foreground comparison result; taking the foreground comparison result as at least one basic certificate attribute data containing certificate visual features.

19. The certificate forgery detection method based on multi-modal features according to claim 1, wherein the matching of the combined certificate attribute data corresponding to the certificate forgery risk type from the multi-modal feature data comprises: determining at least one to-be-verified certificate attribute corresponding to the certificate forgery risk type; matching the combined certificate attribute data corresponding to the certificate forgery risk type from the multi-modal feature data based on the to-be-verified certificate attribute.

20. The certificate forgery detection method based on multi-modal features according to claim 19, wherein the determination of a certificate forgery detection result corresponding to the to-be-detected certificate image by the combined certificate attribute data comprises: obtaining a certificate forgery interception parameter corresponding to the to-be-verified certificate attribute; determining a certificate forgery detection result corresponding to the to-be-detected certificate image based on a matching degree between the combined certificate attribute data and the certificate forgery interception parameter.

21. The certificate forgery detection method based on multi-modal features according to claim 1, wherein the determination of at least one certificate forgery risk type corresponding to the to-be-detected certificate image comprises: extracting certificate identification information corresponding to the to-be-detected certificate image, and determining a first certificate type corresponding to the to-be-detected certificate image by the certificate identification information; determining at least one certificate forgery risk type corresponding to the to-be-detected certificate image according to the first certificate type.

22. The certificate forgery detection method based on multi-modal features according to claim 1, wherein the determination of at least one certificate forgery risk type corresponding to the to-be-detected certificate image comprises: determining a second certificate type designated for the to-be-detected certificate image; determining at least one certificate forgery risk type corresponding to the to-be-detected certificate image according to an interception risk event corresponding to the second certificate type.

23. A certificate forgery detection device based on multi-modal features, comprising: a multi-modal feature extraction module configured to obtain a to-be-detected certificate image, and determine multi-modal feature data corresponding to the to-be-detected certificate image, the multi-modal feature data comprising different types of basic certificate attribute data; a risk type determination module configured to determine at least one certificate forgery risk type corresponding to the to-be-detected certificate image; a certificate attribute data combination module configured to match combined certificate attribute data corresponding to the certificate forgery risk type from the multi-modal feature data; a detection result output module configured to determine a certificate forgery detection result corresponding to the to-be-detected certificate image by the combined certificate attribute data.

24. A storage medium having stored thereon a computer program, wherein, The computer program is executed by a processor to implement the steps of the method of any one of claims 1-22. The computer program is executed by a processor to implement the steps of the method of any one of claims 1-22.

25. An electronic device, comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded and executed by the processor to perform the steps of the method according to any one of claims 1 to 22.

Images