Voucher auditing method and device

Abstract

One or more embodiments of the invention disclose a voucher auditing method and device. The method comprises the following steps: firstly, receiving an auditing request of a target certificate, the auditing request comprising a certificate image of the target certificate, obtaining key information contained in the certificate image, and obtaining reference information which corresponds to the key information and is used for auditing the target certificate; then visual features corresponding to the key information of the voucher image and reference features corresponding to the reference information are extracted based on a pre-trained content comparison model, finally the similarity between the visual features and the reference features is calculated, and an audit result of the target voucher is determined based on the calculated similarity.

Description

Technical Field

[0001] This document relates to the field of image recognition technology, and in particular to a method and device for verifying credentials. Background Art

[0002] At present, with the development of image recognition technology, the application scope of image recognition is becoming more and more extensive. At the same time, as people pay more and more attention to their own privacy data, the review of credential information is becoming more and more extensive. For example, in business scenarios such as platform user account opening, merchant settlement, and account unblocking, users are usually required to provide relevant credential information to prove the user's relevant information, so as to realize the review of the credential information provided by the user, and determine whether the user can open an account, whether the merchant is allowed to settle in, and whether the account can be unblocked based on the review results. To this end, it is necessary to provide an effective credential review method that takes into account both the review accuracy and the review efficiency. Summary of the invention

[0003] On the one hand, one or more embodiments of the present specification provide a credential review method, comprising: receiving a review request for a target credential, the review request including a credential image of the target credential; obtaining key information contained in the credential image, and obtaining benchmark information corresponding to the key information for reviewing the target credential; extracting visual features corresponding to the key information of the credential image and benchmark features corresponding to the benchmark information based on a pre-trained content comparison model, wherein the content comparison model is a model obtained by model training based on historical images, historical benchmark information corresponding to the key information of the historical images, and a preset contrastive learning loss function; calculating the similarity between the visual features and the benchmark features, and determining the review result of the target credential based on the calculated similarity.

[0004] On the other hand, one or more embodiments of the present specification provide a credential auditing device, comprising: an audit request receiving module, receiving an audit request for a target credential, the audit request including a credential image of the target credential; an information acquisition module, acquiring key information contained in the credential image, and acquiring benchmark information corresponding to the key information for auditing the target credential; a feature extraction module, extracting visual features corresponding to the key information of the credential image and benchmark features corresponding to the benchmark information based on a pre-trained content comparison model, wherein the content comparison model is a model obtained by model training based on historical images, historical benchmark information corresponding to the key information of the historical images, and a preset contrastive learning loss function; an audit module, calculating the similarity between the visual features and the benchmark features, and determining the audit result of the target credential based on the calculated similarity.

[0005] On the other hand, one or more embodiments of the present specification provide an electronic device, comprising: a processor; and a memory arranged to store computer-executable instructions, which, when the executable instructions are executed, enable the processor to: receive a review request for a target credential, the review request including a credential image of the target credential; obtain key information contained in the credential image, and obtain benchmark information corresponding to the key information for reviewing the target credential; extract visual features corresponding to the key information of the credential image and benchmark features corresponding to the benchmark information based on a pre-trained content comparison model, the content comparison model being a model obtained by model training based on historical images, historical benchmark information corresponding to the key information of the historical images, and a preset contrastive learning loss function; calculate the similarity between the visual features and the benchmark features, and determine the review result of the target credential based on the calculated similarity.

[0006] On the other hand, one or more embodiments of the present specification provide a storage medium for storing a computer program, which can be executed by a processor to implement the following process: receiving a review request for a target credential, the review request including a credential image of the target credential; obtaining key information contained in the credential image, and obtaining benchmark information corresponding to the key information for reviewing the target credential; extracting visual features corresponding to the key information of the credential image and benchmark features corresponding to the benchmark information based on a pre-trained content comparison model, wherein the content comparison model is a model obtained by model training based on historical images, historical benchmark information corresponding to the key information of the historical images, and a preset contrastive learning loss function; calculating the similarity between the visual features and the benchmark features, and determining the review result of the target credential based on the calculated similarity.

[0007] On the other hand, one or more embodiments of the present specification also provide a computer program product, including a computer program, which implements the following process when executed by a processor: receiving a review request for a target credential, the review request including a credential image of the target credential; obtaining key information contained in the credential image, and obtaining benchmark information corresponding to the key information for reviewing the target credential; extracting visual features corresponding to the key information of the credential image and benchmark features corresponding to the benchmark information based on a pre-trained content comparison model, wherein the content comparison model is a model obtained by model training based on historical images, historical benchmark information corresponding to the key information of the historical images, and a preset contrastive learning loss function; calculating the similarity between the visual features and the benchmark features, and determining the review result of the target credential based on the calculated similarity. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0009] Figure 1 is a schematic flow chart of a method for reviewing a voucher according to an embodiment of the present specification; Figure 2 It is a schematic diagram of the principle of credential information review in the relevant technology; Figure 3 It is a schematic diagram of the implementation principle of the method for reviewing a voucher according to an embodiment of this specification; Figure 4 is a schematic block diagram of a credential review device according to an embodiment of the present specification; Figure 5 It is a schematic block diagram of an electronic device according to an embodiment of the present specification. DETAILED DESCRIPTION

[0010] One or more embodiments of the present specification provide a method and device for reviewing a credential.

[0011] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0012] like Figure 1 As shown, an embodiment of this specification provides a method for reviewing credentials, and the execution subject of the method can be a terminal device or a server, wherein the terminal device can be a certain terminal device such as a mobile phone, a tablet computer, or a computer device such as a laptop or a desktop computer, or an IoT device (specifically such as a smart watch, a car device, etc.). The server can be an independent server, or a server cluster composed of multiple servers, etc. The server can be a background server in the financial field or the online shopping field, or a background server of an application, etc. In this embodiment, the server is used as an example for detailed description. For the execution process of the terminal device, please refer to the following related content, which will not be repeated here. The method can specifically include the following steps: In step S102, a review request of a target credential is received, wherein the review request includes a credential image of the target credential.

[0013] In various business scenarios such as payment platform user account opening, merchant settlement, account unblocking, business access, etc., users are often required to submit some target credentials (such as qualification certificates, passports, business licenses, etc.) to prove the user's relevant information.

[0014] like Figure 2 As shown, the OCR (Optical Character Recognition) model and the information extraction model can be used to review the credential information. Specifically, for the input credential image, the text lines in the credential image are first extracted through the text detection module of the OCR model, and then the text content of each text line is recognized by the text recognition module, and then the key information fields in the text content are extracted through the information extraction model. Finally, the extracted key information fields are compared with the information reserved by the user. If the information is consistent, the intelligent review is passed, otherwise the review fails. However, it is impossible to achieve a high review accuracy and efficiency only through the OCR model and the information extraction model.

[0015] In order to verify whether the user's real information is consistent with the information on the target credential, the embodiment of this specification further provides a credential review method.

[0016] In practice, the types of target credentials may include, according to different business scenarios: credentials used for account opening, credentials required for merchant entry, credentials required for account unlocking, credentials required for financial business review, or credentials required for accessing new businesses, etc. The target credentials may specifically be passports, business licenses, teacher qualification certificates, and other documents.

[0017] If the target credential is a qualification certificate required for account unlocking, the credential image of the target credential is an image of the qualification certificate, for example, it can be an electronic scan of the qualification certificate, a photo taken based on the qualification certificate, etc.

[0018] In step S104, key information included in the voucher image is obtained, and reference information corresponding to the key information and used for reviewing the target voucher is obtained.

[0019] The key information contained in the credential image refers to the key information used to verify whether the target credential is legal. The key information may be one or more of the account identification information, user identity information, key date information, and account key address information in the credential image. The benchmark information refers to the information containing the key information of the target credential stored before the current business occurs, such as the real account identification information, real user identity information, real key date information, and real key address information of the account. Taking the account unrestricted business as an example, the target credential may be the qualification certificate A provided for unrestricted business, then the credential image of the target credential is the qualification certificate photo A1 corresponding to the qualification certificate A, then the key information A2 contained in the qualification certificate photo A1 may be the user name, date of birth, address information, etc. The benchmark information is the key information B2 related to the qualification certificate B stored on the unrestricted business platform before the unrestricted business occurs (such as when the user opens an account). Among them, the qualification certificate A is the target credential to be reviewed, the key information A2 is the information to be reviewed, and the qualification certificate B is the real certificate used for reference during the review, and the benchmark information B2 is the information used for reference during the review.

[0020] In implementation, the reference information may be text information, image information, or may include both text information and image information.

[0021] According to different types of target credentials and business requirements, the key information in the credential image may include one or more. If it is necessary to obtain multiple key information contained in the credential image according to the current business requirements, it is necessary to obtain the benchmark information corresponding to each key information for reviewing the target credential.

[0022] In step S106 , visual features corresponding to the key information of the credential image and reference features corresponding to the reference information are respectively extracted based on the pre-trained content comparison model.

[0023] Among them, the content comparison model is a model obtained by model training based on historical images, historical benchmark information corresponding to key information of historical images, and a preset contrastive learning loss function.

[0024] In implementation, the credential image and benchmark information of the target credential can be respectively input into the content comparison model at the same time. Based on the credential image and its key information, the visual features corresponding to the key information are extracted through the content comparison model; based on the benchmark information of the target credential, the benchmark features corresponding to the benchmark information are extracted through the content comparison model.

[0025] In implementation, when extracting visual features, the key image fields can be extracted by performing region detection processing on the credential image (based on the region detection module), and the extracted key image fields are encoded (based on the region encoding module) to obtain the visual features corresponding to the key information. The key image fields can also be extracted by performing text detection and text recognition processing on the credential image (based on the OCR model), and the extracted key image fields are encoded to obtain the visual features corresponding to the key information. The OCR model is used to identify all text content from the image, and usually includes two parts: text detection (extracting text areas in the image) processing and text recognition (identifying the character content in the text area of ​​the image).

[0026] The preset contrastive learning loss function InfoNCE (Noise Constrastive Estimation) is used to compare the visual features of each voucher image with its corresponding benchmark features, to advance the distance between the visual features in each voucher image and its matching benchmark features, and to increase the distance between the visual features in each voucher image and its mismatched benchmark features.

[0027] In step S108, the similarity between the visual feature and the reference feature is calculated, and the audit result of the target credential is determined based on the calculated similarity.

[0028] In implementation, cosine similarity, Euclidean distance, or dot product calculation can be used to obtain the similarity between the visual feature and the reference feature. If the similarity between the two is greater than or equal to a preset similarity threshold, the audit result of the target credential is determined to be passed. If the similarity is less than the preset similarity threshold, the audit result of the target credential is determined to be failed.

[0029] The embodiment of the present specification provides a method for auditing a voucher. The method first receives an audit request for a target voucher, wherein the audit request includes a voucher image of the target voucher, obtains key information contained in the voucher image, and obtains reference information corresponding to the key information for auditing the target voucher. Then, based on a pre-trained content comparison model, visual features corresponding to the key information of the voucher image and reference features corresponding to the reference information are extracted respectively. Finally, the similarity between the visual features and the reference features is calculated, and the audit result of the target voucher is determined based on the calculated similarity. In this way, the visual features corresponding to the key information of the voucher image and the reference features corresponding to the reference information are extracted respectively through the pre-trained content comparison model, and the similarity between the visual features and the reference features is calculated. The intelligent audit task of the voucher is replaced from the complex audit process of file detection, text recognition, information extraction, and information comparison to the comparison process of the visual features and the reference features of the voucher image, which greatly reduces the process and difficulty of the intelligent audit task, thereby effectively improving the efficiency and accuracy of the voucher image audit. In addition, the method in the embodiment of the present specification only extracts features for the key information in the voucher image, and does not need to identify and extract features for all information in the voucher image, which can effectively improve the efficiency of the voucher image audit. Moreover, only one content comparison model is needed to realize the comparison between visual features and reference features. The number of models adopted is small, which can further reduce the error of data transmission and improve the audit efficiency and accuracy of credentials.

[0030] In the embodiment of this specification, the reference information in the above step S104 can be text information or a reference image. If the reference information is text information, the processing of the above step S106 can be various. An optional processing method is provided below, and the specific processing can refer to the following steps A2-A8.

[0031] In step A2, based on the key information of the credential image, region detection is performed on the credential image by using the first sub-model of the preset image-text cross-modal model to obtain the key image field of the credential image.

[0032] If the reference information is text information, the content comparison model can adopt a cross-modal image-text model, which includes a first sub-model and a second sub-model. The first sub-model is used to perform region detection and encoding processing on the credential image, and the second sub-model is used to perform word segmentation and encoding processing on the text information.

[0033] In implementation, the key image fields of the voucher image can be extracted through the region detection module based on the key information, such as the account name, date of birth, address information, identity information, etc. in the medical insurance family account image. Based on the key information of the voucher image, the first sub-model is used to perform regional detection on the voucher image instead of performing full detection on the voucher image, which can greatly reduce the scope of text detection, thereby effectively improving the efficiency of voucher review.

[0034] In step A4, the key image field is encoded by the first sub-model of the preset image-text cross-modal model to obtain visual features corresponding to the key information of the voucher image.

[0035] In implementation, each image region (key image field region) can be encoded through a region encoding module to extract corresponding visual features. The region encoding module in the first sub-model can use a ViT neural network (vison transformer) as the backbone network structure, or a CNN (Convolutional Neural Network) network.

[0036] In step A6, the text information is segmented using the second sub-model of the preset text-image cross-modal model to obtain text information after the segmentation process.

[0037] In implementation, the text information as the reference information can be input into the second sub-model of the graphic-text cross-modal model, and the second sub-model can be used to perform word segmentation processing (i.e., tokenization processing) on ​​the text information. When the second sub-model performs word segmentation processing on the text information, it can be word segmentation processing at the character granularity (i.e., splitting each character as the smallest unit) or word segmentation processing (i.e., splitting each word as the smallest unit). Accordingly, the text information after word segmentation processing can be text information at the character granularity or text information at the word granularity.

[0038] In step A8, the text information after word segmentation is encoded by the second sub-model of the preset graphic-text cross-modal model to obtain text character features corresponding to the text information, and the text character features are used as reference features corresponding to the reference information.

[0039] After the text information is segmented, the second sub-model encodes the segmented text information (i.e., text token vectorization) to obtain the corresponding text character features. If there are multiple text lines, each text line is encoded to obtain the text character features of each text line.

[0040] In the embodiments of this specification, the processing of the above step A6 can be varied. An optional processing method is provided below. For details, please refer to the processing of the following step A62.

[0041] In step A62, the text information is segmented at the character granularity by using the second sub-model of the preset graphic-text cross-modal model to obtain the text information at the character granularity after the segmentation processing.

[0042] Taking Zhang San as an example, each character can be split as the smallest unit. After the word segmentation of the text information "Zhang San", it is split into text information of two character granularities, "Zhang" and "San". By performing word segmentation on the text information at the character granularity and obtaining the text information at the character granularity, the accuracy of the credential image review can be further improved.

[0043] If the reference information is a reference image, the processing of the above step S106 can be varied. An optional processing method is provided below. For details, please refer to the processing of the following steps B2-B8.

[0044] In step B2, based on the key information of the voucher image, a region detection is performed on the voucher image by using a preset image comparison model to obtain a first key image field of the voucher image.

[0045] If the reference information is a reference image, the content comparison model may adopt an image comparison model, which processes the credential image in the same manner as the reference image.

[0046] In step B4, the first key image field is encoded using an image comparison model to obtain visual features corresponding to the key information of the voucher image.

[0047] In step B6, the reference image is subjected to region detection by using an image comparison model to obtain a second key image field of the reference image.

[0048] In step B8, the second key image field is encoded using a preset image comparison model to obtain a reference feature corresponding to the reference image.

[0049] The processing of the above steps B2-B4 and the processing of steps B6-B8 can refer to the processing of the previous steps A2-A4, and will not be repeated here.

[0050] In the embodiments of this specification, the processing of the above step S108 can be varied. An optional processing method is provided below. For details, please refer to the processing of the following steps S1082-S1088.

[0051] In step S1082, based on the cosine similarity algorithm, the similarity between the visual feature corresponding to each key information of the voucher image and the reference feature is calculated.

[0052] In step S1084, for any key information, if the similarity is greater than or equal to the preset similarity threshold, the audit result of the target credential is determined to be passed.

[0053] In implementation, the preset similarity threshold may be a similarity threshold specified according to user requirements, or may be a similarity threshold determined according to the type of the target credential.

[0054] In step S1086, if the similarity is less than a preset similarity threshold, the audit result of the target credential is determined to be failed.

[0055] It should be noted that if there are multiple key information, each of the multiple key information must be consistent with its corresponding benchmark information to determine whether the target voucher has passed the review, otherwise the target voucher has failed the review.

[0056] In the embodiment of the present specification, the similarity threshold preset in the above step S1084 is determined according to the type of the target credential. Specifically, the business scenario can be determined according to the type of the target credential. In the business scenario where the risk needs to be reduced, a stricter similarity threshold can be set, and in the business scenario where the user experience needs to be improved, a looser similarity threshold can be set. For example: for the target credential related to illegal financial business, the preset similarity threshold value is higher, and for the target credential related to the new account opening business, the preset similarity threshold value is lower. The above method of determining the preset similarity threshold according to the type of the target credential can effectively improve the flexibility of the similarity threshold setting, so that the set similarity threshold can be adapted to different business scenarios, which is conducive to improving the scope of application of the credential image review method and improving the review efficiency.

[0057] In the embodiment of this specification, the training method of the content comparison model in the above step S106 can be various. An optional processing method is provided below. For details, please refer to the processing of the following steps C2-C8.

[0058] In step C2, a plurality of historical images are obtained, and key information of each historical image and historical text information corresponding to the key information of the historical image and used for reviewing the historical image are determined; In step C4, the historical image and the historical benchmark information are input into the image-text cross-modal model, and based on the key information of the historical image, the historical image is subjected to region detection by the first sub-model of the image-text cross-modal model to obtain the historical key image field of the historical image, and the historical key image field is encoded by the first sub-model to obtain the historical visual features corresponding to the key information of the historical image; In step C6, the historical text information is segmented by the second sub-model of the image-text cross-modal model to obtain the historical text information after the segmentation, and the historical text information is encoded by the second sub-model to obtain the historical text character features corresponding to the historical text information; In step C8, the first sub-model and the second sub-model are jointly trained based on the historical visual features, the historical text character features and the preset contrastive learning loss function to obtain a trained image-text cross-modal model.

[0059] In implementation, the contrastive learning loss function InfoNCE can be calculated using the following formula: , Where N is the number of samples in the batch, is the visual feature (i.e., image region feature), is the text character feature, is the similarity between visual features and text character features (dot product or cosine similarity can be used), is a temperature parameter used to control the smoothness of the distribution. During the model training process, through cross-modal contrast learning between images and texts, the model can associate the semantic information of the visual content of the image area with the text character content.

[0060] The schematic diagram of the implementation principle of the credential audit method of the embodiment of this specification can be found in Figure 3 As shown, Figure 3 In the text information is used as the reference information, Figure 3 It can be seen that the credential image of the target credential is obtained and the reference information corresponding to the key information of the credential image is determined (i.e. Figure 3 After the credential field content in the image is obtained, the credential image and its corresponding benchmark information are respectively input into the pre-trained image-text cross-modal model. The first sub-model of the image-text cross-modal model is used to perform region extraction processing (i.e., region detection processing) and region encoding processing on the credential image to obtain the visual features corresponding to the key information of the credential image. The second sub-model of the image-text cross-modal model is used to perform character tokenization (character granularity word segmentation) processing and char encoding processing (character granularity encoding processing) on ​​the benchmark information to obtain the text character features corresponding to the credential image. Finally, the visual features and text character features are compared to determine the similarity between the two.

[0061] In summary, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recorded in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing can be advantageous.

[0062] The above is a method for verifying a credential provided in one or more embodiments of this specification. Based on the same idea, one or more embodiments of this specification also provide a device for verifying a credential, such as Figure 4 shown.

[0063] The credential auditing device includes: an audit request receiving module 210, an information acquisition module 220, a feature extraction module 230 and an auditing module 240, wherein: The audit request receiving module 210 receives an audit request for a target credential, wherein the audit request includes a credential image of the target credential; The information acquisition module 220 acquires key information contained in the credential image and acquires reference information corresponding to the key information for reviewing the target credential; A feature extraction module 230 extracts visual features corresponding to the key information of the voucher image and reference features corresponding to the reference information based on a pre-trained content comparison model, wherein the content comparison model is a model obtained by model training based on the historical image, the historical reference information corresponding to the key information of the historical image, and a preset contrastive learning loss function; The audit module 240 calculates the similarity between the visual feature and the reference feature, and determines the audit result of the target credential based on the calculated similarity.

[0064] In the embodiment of this specification, if the reference information is text information, the feature extraction module 230 includes: An image region detection unit, based on the key information of the voucher image, performs region detection on the voucher image through a first sub-model of a preset image-text cross-modal model to obtain a key image field of the voucher image; An image region encoding unit encodes the key image field through the first sub-model of the preset image-text cross-modal model to obtain visual features corresponding to the key information of the voucher image; A text information word segmentation processing unit performs word segmentation processing on the text information through a second sub-model of a preset image-text cross-modal model to obtain text information after word segmentation processing; The text information encoding unit encodes the text information after word segmentation through the second sub-model of the preset image-text cross-modal model to obtain text character features corresponding to the text information, and uses the text character features as reference features corresponding to the reference information.

[0065] In the embodiment of this specification, if the reference information is a reference image, the feature extraction module 230 includes: A first key image field acquisition unit, based on the key information of the voucher image, performs region detection on the voucher image through a preset image comparison model to obtain a first key image field of the voucher image; A first key image field encoding unit, which encodes the first key image field through an image comparison model to obtain visual features corresponding to key information of the voucher image; A second key image field acquisition unit performs region detection on the reference image through an image comparison model to obtain a second key image field of the reference image; The second key image field encoding unit performs encoding processing on the second key image field through a preset image comparison model to obtain a reference feature corresponding to the reference image.

[0066] In the embodiment of this specification, the audit module 240 includes: A cosine similarity calculation unit, which calculates the similarity between the visual feature corresponding to each key information of the voucher image and the reference feature based on the cosine similarity algorithm; The judgment unit determines that, for any key information, if the similarity is greater than or equal to a preset similarity threshold, the audit result of the target credential is passed; if the similarity is less than the preset similarity threshold, the audit result of the target credential is failed.

[0067] In the embodiment of the present specification, the similarity threshold preset in the judgment unit is determined according to the type of the target credential.

[0068] In the embodiment of this specification, the credential review device further includes a model training module, and the model training module includes: A historical information acquisition unit, which acquires a plurality of historical images, determines key information of each historical image and historical text information corresponding to the key information of the historical image and used to review the historical image; The historical visual feature acquisition unit inputs the historical image and the historical benchmark information into the image-text cross-modal model, performs region detection on the historical image based on the key information of the historical image through the first sub-model of the image-text cross-modal model to obtain the historical key image field of the historical image, and performs encoding processing on the historical key image field through the first sub-model to obtain the historical visual features corresponding to the key information of the historical image; A historical text character feature acquisition unit performs word segmentation processing on the historical text information through the second sub-model of the image-text cross-modal model to obtain the historical text information after word segmentation processing, and performs encoding processing on the historical text information through the second sub-model to obtain the historical text character features corresponding to the historical text information; The training unit jointly trains the first sub-model and the second sub-model based on historical visual features, historical text character features and a preset contrastive learning loss function to obtain a trained image-text cross-modal model.

[0069] The embodiment of the present specification provides a credential auditing device, which first receives an audit request of a target credential through an audit request receiving module, wherein the audit request includes a credential image of the target credential, obtains key information contained in the credential image through an information acquisition module, and obtains reference information corresponding to the key information for auditing the target credential, and then uses a feature extraction module to extract visual features corresponding to the key information of the credential image and reference features corresponding to the reference information based on a pre-trained content comparison model, and finally calculates the similarity between the visual features and the reference features through the audit module, and determines the audit result of the target credential based on the calculated similarity. In this way, the visual features corresponding to the key information of the credential image and the reference features corresponding to the reference information are extracted respectively through the pre-trained content comparison model, and the visual features and the similarity between the reference features are calculated, and the credential intelligent audit task is replaced from a complex audit process such as file detection, text recognition, information extraction, and information comparison to a comparison process of the visual features of the credential image and the reference features, which greatly reduces the process and difficulty of the intelligent audit task, thereby effectively improving the efficiency and accuracy of the credential image audit. In addition, the method in the embodiment of this specification only extracts features from key information in the credential image, and does not need to identify and extract features from all information in the credential image, which can effectively improve the efficiency of credential image review. Moreover, only one content comparison model is needed to achieve the comparison between visual features and reference features, and the number of models used is small, which can further reduce data transmission errors and improve the review efficiency and accuracy of credential.

[0070] Those skilled in the art should understand that the above-mentioned credential auditing device can be used to implement the credential auditing method described above, and the detailed description thereof should be similar to that described in the above method part, and will not be further elaborated here to avoid tediousness.

[0071] Based on the same idea, one or more embodiments of this specification also provide an electronic device, such as Figure 5As shown. The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors 301 and memory 302, and the memory 302 may store one or more storage applications or data. Among them, the memory 302 may be a temporary storage or a permanent storage. The application stored in the memory 302 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 301 can be configured to communicate with the memory 302 to execute a series of computer executable instructions in the memory 302 on the electronic device. The electronic device may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more input and output interfaces 305, and one or more keyboards 306.

[0072] Specifically in this embodiment, the electronic device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions for the electronic device, and the one or more programs are configured to be executed by one or more processors, including the following computer executable instructions: receiving a review request for a target credential, the review request including a credential image of the target credential; Acquire key information contained in the credential image, and acquire reference information corresponding to the key information for reviewing the target credential; Based on a pre-trained content comparison model, visual features corresponding to key information of the voucher image and reference features corresponding to the reference information are respectively extracted, wherein the content comparison model is a model obtained by model training based on historical images, historical reference information corresponding to key information of the historical images, and a preset contrastive learning loss function; The similarity between the visual features and the reference features is calculated, and the audit result of the target credential is determined based on the calculated similarity.

[0073] One or more embodiments of this specification provide a storage medium, the storage medium is used to store computer executable instructions, and the executable instructions implement the following process when executed by a processor: receiving a review request for a target credential, the review request including a credential image of the target credential; Acquire key information contained in the credential image, and acquire reference information corresponding to the key information for reviewing the target credential; Based on a pre-trained content comparison model, visual features corresponding to key information of the voucher image and reference features corresponding to the reference information are respectively extracted, wherein the content comparison model is a model obtained by model training based on historical images, historical reference information corresponding to key information of the historical images, and a preset contrastive learning loss function; The similarity between the visual features and the reference features is calculated, and the audit result of the target credential is determined based on the calculated similarity.

[0074] One or more embodiments of the present specification further provide a computer program product, including a computer program, which implements the following process when executed by a processor: receiving a review request for a target credential, the review request including a credential image of the target credential; Acquire key information contained in the credential image, and acquire reference information corresponding to the key information for reviewing the target credential; Based on a pre-trained content comparison model, visual features corresponding to key information of the voucher image and reference features corresponding to the reference information are respectively extracted, wherein the content comparison model is a model obtained by model training based on historical images, historical reference information corresponding to key information of the historical images, and a preset contrastive learning loss function; The similarity between the visual features and the reference features is calculated, and the audit result of the target credential is determined based on the calculated similarity.

[0075] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0076] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0077] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0078] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0079] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0080] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system, or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] One or more embodiments of this specification are described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0082] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0083] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0084] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0085] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0086] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0087] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0088] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0089] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0090] The above description is only one or more embodiments of this specification and is not intended to limit this document. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of one or more embodiments of this specification.

Claims

1. A method for reviewing a voucher, comprising: receiving a review request for a target credential, wherein the review request includes a credential image of the target credential; Acquire key information contained in the credential image, and acquire reference information corresponding to the key information and used for reviewing the target credential; extracting visual features corresponding to the key information of the voucher image and reference features corresponding to the reference information based on a pre-trained content comparison model, wherein the content comparison model is a model obtained by model training based on historical images, historical reference information corresponding to the key information of the historical images, and a preset contrastive learning loss function; The similarity between the visual feature and the reference feature is calculated, and the audit result of the target credential is determined based on the calculated similarity.

2. According to the method of claim 1, the reference information is text information, the content comparison model is a cross-modal image-text model, and the pre-trained content comparison model extracts visual features corresponding to the key information of the voucher image and reference features corresponding to the reference information, respectively, including: Based on the key information of the credential image, performing region detection on the credential image through a first sub-model of a preset image-text cross-modal model to obtain a key image field of the credential image; The key image field is encoded by the first sub-model of the preset image-text cross-modal model to obtain visual features corresponding to the key information of the voucher image; Performing word segmentation processing on the text information through the second sub-model of the preset image-text cross-modal model to obtain text information after word segmentation processing; The text information after word segmentation is encoded by the second sub-model of the preset graphic-text cross-modal model to obtain text character features corresponding to the text information, and the text character features are used as reference features corresponding to the reference information.

3. According to the method of claim 2, the text information after word segmentation processing is text information at character granularity.

4. According to the method of claim 1, the reference information is a reference image, the content comparison model is an image comparison model, and the pre-trained content comparison model extracts visual features corresponding to the key information of the voucher image and reference features corresponding to the reference information, respectively, including: Based on the key information of the voucher image, performing region detection on the voucher image by using a preset image comparison model to obtain a first key image field of the voucher image; Encoding the first key image field by using the image comparison model to obtain visual features corresponding to the key information of the voucher image; Performing region detection on the reference image by using the image comparison model to obtain a second key image field of the reference image; The second key image field is encoded using the preset image comparison model to obtain a reference feature corresponding to the reference image.

5. The method according to claim 1, wherein the target credential comprises: The key information includes one or more of the account identification information, user identity information, key date information, and key address information of the account in the credential image, the credentials used for account opening, the credentials required for merchant settlement, the credentials required for account unlocking, the credentials required for financial business review, or the credentials required for accessing new business.

6. The method according to claim 1, calculating the similarity between the visual feature and the reference feature, and determining the audit result of the target credential based on the calculated similarity, comprising: Calculating the similarity between the visual feature corresponding to each key information of the voucher image and the reference feature based on a cosine similarity algorithm; For any key information, if the similarity is greater than or equal to a preset similarity threshold, the audit result of the target credential is determined to be passed; If the similarity is less than a preset similarity threshold, the audit result of the target credential is determined to be failed.

7. The method according to claim 6, wherein the preset similarity threshold is determined according to the type of the target credential.

8. According to the method of claim 2, the training method of the image-text cross-modal model comprises: Acquire a plurality of historical images, determine key information of each of the historical images and historical text information corresponding to the key information of the historical images and used to review the historical images; Inputting the historical image and the historical reference information into a graphic-text cross-modal model, performing region detection on the historical image through a first sub-model of the graphic-text cross-modal model based on the key information of the historical image to obtain a historical key image field of the historical image, and encoding the historical key image field through the first sub-model to obtain a historical visual feature corresponding to the key information of the historical image; Performing word segmentation processing on the historical text information through the second sub-model of the image-text cross-modal model to obtain the historical text information after word segmentation processing, and performing encoding processing on the historical text information through the second sub-model to obtain historical text character features corresponding to the historical text information; The first sub-model and the second sub-model are jointly trained based on the historical visual features, the historical text character features and a preset contrastive learning loss function to obtain a trained image-text cross-modal model.

9. A credential audit device, comprising: An audit request receiving module receives an audit request for a target credential, wherein the audit request includes a credential image of the target credential; An information acquisition module, which acquires key information contained in the credential image, and acquires reference information corresponding to the key information and used for reviewing the target credential; A feature extraction module, which extracts visual features corresponding to the key information of the voucher image and reference features corresponding to the reference information based on a pre-trained content comparison model, wherein the content comparison model is a model obtained by model training based on historical images, historical reference information corresponding to the key information of the historical images, and a preset contrastive learning loss function; The audit module calculates the similarity between the visual feature and the reference feature, and determines the audit result of the target credential based on the calculated similarity.

10. An electronic device, comprising: processor; as well as a memory arranged to store computer executable instructions which, when executed, cause the processor to: receiving a review request for a target credential, wherein the review request includes a credential image of the target credential; Acquire key information contained in the credential image, and acquire reference information corresponding to the key information and used for reviewing the target credential; extracting visual features corresponding to the key information of the voucher image and reference features corresponding to the reference information based on a pre-trained content comparison model, wherein the content comparison model is a model obtained by model training based on historical images, historical reference information corresponding to the key information of the historical images, and a preset contrastive learning loss function; The similarity between the visual feature and the reference feature is calculated, and the audit result of the target credential is determined based on the calculated similarity.

Images