Method and device for auditing application information

Abstract

The invention discloses a method and device for auditing application information, and the method comprises the steps: obtaining the application information submitted by a user, the application information comprising a first picture based on an application form filled by the user and a second picture for carrying out the character shooting of the user; extracting character content and character recognition accuracy corresponding to each application item from the first picture; extracting a background sub-picture and a human face sub-picture from the second picture, and determining picture recognition results and picture recognition accuracy corresponding to the background sub-picture and the human face sub-picture respectively; and according to the text content and the text recognition accuracy corresponding to each application item, and the picture recognition results and the picture recognition accuracy corresponding to the background sub-picture and the human face sub-picture respectively, determining whether the application information passes auditing or not. By the adoption of the method, the accuracy and efficiency of determining whether the application information passes auditing or not can be improved, and meanwhile the possibility of user information leakage is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of credit card application, and in particular to a method and device for reviewing application information. Background Art

[0002] With the rapid development of the financial market, the financial field based on credit cards has become one of the directions for banks and other financial institutions to expand their business.

[0003] In the existing credit card approval process, there are many aspects that require manual participation, such as confirming whether there are defects in the application form filled out by the user, whether the photo uploaded by the user is clear, etc., in order to determine the authenticity of the credit card application information. However, manual review carries the risk of errors caused by visual recognition errors, and due to limited manual efficiency, it affects the user's credit card approval cycle and increases the working time cost of the staff. Summary of the invention

[0004] The present application provides a method and device for reviewing application information, which are used to improve the accuracy and efficiency of determining whether application information has been reviewed and approved, while also reducing the possibility of user information leakage.

[0005] In a first aspect, an embodiment of the present application provides a method for reviewing application information, which can be executed by a device for reviewing application information, the method comprising: obtaining application information submitted by a user, the application information comprising a first picture based on an application form filled out by the user and a second picture of a portrait of the user; extracting text content and text recognition accuracy corresponding to each application item from the first picture; extracting a background sub-image and a face sub-image from the second picture, and determining image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image, respectively; determining whether the application information has been reviewed and approved based on the text content and text recognition accuracy corresponding to each application item, and the image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image, respectively.

[0006] By adopting the above method, the text content corresponding to each application item in the application form is extracted through the device for reviewing the application information, and the background sub-image and the face sub-image are extracted, and the image recognition results of the background sub-image and the face sub-image are determined, and whether the review is passed is determined according to the text content and the image recognition results corresponding to each application item. In this way, by automatically determining whether the review information is passed through the device, the workload of manual review can be reduced, the efficiency of reviewing information can be improved, and the risk of user information being leaked can be reduced.

[0007] In one possible implementation, whether the application information has been approved is determined based on the text content and text recognition accuracy corresponding to each application item, as well as the image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image, respectively, including: if the text recognition accuracy corresponding to any application item is lower than a first threshold, obtaining the text content corresponding to the application item fed back by the reviewer; if the image recognition accuracy corresponding to any sub-image is lower than a second threshold, obtaining the image recognition result of the sub-image fed back by the reviewer; based on the text content corresponding to each application item after feedback and each image recognition result after feedback, determining whether the application information has been approved.

[0008] By adopting the above method, it is possible to avoid the situation where errors occur in the review and the need to continue to determine whether the review information has passed the review. By feeding back application items and images with low recognition accuracy to the reviewer for review, and determining whether the application information has passed based on the text content and image recognition results fed back by the reviewer, the accuracy of the review application information can be improved.

[0009] In a possible implementation, based on the text content corresponding to each application item after feedback and the image recognition results after feedback, it is determined whether the application information has been approved, including: for at least one application item, determining whether the text content corresponding to the application item conforms to the set format, and / or whether the text content corresponding to the application item conforms to the set content, and determining whether the application item is abnormal; based on whether each image recognition result is abnormal and whether each application item is abnormal, determining whether the application information has been approved.

[0010] In a possible implementation, whether the application information is approved is determined based on whether each image recognition result is abnormal and whether each application item is abnormal, including: if the application item is abnormal, the application item corresponds to a first score; if the application item is not abnormal, the application item corresponds to a second score; the first score is less than the second score; if the image recognition result is abnormal, the image recognition result corresponds to a third score; if the image recognition result is not abnormal, the image recognition result corresponds to a fourth score; the third score is less than the fourth score; based on the sum of the scores corresponding to each application item and the scores corresponding to each image recognition result and a preset threshold score, it is determined whether the application information is approved.

[0011] By adopting the above method, by scoring whether the application items are abnormal and scoring the image recognition results, it is possible to more intuitively and comprehensively judge whether the application information has been approved.

[0012] In one possible implementation, the image recognition result corresponding to the background sub-image includes a first image recognition result and / or a second image recognition result; determining the image recognition result corresponding to the background sub-image includes: performing abnormal recognition on the background sub-image, if there is abnormal text, the first image recognition result is yes; if there is no abnormal text, the first image recognition result is no; and / or performing abnormal recognition on the background sub-image, if there is an abnormal background, the second image recognition result is yes; if there is no abnormal background, the second image recognition result is no.

[0013] By adopting the above method, a more detailed review of the background sub-image can be conducted to identify whether there are abnormal texts or abnormal backgrounds in the background sub-image, thereby facilitating the screening of application information with abnormal background sub-images.

[0014] In a possible implementation, the image recognition result corresponding to the face sub-image includes a third image recognition result, and / or a fourth image recognition result, and / or a fifth image recognition result; determining the image recognition result corresponding to the face sub-image includes: determining at least one of the following based on the face sub-image: whether there is an abnormal expression, whether there is a fraudulent face, and whether there is a different face; performing abnormal recognition on the face sub-image, and if there is an abnormal expression, the third image recognition result is yes; if there is no abnormal expression, the third image recognition result is no; performing abnormal recognition on the face sub-image, and if there is a fraudulent face, the fourth image recognition result is yes; if there is no fraudulent face, the fourth image recognition result is no; performing abnormal recognition on the face sub-image, and if it is a different face compared to the ID face, the fifth image recognition result is yes, and if there is no different face, the fifth image recognition result is no.

[0015] By adopting the above method, a more detailed review of the face sub-image can be conducted to identify whether the face sub-image has abnormal expressions, fraudulent faces, and different faces, so as to facilitate the screening of application information with abnormal face sub-images.

[0016] In a second aspect, an embodiment of the present application provides a method for reviewing application information, the device comprising: an acquisition module for acquiring application information submitted by a user, the application information comprising a first picture based on an application form filled out by the user and a second picture of a portrait of the user; an extraction module for extracting text content and text recognition accuracy corresponding to each application item from the first picture; and extracting a background sub-image and a face sub-image from the second picture; a determination module for determining image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image respectively; the determination module is also used to determine whether the application information is reviewed and approved based on the text content and text recognition accuracy corresponding to each application item, and the image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image respectively.

[0017] In a possible implementation, the acquisition module is also used to, if the text recognition accuracy corresponding to any application item is lower than a first threshold, obtain the text content corresponding to the application item fed back by the reviewer; if the image recognition accuracy corresponding to any sub-image is lower than a second threshold, obtain the image recognition result of the sub-image fed back by the reviewer; the determination module is specifically used to determine whether the application information has been reviewed and approved based on the text content corresponding to each application item after feedback and the image recognition results after feedback.

[0018] In one possible implementation, the determination module is specifically used to determine, for at least one application item, whether the text content corresponding to the application item conforms to a set format, and / or whether the text content corresponding to the application item conforms to a set content, and determine whether the application item is abnormal; based on whether each image recognition result is abnormal and whether each application item is abnormal, determine whether the application information is reviewed and approved.

[0019] In a possible implementation, if the application item is abnormal, the application item corresponds to a first score; if the application item is not abnormal, the application item corresponds to a second score; the first score is less than the second score; if the image recognition result is abnormal, the image recognition result corresponds to a third score; if the image recognition result is not abnormal, the image recognition result corresponds to a fourth score; the third score is less than the fourth score; the determination module is specifically used to determine whether the application information is reviewed and approved based on the sum of the scores corresponding to each application item and the scores corresponding to each image recognition result and a preset threshold score.

[0020] In one possible implementation, the device includes an identification module, which is used to perform abnormal identification on the background sub-image, if there is abnormal text, the first image recognition result is yes; if there is no abnormal text, the first image recognition result is no; and / or perform abnormal identification on the background sub-image, if there is an abnormal background, the second image recognition result is yes; if there is no abnormal background, the second image recognition result is no.

[0021] In one possible implementation, the determination module is specifically used to determine at least one of the following based on the facial sub-image: whether there is an abnormal expression, whether there is a fraudulent face, and whether there is a different face; the recognition module is also used to perform abnormal recognition on the facial sub-image, and if there is an abnormal expression, the third image recognition result is yes; if there is no abnormal expression, the third image recognition result is no; perform abnormal recognition on the facial sub-image, and if there is a fraudulent face, the fourth image recognition result is yes; if there is no fraudulent face, the fourth image recognition result is no; perform abnormal recognition on the facial sub-image, and if it is a different face compared to the ID face, the fifth image recognition result is yes, and if there is no different face, the fifth image recognition result is no.

[0022] In a third aspect, an embodiment of the present application further provides a device for reviewing application information, the device comprising a memory and a processor, the memory being used to store computer programs or instructions; the processor being used to call the computer programs or instructions stored in the memory to execute a method as in any possible implementation of the first aspect.

[0023] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which instructions are stored. When a computer reads and executes the instructions, the computer executes a method in any possible implementation of the first aspect.

[0024] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein instructions are stored in the computer program product. When a computer reads and executes the instructions, the computer executes the method in any possible implementation of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1A flowchart corresponding to a method for reviewing application information provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of the internal modules of a device 2000 for reviewing application information provided in an embodiment of the present application;

[0028] Figure 3 A schematic diagram of the structure of an application information review device 3000 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0030] At present, when applying for a credit card, a user needs to submit an application form filled out by the user to the bank. The application form usually includes the user's personal information, such as name, gender, date of birth, ID number or passport number, contact information; it can also include the user's signature and statement, such as the applicant's handwritten signature, consent terms, and statement of the authenticity of the information provided; it can also include other information, such as educational background, marital status, etc. The application form can also include other information, and the application form templates of different banks can also be different. The embodiment of this application does not limit the specific content in the application form. In addition, the user also needs to submit some photos, such as a photo of the user holding the ID card and the bank lobby manager, a photo of the user holding the ID card and the bank's ground sales staff, or a photo of the user holding the ID card and his own face. Different banks have different requirements for the types of photos submitted by users. Some banks also require users to submit proof of income and proof of work, etc. The embodiment of this application does not specifically limit the types of photos submitted by users.

[0031] In the existing credit card review process, staff are required to review the application form information and photo information one by one, that is, to review whether there are any missing items in the application form filled out by the user, whether the filled-in content is true, whether the signature meets the requirements, and whether the ID card photo is clear, in order to determine the authenticity of the application form information. However, due to the large number and complexity of the review content, the review efficiency is low, and there are audit errors caused by staff's naked eye recognition errors, which will also affect the accuracy of the review. In addition, manual review will have the risk of user privacy and security being leaked. Based on this, the embodiment of the present application provides a method for reviewing application information, which is used to improve the accuracy and efficiency of determining whether the application information has been reviewed and approved, and reduce the risk of user information leakage.

[0032] Figure 1A flowchart corresponding to a method for reviewing application information provided in an embodiment of the present application is provided. The flowchart can be executed by a device for reviewing application information, such as Figure 1 As shown, the process includes the following steps:

[0033] Step 101 : obtaining application information submitted by a user, wherein the application information includes a first picture based on an application form filled in by the user and a second picture of the user.

[0034] Specifically, after receiving the paper application form filled out by the user, the staff will take a photo of the paper application form to generate a first picture, and receive a second picture submitted by the user. The second picture can be a picture of the user holding the front and back of the ID card photo with his or her own face or a picture of the lobby manager.

[0035] Step 102, extracting text content and text recognition accuracy corresponding to each application item from the first image, extracting background sub-image and face sub-image from the second image, and determining image recognition results and image recognition accuracy corresponding to the background sub-image and face sub-image respectively.

[0036] Specifically, the text content corresponding to each application item can be directly extracted from the first picture, and the recognition result of the second picture can be determined. The text content corresponding to each application item and the recognition accuracy of the text can also be extracted from the first picture according to the first model, and the background sub-image and the face sub-image can be extracted from the second picture according to the second model, and the picture recognition results and picture recognition accuracy corresponding to the background sub-image and the face sub-image can be determined. The technology for recognizing text from a picture can be optical character recognition (OCR) technology. At present, there are multiple models using OCR technology, including OCR.space model, Baidu OCR model and Google Cloud Vision API, etc. The model using OCR technology is the first model of this application, and the embodiment of this application does not limit the type of the first model. There are also multiple models for extracting background sub-images and face sub-images from pictures, such as single-stage multi-layer target detection (Single Shot MultiBox Detector, SSD) model, model based on image segmentation (GrabCut), etc., and the embodiment of this application does not specifically limit the type of the second model.

[0037] It is understandable that the accuracy of extracting the text content corresponding to each application item from the first image using the existing first model is not ideal. Therefore, in order to make the first model more suitable for the method of reviewing application information of this application, the first model needs to be further improved and trained.

[0038] (1) Training the first model

[0039] Exemplarily, a plurality of first sample images are obtained, and the plurality of first sample images are divided into first training sample images and first test sample images in a ratio of 80% and 20%, and the plurality of first training sample images are input into a device where the first model is located, and the device may be a computer, a server, or other device. Taking one of the first training sample images as an example, the first model can extract the text content corresponding to each application item in the training sample image and the accuracy of text recognition, but at this time, the content extracted by the first model for the training sample image may be wrong, such as the extracted text content is wrong, or the text content corresponding to the application item is omitted. In view of the error in the text content extracted by the first model, the staff can modify the output of the first model based on the training sample image, that is, based on the text content extracted by the first model, modify the wrong text content to the correct text content. For example, in the first training sample image, the age information in the user's personal information is 21, and the age information extracted by the first model is 2, then the staff will modify the age information to 21 based on the first training sample image. In the case where the first model misses to extract the text content corresponding to the application item, the staff can perform auxiliary framing on the first training sample image. For example, if the first model misses to extract the user's personal signature part, the staff can frame the personal signature part in the first training sample image, so that the first model can extract the first training sample image again, and can focus on the content recognition in the frame. In this way, the correct text content corresponding to each application item can be obtained by the staff framing the first training sample image and modifying the extracted text content. In addition, the first model can also extract the recognition accuracy of the text content corresponding to each application item. The recognition accuracy is used to indicate the accuracy of the first model for the extracted content. Optionally, the staff can modify the accuracy of the extracted content of the first model, improve the accuracy of the extracted correct content, and reduce the accuracy of the extracted incorrect content. As for the degree of improvement or reduction, it can be determined by the staff's experience. According to the same method, multiple first training sample images are input into the first model to obtain the correct text content and text recognition accuracy corresponding to each application item in the multiple first training sample images. Then, the correct text content and text recognition accuracy corresponding to each application item in the multiple first training sample images and the multiple first training sample images are input into the first model.

[0040] Further, a plurality of first test sample images are input into the first model, and the first model extracts the text content and text recognition accuracy corresponding to each application item in each first test sample image. According to the text content corresponding to each application item in each first test sample image extracted by the first model and each first test sample image, determine whether the text content corresponding to each application item in each first test sample image is correct. According to the text recognition accuracy corresponding to each application item extracted by the first model and whether the text content corresponding to each application item is correct, determine the first threshold. For example, if the text recognition accuracy of the application item extracted by the first model is higher than 70%, all text contents are correctly recognized, and if it is lower than 70%, there are recognition errors, then the first threshold can be set to 70%. According to whether the first model correctly recognizes the text content corresponding to each application item in each first test sample image, calculate the accuracy of the first model. If there are 1000 application items in all first test sample images, and the text contents corresponding to 950 application items are correctly recognized, then the accuracy of the first model is 95%. According to the preset first model accuracy threshold, the preset first model accuracy threshold can be 90%. If the accuracy of the first model exceeds the preset first model accuracy threshold, the first model training is successful. If the accuracy of the first model does not exceed the preset first model accuracy threshold, continue to obtain the first training sample image and continue to train the first model.

[0041] (2) Training the second model

[0042] Similarly, the existing second model can be used to extract the background sub-image and the face sub-image from the second image, but the image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image are not ideal. Therefore, the second model also needs to be further improved and trained.

[0043] Specifically, in order to more accurately identify the second image, the second model may include multiple sub-models, such as a first sub-model for identifying abnormal text on the background sub-image, a second sub-model for identifying abnormal background on the background sub-image, a third sub-model for identifying abnormal expression on the face sub-image, a fourth sub-model for identifying fraudulent faces on the face sub-image, and a fifth sub-model for identifying different faces on the face sub-image. Among them, abnormal text may include text that is not related to credit card application, such as advertisements, insulting language, etc.; abnormal background may be illegal and suspicious backgrounds such as hospitals; abnormal and suspicious expressions are abnormal expressions such as rolling eyes, vomiting, contempt, and panic; fraudulent faces refer to faces of fraudulent users stored in the system within a historical time period; different faces refer to faces in the face sub-image that are not the same as the faces in the user's ID card, and not being the same means that the ID card is not the person's ID card.

[0044] Obtain multiple second sample images, and divide the multiple second sample images into second training sample images and second test sample images in a ratio of 80% and 20%, wherein the multiple second training sample images include various abnormal images and normal images, and the second test sample images also include various abnormal images and normal images. Input the labels corresponding to the multiple second training sample images and the multiple second training sample images into each sub-model. Specifically, for the first sub-model, the image with abnormal text in the background sub-image is a bad sample, and the image without abnormal text in the background sub-image is a good sample. In other words, even if there are other abnormalities in the image, as long as there is no abnormal text in the background sub-image, it is a good sample. For the second sub-model, the image with abnormal background in the background sub-image is a bad sample, and the image without abnormal background in the background sub-image is a good sample. Even if there are other abnormalities in the image, as long as there is no abnormal background in the background sub-image, it is a good sample. The same is true for other sub-models, and the embodiments of the present application are not repeated here.

[0045] Multiple second training sample images and labels of each second training sample image are input into the first sub-model. For the first sub-model, the label of the second training sample image is whether or not there is abnormal text in the background sub-image. Multiple second training sample images and labels of each second training sample image are input into the second sub-model. For the second sub-model, the label of the second training sample image is whether or not there is an abnormal background in the background sub-image. For other sub-models, the labels of the second training sample images are also adaptively changed, and the embodiments of the present application will not be repeated here.

[0046] Further, a plurality of second test sample images are input into each sub-model, each sub-model extracts a background sub-image or a face sub-image, and determines the image recognition result and image recognition accuracy corresponding to the background sub-image or the face sub-image. For the first sub-model, the first sub-model is a model for abnormal text recognition of the background sub-image, so the first sub-model extracts the background sub-image and determines the image recognition result and image recognition accuracy of the background sub-image. The first sub-model determines that the recognition result of the background sub-image is that there are abnormal texts in the background sub-image or that there are no abnormal texts in the background sub-image, and determines the image recognition accuracy of the background sub-image, that is, the accuracy of the recognition result of the first sub-model. For example, the first sub-model extracts the background sub-image from one of the second test sample images, determines that the recognition result of the background sub-image is that there are abnormal texts in the background sub-image, and the recognition accuracy is 80%, that is, the accuracy of the first sub-model for the recognition result is 80%. According to the recognition result of each second test sample image and each second test sample image by the first sub-model, it is determined whether the recognition result of each second test sample image is correct. According to whether the recognition results of each second test sample image are correct and the recognition accuracy of each second test sample image, the second threshold is determined. For example, if the recognition results of the first sub-model for images with a recognition accuracy higher than 80% are all correct, the second threshold can be positioned at 80%. According to whether the recognition results of the first sub-model for each second test sample image are correct, the accuracy of the first sub-model is calculated. If there are 1000 second test sample images tested by the first sub-model, and 930 sample images are correctly recognized, the accuracy of the first sub-model is 93%. According to the preset first sub-model accuracy threshold, the preset first sub-model accuracy threshold can be 90%. If the accuracy of the first sub-model exceeds the preset first sub-model accuracy threshold, the first sub-model training is successful. If the accuracy of the first sub-model does not exceed the preset first sub-model accuracy threshold, the second training sample image continues to be obtained and the first sub-model continues to be trained. How to train other sub-models and determine the second thresholds of other sub-models is the same as the above method, and the embodiment of the present application will not be elaborated on this.

[0047] Optionally, the embodiment of the present application can train and test multiple first models and second models, and select models with higher accuracy as the first model and the second model. Optionally, the number of first models in the embodiment of the present application can be multiple. Since the formats of the application forms of each bank are not the same, the application forms with the same format can be grouped together, and the accuracy of each first model for each group of application forms is calculated to determine the first model corresponding to each group of application forms. Optionally, since the training model also has requirements for the equipment, powerful computing power is the key to the training model, and high-performance GPUs can accelerate parallel computing tasks. In the training model, ensure that the GPU's video memory capacity is sufficient to accommodate the model parameters and intermediate data, and evaluate the processing speed and parallel processing capabilities required by the model based on the computing requirements of the model.

[0048] After the first model is trained, the text content and text recognition accuracy corresponding to each application item are extracted from the first picture according to the first model. The application item and the text content corresponding to the application item can be saved in the form of a key-value pair. After the second model is trained, the background sub-image and the face sub-image are extracted from the second picture according to the second model, and the image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image are determined. Specifically, the background sub-image is extracted from the second picture according to the first sub-model in the second model, and the background sub-image is identified as abnormal. If there is abnormal text, the first image recognition result of the second picture is that the background sub-image has abnormal text; if there is no abnormal text, the first image recognition result of the second picture is that the background sub-image does not have abnormal text. The background sub-image is extracted from the second picture according to the second sub-model, and the background sub-image is identified as abnormal. If there is an abnormal background, the second image recognition result of the second picture is that the background sub-image has an abnormal background; if there is no abnormal background, the second image recognition result of the second picture is that the background sub-image does not have an abnormal background. According to the third sub-model, a face sub-image is extracted from the second image, and abnormal recognition is performed on the face sub-image. If there is an abnormal expression, the third image recognition result of the second image is that the face sub-image has an abnormal expression; if there is no abnormal expression, the third image recognition result of the second image is that the face sub-image does not have an abnormal expression. According to the fourth sub-model, a face sub-image is extracted from the second image, and abnormal recognition is performed on the face sub-image. If there is a fraudulent face, the fourth image recognition result of the second image is that the face sub-image has a fraudulent face; if there is no fraudulent face, the fourth image recognition result of the second image is that the face sub-image does not have a fraudulent face. According to the fifth sub-model, a face sub-image is extracted from the second image, and abnormal recognition is performed on the face sub-image. If there are different faces, the fifth image recognition result of the second image is that the face sub-image has different faces; if there are no different faces, the fifth image recognition result of the second image is that the face sub-image does not have different faces. In addition, the first image recognition result, the second image recognition result, the third image recognition result, the fourth image recognition result, and the fifth image recognition result all have corresponding recognition accuracy.

[0049] Step 103, determining whether the application information has been approved based on the text content and text recognition accuracy corresponding to each application item, and the image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image respectively.

[0050] Specifically, if the text recognition accuracy corresponding to any application item is lower than the first threshold, it means that the output of the first model for the text content corresponding to the application item is not necessarily accurate, and the first picture is sent to the reviewer to obtain the text content corresponding to the application item feedback from the reviewer; if the image recognition accuracy corresponding to any background sub-image or any face sub-image is lower than the second threshold, it means that the image recognition result of the second model for the background sub-image or the face sub-image is not necessarily accurate, and the second picture is sent to the reviewer to obtain the image recognition result of the background sub-image or the face sub-image feedback from the reviewer.

[0051] According to the text content corresponding to each application item after feedback and the image recognition results after feedback, determine whether the application information has been reviewed and approved. Specifically, determine whether the text content corresponding to the application item conforms to the set format, and / or whether the text content corresponding to the application item conforms to the set content, and determine whether the application item is abnormal. For example, the text content corresponding to the application item includes a mobile phone number. If the mobile phone number is not 11 digits, it is determined that the text content corresponding to the application item does not conform to the set format; for example, the text content corresponding to the application item includes an ID card number. According to the photo of the user holding the ID card in the second picture, the ID card number of the user can be determined, and the ID card number of the user is saved in the system. If the ID card number corresponding to the application item is inconsistent with the ID card number of the user in the system, it is determined that the text content corresponding to the application item does not conform to the set content. It can be understood that the text content corresponding to the application item can also include other content, and the user can also provide other content when applying for a credit card. For example, the text content corresponding to the application item can include an average monthly income. The user can provide the user's salary flow information when applying for a credit card, and the user's salary flow information is saved in the system. If the average monthly income corresponding to the application item is inconsistent with the user's salary flow in the system, it is determined that the text content corresponding to the application item does not conform to the set content. If there are many abnormal application items and abnormal image recognition results, it can be determined that the audit information has not passed; if there are few abnormal application items and abnormal image recognition results, it can be determined that the audit information has passed.

[0052] Optionally, the application information is scored according to whether each application item is abnormal and whether each image recognition result is abnormal, and whether the audit information is passed is determined according to the total scoring result. Exemplarily, if the application item is abnormal, the application item corresponds to a first score, if the application item is normal, the application item corresponds to a second score, wherein the first score is less than the second score; if the image recognition result is abnormal, the image recognition result corresponds to a third score; if the image recognition result is normal, the image recognition result corresponds to a fourth score, wherein the third score is less than the fourth score. For example, an application form may include 10 application items. If the text content corresponding to the application item is abnormal, it can be set to 0 points, that is, the first score is 0 points. If the text content corresponding to the application item is normal, it can be set to 5 points, that is, the second score is 5 points. Optionally, the second score can also be set to different scores according to the importance of the application item. The embodiment of the present application does not specifically limit the specific values ​​of the first score and the second score.

[0053] The picture recognition result is normal or abnormal. The picture recognition result corresponding to a second picture includes 5 picture recognition results. If 3 picture recognition results are abnormal, it can be determined that the recognition result of the second picture is abnormal, otherwise the recognition result of the second picture is normal. Optionally, each picture recognition result can also be scored. If the first picture recognition result is abnormal, it can be set to 0 points. If the first picture recognition result is normal, it can be set to 10 points. The normal or abnormal scores corresponding to the second picture recognition result, the third picture recognition result, the fourth picture recognition result and the fifth picture recognition result can be the same as the first picture recognition result, or different from the first picture recognition result. The embodiment of the present application does not limit this. According to the sum of the scores corresponding to each application item and the scores corresponding to each picture recognition result and the preset threshold score, determine whether the application information is approved. The preset threshold score can be set according to the experience of the staff. If the sum of the scores corresponding to each application item and the scores corresponding to each picture recognition result exceeds the preset threshold, it is determined that the audit information submitted by the user is approved. If the sum of the scores corresponding to each application item and the scores corresponding to each picture recognition result does not exceed the preset threshold, it is determined that the audit information submitted by the user is not approved.

[0054] Figure 2 A schematic diagram of the internal modules of an application information review device 2000 provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the device may include: an acquisition module 201 , an extraction module 202 , a determination module 203 , and an identification module 204 .

[0055] Among them, the acquisition module 201 is used to obtain the application information submitted by the user, and the application information includes a first picture based on the application form filled out by the user and a second picture of the user; the extraction module 202 is used to extract the text content and text recognition accuracy corresponding to each application item from the first picture; and to extract the background sub-image and the face sub-image from the second picture; the determination module 203 is used to determine the image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image respectively; the determination module 203 is also used to determine whether the application information is reviewed and approved based on the text content and text recognition accuracy corresponding to each application item, and the image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image respectively.

[0056] In a possible implementation, the acquisition module 201 is also used to, if the text recognition accuracy corresponding to any application item is lower than a first threshold, obtain the text content corresponding to the application item fed back by the reviewer; if the image recognition accuracy corresponding to any sub-image is lower than a second threshold, obtain the image recognition result of the sub-image fed back by the reviewer; the determination module 203 is specifically used to determine whether the application information has been reviewed and approved based on the text content corresponding to each application item after feedback and the image recognition results after feedback.

[0057] In one possible implementation, the determination module 203 is specifically used to determine, for at least one application item, whether the text content corresponding to the application item conforms to a set format, and / or whether the text content corresponding to the application item conforms to a set content, and whether the application item is abnormal; based on whether each image recognition result is abnormal and whether each application item is abnormal, determine whether the application information has been reviewed and approved.

[0058] In a possible implementation, if the application item is abnormal, the application item corresponds to a first score; if the application item is not abnormal, the application item corresponds to a second score; the first score is less than the second score; if the image recognition result is abnormal, the image recognition result corresponds to a third score; if the image recognition result is not abnormal, the image recognition result corresponds to a fourth score; the third score is less than the fourth score; the determination module 203 is specifically used to determine whether the application information is reviewed and approved based on the sum of the scores corresponding to each application item and the scores corresponding to each image recognition result and a preset threshold score.

[0059] In one possible implementation, the device includes an identification module 204, and the identification module 204 is used to perform abnormal identification on the background sub-image, if there is abnormal text, the first image recognition result is yes; if there is no abnormal text, the first image recognition result is no; and / or perform abnormal identification on the background sub-image, if there is an abnormal background, the second image recognition result is yes; if there is no abnormal background, the second image recognition result is no.

[0060] In a possible implementation, the determination module 203 is specifically used to determine at least one of the following based on the facial sub-image: whether there is an abnormal expression, whether there is a fraudulent face, and whether there is a different face; the recognition module 204 is also used to perform abnormal recognition on the facial sub-image, and if there is an abnormal expression, the third image recognition result is yes; if there is no abnormal expression, the third image recognition result is no; perform abnormal recognition on the facial sub-image, and if there is a fraudulent face, the fourth image recognition result is yes; if there is no fraudulent face, the fourth image recognition result is no; perform abnormal recognition on the facial sub-image, and if it is a different face compared to the ID face, the fifth image recognition result is yes, and if there is no different face, the fifth image recognition result is no.

[0061] Figure 3 The structure diagram of a device 3000 for reviewing application information provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, it includes at least one processor 301 and a memory 302 connected to the at least one processor 301. The specific connection medium between the processor 301 and the memory 302 is not limited in the embodiment of the present application. Figure 3 For example, the processor 301 and the memory 302 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0062] In the embodiment of the present invention, the memory 302 stores instructions that can be executed by at least one processor 301. The at least one processor 301 can implement the steps of the above-mentioned method for reviewing application information by executing the instructions stored in the memory 302.

[0063] The processor 301 is the control center of the computer device, and can use various interfaces and lines to connect various parts of the computer device, and perform resource settings by running or executing instructions stored in the memory 302 and calling data stored in the memory 302. Optionally, the processor 301 may include one or more processing units, and the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 301. In some embodiments, the processor 301 and the memory 302 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on independent chips.

[0064] The processor 301 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0065] The memory 302 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 302 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 302 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 302 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0066] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0067] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0068] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0070] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for reviewing application information, characterized in that: The method comprises: Acquire application information submitted by the user, the application information including a first picture based on an application form filled in by the user and a second picture of the user; Extracting text content and text recognition accuracy corresponding to each application item from the first image; Extracting a background sub-image and a face sub-image from the second image, and determining image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image respectively; Determine whether the application information is approved based on the text content and text recognition accuracy corresponding to each application item, and the image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image respectively; The image recognition result corresponding to the background sub-image includes a first image recognition result and / or a second image recognition result; Determining the image recognition result corresponding to the background sub-image includes: Perform abnormal identification on the background sub-image, if there is abnormal text, the first image identification result is yes; if there is no abnormal text, the first image identification result is no; and / or Anomalies are identified on the background sub-image. If an abnormal background exists, the second image recognition result is yes; if no abnormal background exists, the second image recognition result is no.

2. The method according to claim 1, characterized in that Determining whether the application information is approved based on the text content and text recognition accuracy corresponding to each application item, and the image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image, including: If the text recognition accuracy corresponding to any application item is lower than the first threshold, obtaining the text content corresponding to the application item fed back by the reviewer; If the image recognition accuracy corresponding to any sub-image is lower than the second threshold, obtaining the image recognition result of the sub-image fed back by the reviewer; Based on the text content corresponding to each application item after feedback and the image recognition results after feedback, determine whether the application information has been reviewed and approved.

3. The method according to claim 2, characterized in that Based on the text content corresponding to each application item and the image recognition results after feedback, determine whether the application information has been reviewed and approved, including: For at least one application item, determine whether the text content corresponding to the application item conforms to a set format and / or whether the text content corresponding to the application item conforms to a set content, and determine whether the application item is abnormal; Based on whether each image recognition result is abnormal and whether each application item is abnormal, determine whether the application information is approved.

4. The method according to claim 3, characterized in that Determine whether the application information is approved based on whether the image recognition results are abnormal and whether the application items are abnormal, including: If the application item is abnormal, the application item corresponds to a first score; if the application item is not abnormal, the application item corresponds to a second score; the first score is less than the second score; If the image recognition result is abnormal, the image recognition result corresponds to a third score; if the image recognition result is not abnormal, the image recognition result corresponds to a fourth score; the third score is less than the fourth score; Whether the application information has been approved is determined based on the sum of the scores corresponding to each application item and each image recognition result and the preset threshold score.

5. The method according to any one of claims 1 to 4, characterized in that The picture recognition result corresponding to the face sub-picture includes the third picture recognition result, and / or the fourth picture recognition result, and / or the fifth picture recognition result; Determining the image recognition result corresponding to the face sub-image includes: Determine at least one of the following based on the face sub-graph: whether there is an abnormal expression, whether there is a fraudulent face, and whether there is a different face; Performing abnormal recognition on the face sub-image, if there is an abnormal expression, the recognition result of the third image is yes; if there is no abnormal expression, the recognition result of the third image is no; Performing abnormality recognition on the face sub-image, if there is a fraudulent face, the fourth image recognition result is yes; if there is no fraudulent face, the fourth image recognition result is no; The face sub-image is subjected to abnormality recognition. If the face is different from the face in the ID card, the recognition result of the fifth image is yes. If there is no different face, the recognition result of the fifth image is no.

6. A device for reviewing application information, characterized in that: The device comprises: An acquisition module, used to acquire application information submitted by a user, wherein the application information includes a first picture based on an application form filled in by the user and a second picture of the user; An extraction module, used to extract text content and text recognition accuracy corresponding to each application item from the first image; and to extract a background sub-image and a face sub-image from the second image; A determination module, used to determine the image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image respectively; The determination module is further used to determine whether the application information is approved according to the text content and text recognition accuracy corresponding to each application item, and the image recognition results and image recognition accuracy corresponding to the background sub-image and the face sub-image respectively; An identification module is used to perform abnormal identification on the background sub-image, if there is abnormal text, the first image recognition result is yes; if there is no abnormal text, the first image recognition result is no; and / or perform abnormal identification on the background sub-image, if there is an abnormal background, the second image recognition result is yes; if there is no abnormal background, the second image recognition result is no.

7. A device for reviewing application information, characterized in that: include: Memory, used to store computer programs or instructions; A processor, configured to call a computer program or instruction stored in the memory to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when a computer reads and executes the instructions, the computer executes the method according to any one of claims 1 to 5.

9. A computer program product, characterized in that The computer program product stores instructions, and when a computer reads and executes the instructions, the computer executes the method according to any one of claims 1 to 5.

Images