Exception handling using instant optical character recognition

Abstract

A system for detecting and resolving anomalies during a document upload process is disclosed. The system may receive, via a document upload application installed on a user device, a document to be uploaded to a user account maintained by the system. The OCR component may be used to detect anomalies associated with the document, and the system may include a component to handle any detected anomalies to prevent termination of the document upload process.

Description

TECHNICAL FIELD

[0001] Aspects relate to systems and methods for implementing exception handling during a document upload process as part of performing instant optical character recognition (OCR) on a document to be uploaded to a backend system. BACKGROUND

[0002] Currently, computer-based (e.g., laptop) or mobile-based (e.g., mobile device) technology allows users to upload images or other electronic versions of documents to a backend system (e.g., a document processing system) for various purposes (e.g., remote deposit of a check, obtaining approval for a credit card, or updating user account information). The documents can include information that is used by the backend system (e.g., at a bank). This information can include user information related to the user initiating the upload, account information related to the account into which the document is being uploaded, and content information related to the content detected in the document. In some cases, certain exceptions can occur during the document upload process, i.e., when the document is being uploaded and processed by the backend system. Examples of such exceptions include problems related to the document image(s) being uploaded or user-specific error conditions associated with the content stored on the document. As one example, a check that a user attempts to upload with a mobile deposit application can not be eligible.

[0003] Current document upload processes have limited (if any) ability to handle such exceptions. That is, when any exception is detected during the upload, the current upload process can automatically and immediately terminate, thereby preventing the upload from occurring, without providing any exception or error handling options for resolving the detected error. The user is then forced to retry the upload attempt. BRIEF DESCRIPTION OF DRAWINGS

[0004] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate the aspects of the present disclosure and, together with the description, further serve to explain the principles of the present disclosure and to enable a person skilled in the pertinent art to make and use the present disclosure.

[0005] Figure 1 is an exemplary document upload environment for providing exception handling during a document upload process in accordance with aspects of the present disclosure.

[0006] Figure 2A is an exemplary flowchart of a document upload process including performing a limit exception process in accordance with aspects of the present disclosure.

[0007] Figure 2B is an exemplary flowchart of a document upload process including performing a secondary review of a document in accordance with aspects of the present disclosure.

[0008] Figure 3is a block diagram of a machine learning system for training a quota model and using the quota model during an upload process according to some embodiments.

[0009] Figure 4 is an example computer system that can be used to implement various embodiments.

[0010] In the drawings, like reference numerals generally refer to like elements throughout. Additionally, the left-most digit(s) of some reference numerals identifies the DETAILED DESCRIPTION

[0011] The following detailed description is made with reference to the accompanying drawings, of which:

[0012] In the following description, numerous specific details are set forth to provide an understanding of the aspects. However, it will be apparent to one skilled in the art that aspects can be practiced without the specific details. In other instances, well-known circuits, system configurations, and process steps have not been described in detail in order to avoid obscuring aspects.

[0013] The drawings showing aspects of the system are semi-diagrammatic and not to scale and, particularly, some of the dimensions are for the clarity of presentation and are shown exaggerated in the drawing FIGs. Similarly, although the views in the drawing FIGs are of a plan, section or another view, certain sections or planes in the drawing FIGs can show only parts of the systems so as not to obscure the aspects with details that are not necessary for understanding the aspects.

[0014] In addition to or in place of the steps or elements mentioned, certain aspects have other steps or elements. These will become apparent to those of ordinary skill in the art by reading the following detailed description, with appropriate reference to the drawings.

[0015] System Overview and Functionality

[0016] System, apparatus, device, method, and / or computer program product embodiments, and / or combinations and sub-combinations thereof, for providing exception handling during a document upload process are provided herein. Exceptions are error conditions that can occur during an upload process to a backend system. These error conditions can be defined by the backend system that manages the document upload process, and can include, for example, deviations of a document from a standard format (determined based on the document type) such as missing expected fields (such as account number, routing number, user information, and other text), and content detected in the document that is greater than a threshold value allowed for the user account to which the document is to be uploaded. Exception handling in the present disclosure can be actions that are triggered upon detection of the detected error(s) for the purpose of mitigating risk to the backend system.

[0017] In some embodiments, the techniques described herein implement a system that performs optical character recognition (OCR) in sequence during a document upload process to detect any exceptions related to a document being uploaded, such as errors in a document image to be uploaded, risk conditions associated with the document (or content within the document), and / or risk conditions associated with a user. In some embodiments, the techniques disclosed herein provide a workflow that includes immediate OCR of a document image taken by a document upload application and uploaded to a backend system to update a user account. The term “immediate OCR” can refer to an OCR process that is automatically initiated at the backend system without user input, and in some examples, is automatically initiated upon receipt of a document image to be uploaded by the backend system. In a non-limiting example, the techniques disclosed herein trigger the transmission of a document image from a document upload application to a backend system upon receipt of one of a first document image or a second document image. The techniques described herein improve the technology associated with uploading documents by performing certain steps of the document upload process in advance (without necessarily waiting for an instruction(s) by a user).

[0018] Upon detecting an error condition, the system can initiate an exception handling process to mitigate the detected error condition without immediately and automatically terminating the upload process. In a non-limiting example, the error condition can be any error associated with the uploaded document image, such as improper alignment of the document within the image, unreadable content within the document image, and unexpected or missing parameters within the document. In another non-limiting example, the error condition can include a risk condition associated with content in the document and / or the user account into which the document is uploaded. The risk condition can indicate that the content in the document is greater than a threshold amount set for a particular document type, the user initiating the upload, or both. In some embodiments, the risk condition can indicate irregular items in the document that can be due to an error in the uploaded image or missing expected data within the document. The risk condition can be based on a history of reviewing the activity of the user account, such as previous upload attempts and account activity and history. In the context of mobile check deposit, the content detected in the check can identify a currency value to be deposited in a bank account of the user and a routing number associated with the issuing bank of the check. The user account can be a bank account (e.g., checking or savings) into which the currency amount specified by the check is to be deposited, and the threshold amount can be a check deposit limit. The content associated with the document can include the currency amount to be deposited into the user account, the issuing bank, the routing number, and the account number.

[0019] In some embodiments, the technology disclosed herein provides a framework for detecting risk conditions associated with a document and / or a user during a document upload process utilizing a machine learning model. The model can generate a confidence score indicating a risk level associated with uploading a particular document. In a non-limiting example, to generate the confidence score, the model can utilize a plurality of different input parameters, such as content detected within the document and user information associated with the user initiating the upload process. The machine learning model can be trained utilizing a machine learning engine for improving the accuracy of the confidence score. In a non-limiting example, the technology disclosed herein provides a machine learning model for identifying an amount limit that can be deposited into a bank account of a user, which is used as a threshold for a currency amount deposited by a check. If the currency amount is greater than the threshold, the model can perform a second determination of whether to approve the upload. The second determination can be based on the confidence score being greater than a second threshold, which represents an amount of risk in allowing the upload to proceed. The machine learning model can be used to dynamically identify the amount limit for each user based on the parameters discussed above, user information, and content extracted from the document to be uploaded. The technology described herein improves the technology associated with processing document uploads by correctly identifying abnormal conditions at least during the upload and providing mechanisms to resolve the abnormality without terminating the upload.

[0020] In some embodiments, the technology described herein provides exception handling mechanisms during an upload process. These mechanisms include implementing a second eye process for reviewing error conditions and providing corrective actions to address the error conditions. In a non-limiting example, the error condition can be an error in a document image, and the corrective action can include re-routing the process to a second eye system for confirming the detected error with a correction. In a non-limiting example, the error condition can be a detected unexpected parameter in a document (e.g., missing or extra content in a document), and the corrective action can be a second human review of the document to verify the validity of the unexpected parameter.

[0021] Accordingly, the technology described herein addresses one or more technical problems existing in the field of online computer systems, particularly in automated document upload processes, which often lack mechanisms for handling exception or error conditions. This problem stems from the typical network communications between a document upload application installed on a user device and a backend system, where errors can occur within these network communications during a document upload process. This problem prevents such backend systems from efficiently processing document upload requests. In a backend system that processes millions of such requests, the effects of inefficiency can quickly accumulate. The technology described herein provides improvements to such systems and their ability to handle errors during document uploads. Accordingly, by modifying the communications between a document upload application and a backend system to accurately identify errors in a document upload process and handle these identified errors, one or more solutions described herein necessarily rooted in computer technology. As will be described in various embodiments of Figures 1-4 The technology described herein reduces or eliminates the problems of conventional document upload processes, as described in various embodiments.

[0022] Various embodiments of these features will now be discussed with reference to the corresponding drawings.

[0023] Figure 1 is an exemplary document upload environment 100 for providing exception handling capabilities to a document upload process in accordance with aspects of the present disclosure. In one example, the environment 100 includes a user device 110 and a backend system 120. Exception handling can be implemented to provide risk control for communications between the user device 110 and the backend system 120. Risk control is a strategy whose aim is to identify, assess, and prepare for any hazards, dangers, errors, and other possible losses or exposures that can interfere with the operation and objectives of the backend system 120. As shown, in some embodiments, the backend system 120 can include an error assessment module 126 that manages errors detected based on security rules during a document upload process.

[0024] By way of a few non-limiting examples, user device 110 can be a desktop, workstation, laptop, notebook, digital assistant, netbook, tablet, smartphone, mobile phone, smartwatch or other wearable device, home appliance, component of the Internet of Things, and / or embedded system, or any combination thereof. User device 110 can allow a user to perform various tasks including accessing, running, or otherwise interacting with applications. In one example, user device 110 can include a document upload application 112 that allows a document to be uploaded to backend system 120. Backend system 120 can include an image database 122, an OCR component 124, and an error evaluation module 126, which can further include a quota model 128. User device 110 can be connected to backend system 120 through a network connection 130, which can be implemented via a wireless communication network, a cable communication network, and / or any combination thereof, as will be apparent to those of skill in the relevant arts, without departing from the spirit and scope of the present disclosure. In embodiments, network connection 130 between user device 110 and backend system 120 can be implemented via a LAN, WAN, PAN, VPN, or other network, and can include the Internet. Network connection 130 can also be a connection via a cloud-based network. In some embodiments, communication between user device 110 and backend system 120 is encrypted and transmitted over a secure network connection 130.

[0025] Document upload application 112 can be configured to display a graphical user interface on a display of user device 110. The graphical user interface can provide an interface for receiving and capturing an image of a document to be uploaded to backend system 120. Document upload application 112 can include a camera function that utilizes a camera (not shown) of user device 110 to capture one or more images of a document to be uploaded to backend system 120. However, any image capture device (e.g., a scanner) can be used, as will be apparent to one of ordinary skill in the art.

[0026] In a conventional upload process, any anomaly detected by the conventional upload application or by the conventional backend system would cause the upload process to terminate and force the user to submit a subsequent upload request. For example, the conventional backend system can request that the user capture the image a second time. Alternatively, an anomaly detected in any of the uploaded images or an anomaly based on certain conditions associated with the document and / or the user can cause the conventional backend system to terminate the upload process and prevent the document from being uploaded.

[0027] As previously described, the techniques of the present disclosure provide a number of improvements over this conventional upload process. Instead of terminating the document upload process, the error evaluation module 126 can communicate with the document upload application 112 to provide an exception handling routine for handling different errors detected via the on-the-fly OCR routine during the upload process. For example, to address errors related to image errors detected by the OCR component 124 (e.g., smudging, unreadable text), the error evaluation module 126 can initiate a routine that includes triggering a prompt at the document upload application 112, receiving a user response via the document upload application 112 indicating whether to initiate a secondary review of the uploaded image in response to the prompt, and subsequently initiating a secondary review of any uploaded image in coordination with the document upload application 112. In some embodiments, the secondary review can be automatically triggered without requiring a user response. The automatic triggering can be based on the type of error detected in the uploaded image or the number of errors. As another example, if the error relates to a discrepancy of the document from a standard or expected format determined by the document type of the document, a secondary review can be initiated to verify the validity of the document based on the discrepancy. As another example, to address exceptions related to upload limits associated with the user (e.g., a maximum limit amount for checks), the error evaluation module 126 can utilize the limit model 128 to determine whether to dynamically approve an over-limit handling of the limit amount of the document and allow the upload process to proceed.

[0028] In some embodiments, the document upload application 112 can be configured with separate application programming interface (API) calls for initiating different aspects of the document upload process. For example, the document upload application 112 can include an OCR API call that is triggered by the document upload application 112 upon receiving one or more document images and an exception handling API call that is triggered upon receiving a signal from the error evaluation module 126.

[0029] For the OCR API call, the document upload application 112 can capture a first image of the document (such as a first page or front side of the document) and a second image of the document (such as an additional page or back side of the document). The document upload application 112 can be configured to trigger the OCR API call immediately upon capturing the first image or wait for a subsequent image (such as the second image to be captured). The document upload application 112 can be further configured to modify one or more of the document images to include an OCR request tag as part of the OCR API call. The backend system 120 can be configured to detect the OCR request tag and transmit the document image to the OCR component 124 upon detecting the OCR request tag.

[0030] In some embodiments, the OCR component 124 receives a document image transmitted from the document upload application 112, performs OCR on the document image, and can pass the results of the OCR process to the error assessment module 126 and / or the limit model 128. The OCR component 124 can detect the contents of the document, which can be used for further processing. In embodiments involving a check to be deposited into a user account, the OCR component 124 can detect the monetary value, the routing number of the issuing bank, and the account number identifying the account from which the check’s funds are to be withdrawn. In other embodiments, the uploaded document can be an application file for opening an account or for applying for a financial instrument such as a credit or debit card, or a user’s identification such as a driver’s license. Such documents can have standard formats and fields and expected information in each field such as a name and address. The OCR component 124 can be used to detect the information in the fields of the document.

[0031] In some embodiments, after the backend system 120 receives the document image(s) from the document upload application 112 and performs OCR on the document image, the document upload process can include additional processing steps. Examples of additional processing steps include performing actions associated with the document, such as storing the document in a storage location and performing an exception handling routine based on any detected exceptions in the document.

[0032] For the exception handling API call, the document upload application 112 can receive a signal from the error assessment module 126 indicating that an error has been detected during the upload process. In some embodiments, the signal can include information about the error and actions that can be taken by the document upload application 112. For example, the actions can cause a prompt to be displayed by the document upload application 112. The prompt can enable input from a user of the document upload application 112 to trigger a secondary review of the document upload. In some embodiments, the signal can include information about the results of an error mitigation process at the backend system 120 and any instructions associated with continuing or terminating the upload process in response to the results of the error mitigation process. In some embodiments, the signal can include a notification of the results of a limit approval.

[0033] In some embodiments, backend system 120 can be implemented as various centralized or decentralized computing devices. For example, backend system 120 can be implemented as one or more servers, grid computing resources, virtualized computing resources, cloud computing resources, peer-to-peer distributed computing devices, server farms, or combinations thereof. Backend system 120 can be centralized in a single device, distributed across multiple devices within a cloud network, distributed across different geographic locations, or embedded within a network. Backend system 120 can communicate with other devices, such as user devices 110. Components of backend system 120, such as image database 122, OCR component 124, and error evaluation module 126, can be implemented within the same device (such as when backend system 120 is implemented as a single device) or as separate devices (such as when backend system 120 is implemented as a distributed system using components connected via a network).

[0034] Image database 122 can be implemented as a network storage resource (e.g., Amazon S3®, Google Cloud Storage®, Microsoft Azure® Storage, etc.) and can be configured to store document images received from the document upload application. Storage Area Network (SAN), Network File System (NFS), etc.) and can be configured to store document images received from the document upload application.

[0035] OCR component 124 can be implemented as a module that employs optical character recognition techniques to detect content in images of documents uploaded to backend system 120.

[0036] Error evaluation module can be implemented as a computing device that detects any errors associated with a document during the document upload process. In some embodiments, the error evaluation can be based on signals from OCR component 124 that indicate errors in images uploaded to image database 122 during the current upload process. In some embodiments, the signals from OCR component 124 can indicate that the content of a document lacks expected parameters or has unknown parameters as compared to a standard format based on the document type of the document. For example, a check can have a standard format that specifies a routing number and an account number, and a document that lacks one of these parameters can be identified as anomalous, triggering a secondary review.

[0037] In some embodiments, the error evaluation can be based on the detected content being transmitted as input by the OCR component 124 to a limit model 128, which can be implemented as a machine learning model trained for risk detection of documents to be uploaded to a user account associated with the backend system 120. That is, given a particular document and its document content, the limit model 128 is trained to determine the presence of a risk involved in allowing the document to be uploaded to the user account. In embodiments where the document is a check, the limit model 128 can be trained to generate a risk evaluation based on the user account to which the check is to be deposited and the check content, such as the check amount, account number, and routing number. When the detected check amount is greater than a preset threshold amount (e.g., defined by user-specific rules), the risk evaluation can reflect a confidence score as to whether to approve the upload for the detected check amount. Certain factors, such as the difference between the check amount and the preset threshold amount, the identity of the issuing bank identified by the routing number, and the historical activity of the user associated with the account, can increase the risk. For example, a check amount that is much greater than a certain threshold can increase the risk, a bank indicated by the routing number can be less reputable (e.g., based on a reputation ranking of known banks), and a newer user account can be at a higher risk due to lack of activity history and trend information for the user account. One or more of these factors can be considered as part of generating a confidence score for the upload. Once generated, the error evaluation module 126 can compare the confidence score to a threshold to determine whether to approve or reject the upload. In additional embodiments, the approval of the upload by the error evaluation module 126 can be a one-time approval (e.g., for the current upload request), a time-based approval (e.g., for a particular time period, such as a week), and / or can be tied to a particular user (e.g., such that future upload requests from the user can be automatically approved).

[0038] As described above, when the content of the document is determined to be greater than a preset confidence level threshold, the output of the limit model 128 can be a confidence score representing a level of risk for approving the upload of the document. The confidence level threshold represents an acceptable level of risk associated with the user or group of users and can be set by the error evaluation module 126. Increasing the confidence level threshold increases the acceptable level of risk associated with approving the upload request. That is, the confidence level threshold is inversely proportional to the acceptable risk level, where the higher the confidence level threshold, the lower the risk of the backend system approving the upload request (e.g., allowing the upload of a document specifying a monetary amount greater than the threshold amount). The error evaluation module 126 can utilize the output to make a decision regarding whether to continue the upload process. The inputs to the limit model 128 can include the information detected by the OCR component 124 as well as user information of the user initiating the upload. In some embodiments, the information detected by the OCR component 124 can be used to obtain additional information to use as inputs to the limit model 128. For example, the OCR component 124 can provide the routing information detected on the check, and the error evaluation module 126 can use the routing information to obtain the issuing bank of the check based on the routing number, which can then be used as additional inputs to the limit model 128 to generate the confidence score.

[0039] Figure 1 The modules described in the detailed description can be implemented as instructions stored on a non-transitory computer readable medium to be executed by one or more computing units, such as a processor, a special purpose computer, an integrated circuit, an integrated circuit core or combination thereof. The non-transitory computer readable medium can be implemented using any number of memory units, such as volatile memory, non-volatile memory, internal memory, external memory or a combination thereof. The non-transitory computer readable medium can be integral to the environment 100 or installed as a removable part of the environment 100.

[0040] The environment 100 can be used to implement various fields of document upload techniques. These fields include financial applications, security applications, and the like, where the documents uploaded into a user account can be subject to fraudulent activity. For example, when handling important documents, such as driver’s licenses, checks, financial documents, and the like. The environment 100 allows for the pre-computation of a determined risk associated with these documents and user accounts prior to completing the document upload process, and also allows for security steps to be taken to increase the security of the document upload process prior to completing the upload.

[0041] Operational methods

[0042] Figure 2A And Figure 2B are example methods of operating the environment 100 in accordance with aspects of the present disclosure to perform instant OCR of document images to be uploaded to a user account and provide the ability to handle exceptions to errors that occur during the upload process.

[0043] Figure 2A is an example method 200A for handling errors associated with quotas related to uploaded documents. Figure 1 For non-limiting example, Figure 2A One or more of the processes described may be performed by one or more devices of the environment 100. In such an embodiment, one or more devices of the environment 100 may execute code in memory to perform certain steps of the method 200A. Figure 2A The method 200A will be discussed below as being performed by one or more components of the environment 100, but other devices not shown may store code and thus perform the method 200A by directly executing the code. Accordingly, the following discussion of the method 200A will refer to Figure 1 As an exemplary non-limiting embodiment of method 200A, the method 200A is provided. Furthermore, the method 200A may be performed by processing logic that may include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executed on a processing device), or a combination thereof. It should be understood that not all steps are required to perform the disclosure provided herein. Furthermore, as will be understood by one of ordinary skill in the art(s), some of these steps may be performed simultaneously or in parallel. Figure 2A Different orders than those shown are executed.

[0044] At step 202, the document upload application 112 installed on the user device 110 captures a document image of a document to be uploaded to the backend system 120. In some embodiments, the document upload application 112 may be configured with a parameter indicating the number of document images to be captured for the document to be uploaded. For example, the graphical user interface provided by the document upload application 112 may be configured to request the capture of an image of the front side of the document and an image of the back side of the document. In some embodiments, the document upload application 112 may be configured to utilize video functionality to capture multiple frames of the document (such as multiple frames of the front side of the document and multiple frames of the back side of the document). The document upload application 112 may receive a request to electronically initiate and execute the document upload process.

[0045] At step 204, the document upload application 112 transmits the document image (or multiple frames) to the backend system 120, which can cache the received image (or frames) in a temporary storage location (such as an S3 bucket using a cache control implementation). The backend system 120 can configure the cache to store the document image for a predetermined period of time (such as the duration of the upload process) before being purged.

[0046] At step 206, the document upload application 112 transmits an OCR request to the backend system 120 to perform OCR on the uploaded document images so that OCR can be performed during the document upload process. In some embodiments, the OCR request can be implemented as a request tag that is incorporated into one or more of the document images transmitted by the document upload application 112 as part of step 204. For example, the document upload application 112 can modify a front-side image or a back-side image of a check to include a request tag before transmitting the front-side image or the back-side image to the backend system. In embodiments, the request tag can be included in only one of the document images. In other embodiments, the OCR request can be implemented as a separate API call transmitted by the document upload application 112 after determining that all images of the document have been captured. In some embodiments, the request tag can be included within a frame of a set of frames representing the document.

[0047] At step 208, the backend system 120 performs OCR on the document images to detect the contents of the document after receiving the OCR request from the document upload application 112. For example, the OCR component 124 can detect a monetary amount specified in a check, a routing number associated with the issuing bank, and / or other account information specified on the document. In embodiments, the detected contents are needed to determine a risk associated with uploading the document to the user account, including updating the user account based on the contents of the document. In embodiments, the risk can be used to determine whether to approve the document upload when an anomaly or error is detected in the upload. As described above, one example of an anomaly is a monetary amount in a check that is greater than a threshold amount associated with the user and / or the user account to which the check is deposited. The risk can reflect a confidence score for allowing the upload process to proceed despite the anomaly or terminating the process. In embodiments where the backend system 120 receives multiple frames from the document upload application 112 as part of a video capture of the document, the OCR component 124 can use frames of the front-side of the document to verify data on the front-side of the document, such as by detecting the same information in one or more frames and performing a comparison of the detected information. For example, the OCR component 124 can detect an account number in each frame of the front-side of a check or a user name in each frame of an application file and determine whether the detected contents are the same between frames. The OCR component 124 can perform a similar process on each set of frames representing a different portion of the document, such as a second page or a back page of the document.

[0048] At step 210A, the error evaluation module 126 of the backend system 120 determines that an anomaly has occurred in uploading the document to the user account. In embodiments, the anomaly relates to whether the monetary amount specified by the check is greater than a threshold amount, i.e., whether the amount is greater than a savings limit associated with the user and / or the user account. In some embodiments, the savings limit or threshold amount can be determined based on a user-specific basis prior to the upload process and stored in memory of the backend system 120. In some embodiments, the savings limit can be determined or updated (if previously determined) dynamically during the upload process. In additional embodiments, the savings limit can be a global limit applicable to all users or a subset of users of the backend system 120.

[0049] At step 212A, if the error evaluation module 126 does not detect any anomalies, the upload process can proceed normally and the document can be uploaded to the backend system 120. For example, the check can be deposited into the user’s account.

[0050] At step 214A, if the error evaluation module 126 does detect an anomaly, such as one related to the monetary amount specified in the document, the error evaluation module 126 can trigger the limit model 128 to determine whether to proceed with the upload process, i.e., approve the limit increase. The limit model 128 can receive as input the content information detected by the OCR component 124, such as the monetary value and routing number, and information associated with the user account, such as account history and transaction history. The account history and transaction history are associated with the user account into which the document is being uploaded and can be obtained from a database connected to the backend system 120. In some embodiments, the output of the error evaluation module 126 is a risk assessment based on the provided inputs. In some embodiments, the risk assessment includes a confidence score or a value that can be compared to a threshold. For example, the confidence score can represent a ranking of the received input compared to training data used to train the limit model 128.

[0051] At step 216A, the error evaluation module 126 determines whether to approve the upload based on the risk assessment determined at step 214A. For example, the error evaluation module 126 can compare the confidence score to a threshold and, if the confidence score is greater than the threshold, can allow the upload to proceed at step 212A.

[0052] At step 218A, if it is determined that the confidence score is less than the threshold, the error evaluation module 126 can reject the upload, thereby terminating the upload process. The backend system 120 can cancel the document upload process, including purging the document image from the image database 122.

[0053] Figure 2Bis an example method 200B of processing errors associated with documents uploaded to the backend system 120. As described above with respect to Figure 1 , one or more processes described with respect to Figure 2B may be performed by one or more devices of the environment 100. In such embodiments, one or more devices of the environment 100 can execute code in memory to perform certain steps of the method 200B. While the method 200B of Figure 2B will be discussed below as being performed by one or more components of the environment 100, other devices not shown can store code and thus can perform the method 200B by directly executing the code. Accordingly, the following discussion of the method 200B will refer to the devices of Figure 1 as an example, non-limiting embodiment of the method 200B. Further, the method 200B can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executed on a processing device), or a combination thereof. It will be understood that not all of the steps need to be taken to perform the disclosure provided herein. Further, some of the steps can be performed concurrently, or in different orders than shown, as will be understood by one of ordinary skill in the art(s). Figure 2B

[0054] Steps 202-208 are performed in a similar manner as described above with respect to Figure 2A .

[0055] At step 210B, the error evaluation module 126 of the backend system 120 determines that the uploaded document has an irregular item, where the irregular item is based on one or more rules established by the OCR component 124. In some embodiments, the determination can be based on the OCR component 124 detecting an irregular item in the document image. Examples of errors or irregular items of the document include unreadable portions of the document image and non-standard formats of the document, such as missing account information. Examples of unreadable portions can include blurred or otherwise undetectable portions of the document image that cannot be read by the OCR component 124. Examples of non-standard formats can include missing / absent document features, such as routing or account numbers or other information expected in the document. Such features can not necessarily indicate that the document is fraudulent, such as a check issued by a state that lacks an account number but is still a legitimate document. The OCR rules can not necessarily be able to detect non-standard but still legitimate documents. Accordingly, a secondary review of such documents can be needed to approve the document for upload to the backend system 120.

[0056] ​The error evaluation module 126 can receive as input the OCR detected content and the results of comparing the detected content to the format rules of the document to be uploaded to the backend system 120. In this embodiment, the output of the error evaluation module 126 is an indication of whether there are irregularities in the document.

[0057] At step 212B, the error evaluation module 126 determines that there are no irregularities and proceeds normally with the upload process.

[0058] At step 214B, the error evaluation module 126 determines that there are irregularities and sends a prompt to the document upload application 112 whether to trigger a secondary review process of the document image to address the detected irregularities. The prompt can include a description of the irregularity and / or an annotated image of the irregularity. The user can then decide to end the upload process and start a new upload process to retake a new image of the document based on the provided information. For example, the prompt can show a portion of the document blocked by the user’s finger indicating that the image needs to be retaken. In some embodiments, the document upload application 112 can be bypassed and the secondary review can be triggered automatically based on certain conditions associated with the irregularity. For example, the type of irregularity, such as the document missing certain expected information, can require a secondary review because the user cannot correct the irregularity to determine whether the missing information is fatal to the upload process. For example, checks issued by certain countries lack an account number and can require a secondary review to verify the authenticity of such a document. The backend system 120 can detect the type of irregularity and determine whether to automatically trigger a secondary review without transmitting a prompt to the document upload application 112. In some embodiments, the secondary review process can be performed by the backend system 120, such as manually reviewed by an agent.

[0059] At step 216B, if the document upload application 112 receives input not to trigger the secondary review process, the document upload application 112 terminates the document upload process.

[0060] At step 218B, if the document upload application 112 receives input to trigger the secondary review process of the backend system 120, the document upload application 112 transmits a command to the backend system 120 that initiates the secondary review process by routing the document image to an agent. In some embodiments, the review can be performed by an agent at the backend system 120. In some embodiments, the agent is a human that manually reviews the document image and provides an approval to the backend system based on the manual review. In some embodiments, the agent is an automated robot that can perform a second OCR process on the document image to determine whether there are irregularities in the second review. If the secondary review process approves the document upload, the upload process continues at step 212B.

[0061] Figure 3 is a block diagram of a machine learning system according to some embodiments, including model training 302 for training a quota model and model usage 314 for using the quota model during an upload process. The machine learning system is a framework that trains the quota model 128.

[0062] The model training 302 includes using training data 304 to train a machine learning engine 310 for outputting a quota model 312. The machine learning engine 310 can include one or more servers (cloud or local) for processing text, such as words, phrases, or sentences, to identify relationships of the words of the training data 304 (e.g., within a sentence). For example, the training data 304 can include customer activity 306 and demographic activity 308 that can be provided as structured data (e.g., using known control structures). Machine learning involves a computer exploring how it can perform a task without being explicitly programmed to perform the task. Machine learning (ML) includes, but is not limited to, artificial intelligence, deep learning, fuzzy learning, supervised learning, unsupervised learning, etc. The machine learning engine 310 builds the quota model 312 based on the training data 304 in order to make predictions or decisions based on new data. For supervised learning, the computer is provided training data 304 with a desired output, such as an amount of risk associated with the training data 304, and the goal is to learn a general rule that maps the training data 304 to the desired output. In another example, for unsupervised learning, the machine learning engine 310 is not given labels and the machine learning engine 310 looks for structure in the training data 304. Unsupervised learning can be an end in itself (discovering hidden patterns in data) or can be a means toward an end (feature learning). The machine learning engine 310 can use various classifiers to map concepts associated with a particular language structure to capture relationships between the concepts and words / phrases / sentences.

[0063] The training data 304 can include customer activity 306 and demographic activity 308. The customer activity 306 can include historical customer data about previous document upload requests. This data can include information about the documents that were uploaded and the results of the uploads (e.g., whether the document was determined to be fraudulent, whether the document upload failed). In some embodiments, previous document upload requests are associated based on the type of upload. For example, the type of upload can include uploading checks with check amounts that exceed a preset threshold and uploading non-standard documents that lack expected document information. The demographic activity 308 can include data associated with check amounts, customers, and different document providers. For example, demographic data for check amounts can include data about documents with similar check amounts (e.g., checks that exceed a preset threshold of $10, checks that exceed a preset threshold of $50, checks that exceed a preset threshold of $100). Demographic data for customers can include data about similar customers (e.g., customers in the same age group, customers with similar bank account activity). Demographic data for document providers includes data about the providers (such as their reputation).

[0064] The training data 304 can include hundreds or thousands of activities related to free-form documents that were previously uploaded to the backend system 120. The underlying model approach can utilize any of GMM HMM (Gaussian Mixture Modeling and Hidden Markov Modeling), Ngram language modeling, and deep neural networks (DNN). Lower error rates can be achieved through continued training and fine-tuning of the model, such as using feedback 326 as a result of the decisions 324 made by the quota model 312. In other words, the output of the current decision can be used to further train the quota model 312. In some embodiments, the machine learning engine 310 is supervised, outputting control associated with input data based on the customer activity 306 and the demographic activity 308. For example, "checks that exceed a preset savings threshold of $50 from customers with a certain pattern of activity and from XYZ issuing bank" are provided with a corresponding answer of known previous control (such as a savings failure). This process is repeated for hundreds or thousands of controls. Although described in an example embodiment for supervised learning, unsupervised learning can be used instead without departing from the scope of the technology described herein.

[0065] In one embodiment, once the limit model 312 is trained, it can be used for subsequent uploads to determine whether to approve or reject the subsequent uploads involving checks with an amount greater than a preset threshold. The model usage 314 can include an upload process 316 that results in the check information 318 and user information 320 being sent to the backend system 120, which are fed as inputs to the limit model 312. The check information 318 includes information from the document that is uploaded and detected by the OCR component 124, such as the account number, routing number, and amount of the check. The user information 320 includes information about the user requesting the upload, and can include information about the user, such as the user’s tenure (e.g., how long the user has been a customer of the backend system 120), the user’s activity (e.g., previous requests involving documents that exceed the preset threshold), and the user’s reputation (e.g., whether the user’s account is in good standing, whether the user’s account was previously in bad standing).

[0066] This information is provided to the limit model 312, which generates a confidence score 322 based on the provided inputs. In some embodiments, the confidence score 322 can reflect a ranking of the provided inputs with respect to the training data 304, where a higher ranking indicates a higher confidence that the requested upload will be successful and present minimal risk to the backend system 120 based on the provided inputs. A lower ranking can indicate a lower confidence, and the requested upload presents a greater risk to the backend system 120. In some embodiments, the confidence score 322 can be represented as a numerical value generated by the limit model 312 based on the provided inputs.

[0067] The limit model 312 can use the confidence score 322 to make a decision about whether to allow the upload to proceed. In some embodiments, the decision 324 is based on comparing the confidence score 322 to a preset threshold, which can be generated on a user-specific or global basis. A user-specific threshold can be associated with each user requesting a document upload, and the monetary amount specified by the document (if applicable) is compared to the user-specific threshold each time the user requests an upload. The user-specific threshold can be generated based on similar inputs received by the limit model 312, such as the user information 320 taking into account the user’s past history, reputation, and account information of the user. For example, a user with a reputation of maintaining an average balance above a certain amount (e.g., $10,000) can have a higher preset threshold than a user that maintains an average balance below that amount. Accordingly, different users can receive different preset thresholds. In some embodiments, the limit model 312 can utilize a global threshold that applies to all users accessing the backend system 120.

[0068] The decision 324 can be used as feedback 326 to the machine learning engine and a notification 328 of the upload process 316. The feedback 326 can further be used to train or optimize the training of the threshold model 312 to improve the accuracy and reliability of the confidence score generated by the threshold model 312. The feedback 326 can include information about the decision (e.g., whether to approve the upload request), information about the document (e.g., the monetary amount, the routing number, the account number), information about the user account into which the document was uploaded (e.g., the threshold for the monetary amount, the difference between the monetary amount detected in the document and the threshold), and information about the outcome of the decision 324 (e.g., whether the uploaded document was later rejected).

[0069] The notification 328 is provided back to the upload process 316 for display by the document upload application 112. The notification 328 displays the outcome of the decision 324 (i.e., whether the upload request was rejected or approved) and can include additional information such as the difference between the monetary amount and the threshold and other reasons for the decision 324.

[0070] System components

[0071] Various embodiments can be implemented, for example, using one or more well-known computer systems, such as the computer system 400 shown in FIG. 4. For example, one or more computer systems 400 can be used to implement any of the embodiments discussed herein, as well as combinations and sub-combinations thereof. Figure 4

[0072] The computer system 400 can include one or more processors (also known as central processing units, or CPUs), such as the processor 404. The processor 404 can be connected to a communication facility or bus 406.

[0073] The computer system 400 can also include user input / output device(s) 403 (such as a monitor, keyboard, pointing device, etc.) that can communicate with the communication facility 406 through user input / output interface(s) 402.

[0074] One or more of the processors 404 can be a graphics processing unit (GPU). In embodiments, a GPU can be a specialized electronic circuit designed to rapidly manipulate mathematically intensive applications. GPUs can have a parallel structure that is efficient at processing large blocks of data in parallel, such as mathematically intensive data common to computer graphics applications, images, video, etc.

[0075] The computer system 400 can also include a main or primary memory 408, such as a random access memory (RAM). The main memory 408 can include one or more levels of cache. The main memory 408 can have control logic (i.e., computer software) and / or data stored therein.​

[0076] Computer system 400 can also include one or more secondary storage devices or memory 410. For example, secondary memory 410 can include a hard disk drive 412 and / or a removable storage drive or device 414. Removable storage drive 414 can be a floppy disk drive, a magnetic tape drive, an optical disk drive, a tape backup device, and / or any other storage device / drive.

[0077] Removable storage drive 414 can interact with a removable storage unit 418. Removable storage unit 418 can include a computer-usable or computer-readable storage device on which computer software (control logic) and / or data can be stored. Removable storage unit 418 can be a floppy disk, a magnetic tape, an optical disk, a DVD, a Blu-ray® optical storage disk, and / or any other computer data storage device. Removable storage drive 414 can read from and / or write to removable storage unit 418.

[0078] Secondary memory 410 can include other devices, apparatuses, components, tools, or other methods for allowing computer system 400 to access computer programs and / or other instructions and / or data. For example, such devices, apparatuses, components, tools, or other methods can include a removable storage unit 422 and an interface 420. Examples of the removable storage unit 422 and the interface 420 can include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, a memory stick and USB port, a memory card and associated memory card slot, and / or other removable storage units and associated interfaces.

[0079] Computer system 400 can further include a communication or network interface 424. Communication interface 424 can enable computer system 400 to communicate and interact with any

[0080] The computer system 400 can also be any of the following (to name just a few non-limiting examples): a personal digital assistant (PDA), a desktop workstation, a laptop or notebook computer, a netbook, a tablet computer, a smartphone, a smart watch or other wearable device, a home appliance, a component of the Internet of Things, and / or an embedded system, or any combination thereof.

[0081] The computer system 400 can be a client or a server accessing or hosting any applications and / or data through any delivery model, including but not limited to a remote or distributed cloud computing solution; local or on-premises software (“on-premises” cloud-based solutions); a “Software as a Service” model (e.g., Content as a Service (CaaS), Digital Content as a Service (DCaaS), Software as a Service (SaaS), Managed Software as a Service (MSaaS), Platform as a Service (PaaS), Desktop as a Service (DaaS), Framework as a Service (FaaS), Backend as a Service (BaaS), Mobile Backend as a Service (MBaaS), Infrastructure as a Service (IaaS), and the like); and / or a hybrid model including any combination of any of the foregoing examples or other services or delivery models.

[0082] Any applicable data structures, file formats, and schemas in the computer system 400 can be derived from the following standards, including but not limited to JavaScript Object Notation (JSON), Extensible Markup Language (XML), Yet Another Markup Language (YAML), Extensible HyperText Markup Language (XHTML), Wireless Markup Language (WML), message packets, XML User Interface Language, or any other functionally similar representation, alone or in combination. Alternatively, proprietary data structures, formats, or schemas can be used, alone or in combination with known or open standards.

[0083] In some embodiments, tangible, non-transitory apparatuses or articles of manufacture comprising tangible, non-transitory computer-usable or -readable medium having control logic (software) stored therein are also referred to herein as computer program products or program storage devices. These include but are not limited to the computer system 400, the main memory 408, secondary memory 410, and removable storage units 418 and 422, and tangible articles of manufacture embodying any combination of the foregoing. Such control logic, program code, when executed by one or more data processing devices, such as the computer system 400, can cause such data processing devices to operate as described herein.

[0084] Based on the teachings of the disclosure provided herein, a person of ordinary skill in the relevant art will be able to contemplate different Figure 4It will be apparent to those skilled in the art that substantial equivalents of the embodiments of the disclosure described herein can be practiced in material other than that specifically described. Thus, all explicit or implicit descriptions of those processes, or of any device or element structure, are to be construed as potentially not being limiting.

[0085] The application is described above by functional building blocks illustrating the functional relationships between various circuitry and / or software components. For purposes of simplicity of the drawings, the boundary between the functional building blocks have been drawn as sharp lines in the drawings. Such sharp lines are not intended to represent a physical boundary but are intended to concretely illustrate the conceptual boundary between various circuitry and / or software components. One of ordinary skill in the art will recognize that the various circuitry and / or software components can share function, and that such sharing can be represented by divisional lines drawing discontinuities between them.

[0086] The foregoing description of specific embodiments will so fully reveal the general nature of the application that others can adapt and / or modify such specific embodiments for various applications and without the exercise of inventive faculty. Therefore, it is the intent of the inventors to be limited only by the scope of the disclosure presented by the language of the following claims whenever and wherever such claims issue from this application either alone, or in combination with other patent applications. Accordingly, there is no intention to limit the application to the expressly disclosed embodiments go so far as one of ordinary skill in the art would understand such to do so would suffice to encompass most if not all conceivable embodiments falling within the scope of the application. Accordingly, the specification is to be regarded in an illustrative manner and applications of the teaching can be directed to other ways and means.

[0087] The term "module" or "unit" mentioned in the disclosure can include software, hardware, or a combination thereof in aspects of the disclosure according to a context in which the term is used. For example, the software can be machine code, firmware, embedded code, or application software. Also for example, the hardware can be a circuit device, a processor, a special purpose computer, an integrated circuit, an integrated circuit core, or a combination thereof. Further, if a module or unit is written in a section of a system or apparatus claim, it is considered that the module or unit includes a hardware circuit device for the purpose and scope of the system or apparatus claim.

[0088] The modules or units in the following description of aspects can be coupled to each other as described or illustrated. The coupling can be direct or indirect, with or without intermediate items between the coupled modules or units. The coupling can be by physical contact or by communication between the modules or units.

[0089] The above detailed description of the disclosed system 100 and aspects thereof is not intended to be exhaustive or to limit the disclosed system 100 to the precise form disclosed. While specific examples of the system 100 were described above for illustrative purposes, various equivalent modifications are possible within the scope of the disclosed system 100, as those skilled in the relevant art will recognize. For example, while processes and methods were presented in a particular order, alternative implementations can perform routines having steps in a different order, or systems having processes or methods with steps can be implemented, and some processes or methods can be deleted, moved, added, subdivided, combined, or modified to provide alternative or sub-combinations. Each of these processes or methods can be implemented in a variety of different ways. Also, while processes or methods were often described above as being performed sequentially, these processes or blocks can instead be performed in parallel, or executed at different times, or in different orders.

[0090] These and other valuable aspects of the present disclosure arise from the technology state advancing to at least the next level. Although the disclosed aspects have been described as best modes or embodiments thereof, it will be understood that many modifications, substitutions, and alterations are possible by those skilled in the art in light of the description herein. Accordingly, all such modifications, substitutions, and alterations are intended to be included within the scope of the included claims. All matters herein or shown in the accompanying drawings are to be interpreted as illustrative only and not as limiting.

Claims

1. A system for providing exception handling during a document upload process of a document, the system comprising: a memory; and at least one processor coupled to the memory and configured to: receive, from an upload application on a user device, a request for a front image of the document and a back image of the document to be uploaded to a user account maintained by the system; detect content of the document based on performing an instant optical character recognition (OCR) of at least one of the front image of the document and the back image of the document; and initiate the exception handling based on the content of the document being greater than a preset threshold amount, wherein the exception handling comprises: sending, to a limit model, the content of the document and user information associated with the user device; receiving, from the limit model, a confidence score based on the content of the document; and determining whether to allow the document upload process to proceed based on the confidence score being greater than a second preset threshold amount.

2. The system of claim 1, wherein the content of the document comprises a monetary amount to be uploaded to the user account.

3. The system of claim 2, wherein the user information associated with the user device comprises a user transaction history for the user account.

4. The system of claim 3, wherein the confidence score is generated based on the user transaction history for the user account, the monetary amount, and the preset threshold amount.

5. The system of claim 2, wherein the processor is further configured to: determine a difference between the monetary amount to be uploaded to the user account and the preset threshold amount; and provide the difference to the limit model.

6. The system of claim 1, wherein the preset threshold amount is a monetary upload limit associated with the user account.

7. The system of claim 1, wherein the document is a check.

8. The system of claim 1, wherein the user account is a check account.

9. The system of claim 1, wherein the processor is further configured to: provide, to a machine learning model engine, training data for training the limit model.

10. The system of claim 9, wherein the training data comprises user activity and demographic activity.

11. A system for providing exception handling during a document upload process of a document, the system comprising: a memory; and at least one processor coupled to the memory and configured to: receive, from an upload application on a user device, a request for a front image of the document and a back image of the document to be uploaded to a user account; detect content of the document based on performing an instant optical character recognition (OCR) of at least one of the front image of the document and the back image of the document; and initiate the exception handling based on the content of the document comprising an exception of a standard format for a document type of the document, wherein the exception handling comprises: Sending a prompt to the document uploading application; In response to sending the prompt, receiving a request from the document upload application to trigger a secondary review process for the document during the document upload process; triggering the secondary review process based on receiving the request; and The document upload process is allowed to proceed based on receiving approval from the secondary review process.

12. The system of claim 11, wherein the anomaly is determined based on a difference between the content of the document and the standard format for the document type of the document.

13. The system of claim 12, wherein the discrepancy comprises at least one of: a missing account number, a missing routing number, and missing user information.

14. The system of claim 12, wherein the standard format for the document type includes expected information in the document.

15. The system of claim 11, wherein the anomaly comprises unreadable text in the content of the document.

16. The system of claim 11, wherein the document type comprises a check and the user account comprises a checking account.

17. A computer-implemented method for providing exception handling during a document upload process for a document, the method comprising: receiving, from an upload application on a user device, a request for a front image of the document and a back image of the document to be uploaded to a user account; detecting content of the document based on performing on-the-fly optical character recognition (OCR) on at least one of the front image of the document and the back image of the document; as well as Initiating the exception handling based on the content of the document being greater than a preset threshold amount, wherein the exception handling comprises: sending the content of the document and the user information associated with the user device to a quota model; receiving, from the quota model, a confidence score based on the content of the document; and A determination is made as to whether to allow the document upload process to proceed based on the confidence score being greater than a second preset threshold value.

18. The method of claim 17, wherein the content of the document includes a monetary amount to be uploaded to the user account.

19. The method of claim 18, wherein the user information associated with the user device includes a user transaction history for the user account.

20. The method of claim 19, wherein the confidence score is generated based on the user transaction history for the user account, the monetary amount, and the preset threshold amount.

Images