Document type detection during upload process

Abstract

A system for document type detection during a document upload process is disclosed. A character recognition process is performed on a document image uploaded to a back-end system. A character recognition process may identify different subsets of characters in a document that are used to identify a document type. Subsets of characters may include keywords and account characters, and each subset may be processed by a different character recognition process. One or both of the subsets of characters may be used to identify a document type, and the identified document type may be used to decide whether to continue the document upload process.

Description

Technical Field

[0001] Aspects relate to systems and methods for detecting document type using optical character recognition (OCR) during an upload process. Background Art

[0002] Currently, computer-based (e.g., laptop) or mobile-based (e.g., mobile device) technologies allow users to initiate a document upload process for uploading images or other electronic versions of documents to a back-end system (e.g., a document processing system) for various purposes. In some cases, the back-end system is configured to only accept certain types of documents for the upload process. For example, the back-end system may only support processing of specific financial documents involving negotiable funds (such as checks or traveler's checks) while not supporting processing of those involving non-negotiable funds (such as deposit slips or bills). For back-end systems that process millions of documents, speed and accuracy are critical when determining document types and whether they are supported by the back-end system. Summary of the Invention

[0003] Provided herein are system, method, and / or computer program product embodiments, and / or combinations and sub-combinations thereof, for detecting a document type during a document upload process.

[0004] An example method embodiment may include steps for detecting the document type of a document during a document upload process. The steps may include a backend system receiving an image including image data representing a document from a mobile device, and starting one or more character recognition engines for detecting information in the image data. For example, the backend system may include a first character recognition engine that performs a first detection process on the image data to detect keywords in the image data. The first detection process may identify or predict the document type of the document based on the keywords. The backend system may also include a second character recognition engine or communicate with a second character recognition engine. The backend system may receive a plurality of numeric characters or account characters identified in the image data from the second character recognition engine. The backend system may use the plurality of numeric characters to perform verification of the identified (predicted) document type as an accepted document type to verify that the identified document type is correctly detected. After verifying that the document type is supported, the backend system may allow the document upload process to continue. After verifying that the document type is not supported, the backend system may transmit a notification to the mobile device indicating that the identified document type is not an accepted document type, and may further cancel the document upload process.

[0005] Some aspects of the present disclosure have other steps or elements in addition to or in place of the steps or elements described above. For those skilled in the art, these steps or elements will become apparent by reading the following detailed description when reference is made to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate various aspects of the disclosure and, together with the description, further serve to explain the principles of the disclosure and to enable one skilled in the relevant art to make and use the disclosure.

[0007] Figure 1 An exemplary document upload environment for providing mid-course adjustment of a document upload process according to aspects of the present disclosure is provided.

[0008] Figure 2 is an example method of a document upload process including performing on-the-fly optical character recognition of the document in accordance with aspects of the present disclosure.

[0009] Figure 3 is an example structure of a device that may be used to implement a system according to aspects of the present disclosure.

[0010] In the drawings, like reference numbers generally refer to like or similar elements. Additionally, the leftmost digit(s) of a reference number generally identifies the drawing in which the reference number first appears. DETAILED DESCRIPTION

[0011] The following aspects are described in sufficient detail to enable those skilled in the art to make and use the disclosure. It is understood that other aspects are apparent based on the disclosure, and that system, process or mechanical changes can be made without departing from the scope of aspects of the disclosure.

[0012] In the following description, numerous specific details are given to provide a thorough understanding of various aspects. However, it is apparent that various aspects can be practiced without these specific details. In order to avoid obscuring aspects, some well-known circuits, system configurations, and process steps are not disclosed in detail.

[0013] The drawings illustrating various aspects of the system are semi-diagrammatic and not to scale. Some dimensions are for clarity and are shown exaggerated in the drawings. Similarly, while the views in the drawings are for ease of description and generally show similar orientations, for the most part, the depictions in the drawings are arbitrary. Generally, the system can operate in any orientation.

[0014] In addition to or in place of the steps or elements mentioned, some aspects have other steps or elements, which will become apparent to those skilled in the art upon reading the following detailed description with reference to the accompanying drawings.

[0015] System Overview and Features

[0016] Provided herein are method, system, computer program product embodiments, and / or combinations and sub-combinations thereof for providing utilization of optical character recognition (OCR) during a document upload process for detecting the type of an uploaded document.

[0017] In some embodiments, the technology described herein provides a system for detecting the type of document(s) uploaded to a backend system. In one embodiment, the backend system may be a remote check deposit platform that receives images of checks (front and back) from a client device (such as a mobile phone). Optical Character Recognition (OCR) can be used to identify keywords and other character information in the document, which is then used to identify the document's type. Examples of keywords include predefined alphanumeric strings such as "check," "money order," and "traveler's check," and examples of character information include an account number or routing number. Some information, such as character information, may be located in expected locations on the document, such as at the bottom portion of the document. In some embodiments, the system may use a combination of character recognition techniques in a two-step process to identify the document type during the document upload process. For example, the system may use keyword recognition technology to generate a prediction of the document type based on matching alphanumeric characters identified in the document, and then use account character recognition technology to identify other characters in the document (e.g., an account number) located in expected locations in the document to verify the prediction. As another example, the system may reverse the technique, using account character recognition technology to generate the prediction and keyword recognition technology to verify the prediction. During the document upload process, the user may be asked to upload one or more documents for various applications. For example, a document upload application may allow users to upload certain documents to a backend system regarding their user accounts (such as checks or traveler's checks). The backend system may implement restrictions on the types of documents that can be uploaded (such as non-negotiable documents like money orders or deposit slips). The backend system may use the techniques described herein to detect the document type during the document upload process and, based on the detection, continue or exit the document upload process.

[0018] The technology described herein solves one or more technical problems existing in the scope of online computer systems, and in particular has a document upload process that does not accurately and effectively identify the document type during the document upload process. This problem with conventional upload processes stems from the typical communication between the document upload application installed on the user device and the back-end system, wherein the document type detection may occur in a third-party system that is separate from the document upload application or the back-end system. The example computer system can utilize multiple components that work together to provide a document upload service. Examples of these components include a client device, a back-end system that implements an application program interface (API) (the application program interface implements communication between the client device and the back-end system), and a third-party system that can be configured to perform functions associated with the uploaded document (such as OCR and verification). In some embodiments, the client device can be implemented as any mobile device that has a document upload application installed. In some embodiments, the API can be implemented with multiple components that interface with the document upload application and process images and other information provided from the client via the document upload application.

[0019] The technology described herein provides improvements to the communication between the document upload application and the backend system and the documents to be identified during the document upload process. One or more of the solutions described herein must be derived from computer technology that modifies the communication between the document upload application and the backend system, and in particular how the document upload application and the backend system interact with each other. The technology described herein reduces or eliminates the need for Figures 1 to 3 The problems with the conventional document upload process are described in various embodiments.

[0020] Various embodiments of these features will now be discussed with respect to the corresponding drawings.

[0021] Figure 1 An exemplary document upload environment 100 is provided for detecting the type of uploaded documents during a document upload process (eg, in connection with a remote deposit) in accordance with aspects of the present disclosure. In one example, the environment 100 includes a user device 110 and a backend system 120.

[0022] In an exemplary embodiment, the user device 110 may be implemented as a mobile device (such as a smartphone or tablet). The user device 110 may include a document upload application 112 configured to connect to a backend system 120, which may include an image database 122, a keyword recognition engine 126a, an account character recognition engine 126b, and a document type model 128. The user device 110 may be connected to the backend system 120 via a network connection 130, which may be implemented via a network (such as the Internet or a cloud network).

[0023] The document upload application 112 may be configured to display a graphical user interface on the user device 110. The graphical user interface may provide the ability to receive images of documents to be uploaded to the backend system 120. The document upload application 112 may include a camera function that utilizes a camera (not shown) of the user device 110 to capture one or more images of the document being uploaded to the backend system 120.

[0024] The techniques of the present disclosure provide several improvements over conventional document type detection processes, in which the type of a document is detected after receiving document images of all documents being uploaded. Additionally, prior art systems typically rely on a backend system to transmit explicit instructions (e.g., to a third-party system) to perform the detection process. For example, when implemented in the context of a mobile deposit process, the third-party system would receive a document image and instructions identifying the document type. In the present disclosure, instead of waiting for instructions from a third-party system to initiate the document type detection process, the document upload application 112 can be configured to initiate the document type detection process at the user device 110, at the backend system 120, or a combination of both.

[0025] In embodiments where the backend system 120 performs a document type detection process, the document upload application 112 may initiate a document upload process that includes automatically transmitting the document's image data upon image capture and before receiving any user input to initiate the upload process or transmitting the image to the backend system 120. Automatically transmitting the document image to the backend system 120 enables the backend system 120 to initiate the document type detection process during the document upload process. The backend system 120 includes a document detector 124 that receives the image data, detects the document type of the document being uploaded, and indicates whether the detected document type is supported by the backend system 120. If the document type is not supported, the backend system 120 may return a message to the document upload application 112 indicating that the document is not supported. The backend system 120 may also send a signal to the document upload application 112 to cancel the upload process. If the document type is supported, the backend system 120 allows the upload process to continue.

[0026] The document detector 124 may include a keyword recognition engine 126a and an account character recognition engine 126b. The keyword recognition engine 126 may perform a keyword recognition process for identifying a predefined alphanumeric string in a document image. In some embodiments, the keyword recognition engine 126a is an optical character recognition (OCR) engine that recognizes alphanumeric characters in a document. The account character recognition engine 126b may perform an account character recognition process for identifying account characters in a document. In some embodiments, the account characters may be located at a predefined position in the document. In some embodiments, the character recognition engine 126b is configured to specifically identify account characters in the document (such as magnetic ink characters on a check), wherein the account characters may be characterized based on a specific position within the document. For example, a document may include a magnetic ink character recognition (MICR) line that includes digital characters identifying a routing number associated with the document. In some embodiments, the keyword recognition engine 126a and the character recognition engine 126b may be implemented as a single engine that performs the functions described above.

[0027] In an embodiment, when the uploaded document is a check, the backend system may receive one or more images of the document being uploaded to a user account maintained by the backend system, and the user account is associated with the document upload application. The backend system may include a component for recognizing characters of the document during the document upload process and detecting the document type based on the recognized characters. The user account may be a bank account (e.g., checking or savings) into which the funds specified by the check are to be deposited. There may be different subsets of characters recognized by the backend system requiring different character recognition tools. For example, a character recognition tool, such as an OCR tool, may be configured to identify a subset of characters associated with predefined alphanumeric characters that identify the document type, such as characters that specify that the document is a financial instrument type (such as a check, money order, deposit slip, traveler's check, or savings bond), to name a few examples. Another character recognition technology, such as a character recognition technology configured to detect account information, may be configured to identify another subset of characters in the document, which subset is configured in a specific format, such as a routing number and account number associated with the financial instrument of the issuing document. In some embodiments, a subset of characters may be identified as predefined characters that are matched to a predefined set of character strings (e.g., "check," "money order"). In some embodiments, another subset of characters may be identified as account characters that are used to identify an account associated with a document. Accordingly, in some embodiments, the subsets of characters may refer to different identifying information on a document. Examples of subsets include a subset that includes information associated with the document type, a subset that includes information associated with a user associated with the document to be uploaded, and a subset that includes information associated with a backend system that provides the document.

[0028] In some embodiments, the backend system 120 may implement rules for allowed or disallowed document types. For example, allowed document types may be certain financial document types, such as those involving transferable funds, and identified by certain keywords and account numbers on the document. Disallowed document types may be certain financial document types, such as those involving non-transferable funds, and identified by other keywords and an unknown or missing account number on the document.

[0029] In certain embodiments, document detector 124 can be implemented in a distributed manner, wherein keyword recognition engine 126a and account character recognition engine 126b are implemented on the separated equipment connected by a network. For example, keyword recognition engine 126a can be implemented on user device 110 or back-end system 120, and account character recognition engine 126b can be implemented on the third-party equipment that is connected to back-end system 120. In such an embodiment, account character recognition engine 126b returns the account character of detection for further processing, such as verifying the predicted document type generated by keyword recognition engine 126a. In certain embodiments, the account character of detection can be the form of a digital character string identifying an account associated with a document. In the example where the document is a check, the account character can identify the routing number associated with a bank.

[0030] In some embodiments, document detector 124 can perform keyword and account character recognition process in sequence. For example, the document type detection process can first start keyword recognition engine 126a to generate the document type of prediction. The prediction can be based on the mapping of keywords to document types maintained by back-end system 120. For example, keyword recognition engine 126a can retrieve the document type based on keyword usage mapping. Then, account character recognition engine 126b can identify the account characters in the document. Back-end system 120 can also maintain the mapping between account characters and document types. Account characters can also indicate that they are associated with document types. Document detector 124 can then perform the verification of the predicted document type from keyword recognition engine 126a based on the routing number identified by account character recognition engine 126b. As another example, document detector 124 can perform these processes in reverse, that is, first perform the account character recognition process, then use the output of the keyword recognition process as verification.

[0031] In some embodiments, the document detector 124 can perform keyword and account character recognition processes in parallel. The identified keywords and account characters can be used together to generate a prediction of the document type. For example, the backend system 120 can store a mapping of keywords and account characters to corresponding document types and can retrieve the document type based on the mapping. In an embodiment where the account character recognition engine 126b can be implemented on a separate device from the keyword recognition engine 126a, the document detector 124 can transmit image data to the account character recognition engine 126b and receive the routing number from the account character recognition engine 126b.

[0032] In some embodiments, the document detector 124 can only perform one character recognition process of the character recognition process. For example, the back-end system 120 may only require that a predefined string be compared with a list of known strings or that the account character be identified, but not both, to identify the document type. The back-end system 120 can establish a threshold confidence level to determine whether the document upload process requires two processes for performing the identification document type. For example, a first character recognition process (keyword recognition or account character recognition) can be performed, which results in providing a predicted document type during the document upload process. The prediction can be associated with a confidence level that the prediction is accurate. The confidence level can be compared with a threshold level to determine whether to perform the second character recognition process.

[0033] In some embodiments, the user device 110 and the back-end system 120 can perform different character recognition processes of the document type detection process in a distributed manner. For example, the document upload application 112 can automatically perform a keyword recognition process for identifying predefined alphanumeric characters in a document image and compare them with predefined strings associated with different document types. The document upload application 112 can identify predefined strings within the document and compare the identified strings with a list of known strings that identify the document as a check, money order, or traveler's check. In embodiments where two processes are to be performed, the back-end system 120 can then perform an account character recognition process to verify the predicted document type generated by the keyword recognition process. In some embodiments, the account character recognition process can be performed first to generate a prediction, and the keyword recognition process can be performed to verify the predicted document type generated by the account character recognition process.

[0034] In some embodiments, the document upload application 112 can be configured with application program interface (API) calls for different character recognition processes for starting the document type detection process. For example, the document upload application 112 can transmit a first API call to start a keyword recognition process and a second API call to start an account character recognition process. After at least one document image has been transmitted to the back-end system 120, the document upload application 112 can transmit API calls at any point during the upload process. For example, after the document upload application 112 transmits the first document image of the document (e.g., the front of the document) to the back-end system 120, the document upload application 112 can then transmit API calls to start the keyword recognition process or the account character recognition process, or it can transmit the second document image of the document (e.g., the back of the document) and wait to transmit any API calls until all images of the document are transmitted to the back-end system 120. For the API calls (API calls for the keyword recognition process or API calls for the account character recognition process), the document upload application 112 can capture the first image of the document (such as the first page or the front of the document) and the second image of the document, such as an additional page or the back of the document. The document upload application 112 may be configured to trigger the API call immediately upon capturing the first image or to wait for a subsequent image, such as a second image to be captured.

[0035] In some embodiments, the document upload application 112 can be further configured to modify one or more document images of the document image to include a request tag detected by the back-end system 120. When the request tag is detected, the back-end system 120 can transmit any received document image to the keyword recognition engine 126a or the account character recognition engine 126b. Based on the settings in the request tag, the settings stored in the back-end system 120, or based on historical data indicating which engine is more accurate in detecting document types, the back-end system 120 can decide which engine (and therefore whether to perform keyword recognition or account character recognition first) to receive the document image first. The technology disclosed herein can dynamically add a request tag to one or more document images in the document image, and transmit these images with the request tag to the back-end system, and the request tag indicates that document type detection is performed immediately when one or more document images in the document image are received.

[0036] In some embodiments, the first recognition engine selected by the backend system 120 receives the document image(s) transmitted from the document upload application 112, performs character recognition (keywords or account characters) on the document image, and generates a predicted document type based on the result of the character recognition. In some embodiments, the backend system 120 can determine whether the predicted document type is sufficient or whether it needs to be verified by the second recognition engine.

[0037] The keyword recognition engine 126a performs character recognition on the document image to identify predefined keywords. The keyword recognition engine 126a may maintain a table that associates predefined keywords with document types. For example, a predefined keyword may be "check," "money order," or "traveler's check," which is associated with the corresponding document type, check, money order, or traveler's check. The backend system 120 may update this table to include more than one predefined keyword associated with a particular document type. For example, there may be more than one keyword associated with a check, more than one keyword associated with a money order, and so on.

[0038] The account character recognition engine 126b performs account character recognition on the document image to identify the account information on the document. Account information is different from keywords because they generally relate to specific user accounts and are usually different (or predefined) for each document. On the contrary, for each document type, keywords are generally the same. For example, checks can share the same keywords (e.g., "checks"), but have different account information. For each document, account information can also be located at a predefined position, such as the bottom of the document. In some embodiments, the account character recognition engine 126b can maintain a table that associates account information with document types. In some embodiments, the account character recognition engine 126b can submit a query including the account information identified to a database, which can identify the document type based on the account information, and then it can respond to a query indicating the document type identified.

[0039] In various aspects, back-end system 120 can be implemented as one or more servers. Back-end system 120 can be implemented as various centralized or decentralized computing devices. For example, back-end system 120 can be a mobile device, a laptop computer, a desktop computer, a grid computing resource, a virtual computing resource, a cloud computing resource, a peer-to-peer distributed computing device, a server farm or a combination thereof. Back-end system 120 can be concentrated in a single device, distributed across multiple devices in a cloud network, distributed across different geographical locations or embedded in a network. The components of back-end system 120 (such as image database 122, keyword recognition engine 126a, account character recognition engine 126b and document type model 128) can be implemented in the same device (such as when back-end system 120 is implemented as a single device) or implemented as separated devices (such as when back-end system 120 is implemented as a distributed system with components connected via a network).

[0040] The image database 122 in the backend system 120 may be implemented as a network storage resource (e.g., Amazon Storage Area Network (SAN), Network File System (NFS), etc.), and is configured to store document images received from a document upload application.

[0041] The document type model 128 in the backend system 120 can be a machine learning model that is trained to assist the keyword identification engine 126a by providing information about possible keywords corresponding to different document types and the locations of those keywords within a document. The document type model 128 can be trained to assist the keyword identification engine 126a in identifying relevant keywords associated with a document type. The document type model 128 can be trained based on images provided by the document upload application 112 or by multiple document upload applications 112 installed across multiple user devices. The machine learning model 128 can maintain and update a list of keywords associated with each document type. In some embodiments, the document type model 128 can also be trained to identify the locations of keywords within each document type. For example, the keyword "check" may appear in a specific location on a check. In some embodiments, the document type model 128 can also identify the locations of keywords based on the account information listed on the document. For example, the keyword "check" may appear in a specific location on a check for one bank, but may appear in a different location on a check for another bank. The document type model 128 can provide some or all of this information—a mapping of keyword(s) to document type, the location of keywords based on document type, the location of keywords based on account information on the document, and the location of keywords based on both document type and account information—to the keyword identification engine 126a, which can use this information to more accurately identify keywords within documents. The backend system 120 can continuously update the mapping by training the document type model 128 based on new image data received from multiple document upload applications installed across different user devices.

[0042] Figure 1 The components described in the embodiment of the present invention may be implemented as instructions stored on a non-transitory computer-readable medium to be executed by one or more computing units (such as a processor, a special-purpose computer, an integrated circuit, an integrated circuit core, or a combination thereof). The non-transitory computer-readable medium may be implemented using any number of memory units (such as volatile memory, non-volatile memory, internal memory, external memory, or a combination thereof). The non-transitory computer-readable medium may be integrated as part of the environment 100 or installed as a removable part of the environment 100.

[0043] Environment 100 can be used in a variety of fields that implement document upload technology. These fields include financial applications, security applications, and the like, where documents are being uploaded into user accounts that may be subject to fraudulent activity. For example, when processing important documents such as driver's licenses, checks, financial documents, and the like, environment 100 allows for more accurate determination of the document type of the uploaded document during the document upload process.

[0044] How to operate

[0045] Figure 2 A is an example method 200 of the operating environment 100 performing document type detection of a document image being uploaded to a user account maintained by the backend system 120 according to aspects of the present disclosure. Figure 1 For non-limiting example, one or more devices of environment 100 may perform Figure 2 In such an embodiment, one or more devices of environment 100 may execute code in memory to perform certain steps of method 200. Figure 2 The method 200 will be discussed below as being performed by one or more components of the environment 100, but other devices not shown may store code and thus perform the method 200 by directly executing the code. Accordingly, the following discussion of the method 200 will refer to Figure 1 As an exemplary non-limiting embodiment of method 200, a device is provided. Moreover, method 200 may be performed by processing logic, which may include hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions executed on a processing device), or a combination thereof. It should be understood that not all steps are required to perform the disclosure provided herein. Further, as will be understood by one of ordinary skill in the art(s), some of the steps may be performed simultaneously, or in parallel. Figure 2 The different orders shown are executed.

[0046] At step 202, the document upload application 112 installed in the user device 110 receives image data of a document (such as by controlling a camera on the user device 110). The image data represents one or more images of the document required by the back-end system to complete the document upload process. In some embodiments, the image data may include front and back captures of a single-page document, such as the front and back of a check. In some embodiments, the image data may include multiple images of pages of a multi-page document or different documents that are uploaded together. In some embodiments, the document upload application 112 may be configured with a parameter indicating the number of document images to be captured of the document being uploaded. For example, the graphical user interface provided by the document upload application 112 may be configured to request the capture of both a front image of the document and a second image of the document based on the number of images required for the back-end system 120 to process the document.

[0047] At step 204 , the document upload application 112 transmits the image data to the backend system 120 , which may cache the received image in a temporary location (such as an S3 bucket using a cache control implementation).

[0048] At step 206a, the document upload application 112 transmits a document type request with image data to the backend system 120 to detect the document type based on the image data uploaded at step 204. In some embodiments, the document upload application 112 may modify the image data to include a document type request detectable by the backend system 120. The document upload application 112 may transmit the modified image data of the document to the backend system 120. For example, before transmitting the front or back image to the backend system, the document upload application 112 may modify the image data of the front or back of a check to include a request tag. In some embodiments, the request tag may be included in only one of the document images. As an alternative to step 206a, the document upload application 112 may perform step 206b, in which the document upload application 112 transmits the document type request as a separate API call. The backend system 120 may receive the document type request (received as part of the image data at step 206a or as a separate API call at step 206b) and initiate document type detection.

[0049] At step 208, document detector 124 can start keyword character recognition engine 126a and account character recognition engine 126b in the order of sequence.For example, document detector 124 can start the first character recognition process implemented by keyword character recognition engine 126a or account character recognition engine 126b.Based on the request received from the document upload application, based on the preference stored in the memory of back-end system 120 or based on which process has provided the prediction of more accurate document type (for example, based on the historical data stored in back-end system 120, determine), back-end system 120 can determine whether to execute keyword or account character recognition process.No matter which engine is selected first, the result of step 208 is the document type of prediction.Keyword character recognition engine 126a provides the document type of prediction based on (multiple) keywords in the identification image data and uses mapping to retrieve the document type associated with (multiple) keywords.Account character recognition engine 126b provides the document type of prediction based on the account characters in the identification image data.Account characters can correspond to the user account associated with a specific document type. For example, routing numbers on certain financial documents are associated with a particular document type, such as checks, money orders, or savings bonds.

[0050] At step 210, the document detector 124 launches the second character recognition engine, which is the engine that was not selected at step 208. The output of the second character recognition engine is used to verify the predicted document type from step 208.

[0051] In some embodiments, the document detector 124 may perform steps 208 and 210 in parallel rather than sequentially. Without waiting for the results of the keyword character recognition and account character recognition processes, the document detector 124 may transmit the image data to both the keyword character recognition engine 126a and the account character recognition engine 126b.

[0052] At step 212, document detector 124 determines whether the detected document type is accepted by backend system 120. This determination may include verifying that the document type generated during step 208 is an accepted document type using account characters. For example, if step 208 predicts that the document is a check based on keywords located in the image data, step 212 may verify the prediction using account characters (such as a routing number) identified in step 210. The term "accept" in this context may refer to whether backend system 120 allows the document type to be uploaded, such as to a user account maintained by backend system 120.

[0053] At step 214, document detector 124 determines that the detected document type is not accepted by backend system 120. Backend system 120 may transmit a notification and a signal to end the current document upload process to user device 110. The notification may specify that the identified document type is not an accepted document type.

[0054] At step 216, the document detector 124 determines that the detected document type is accepted by the backend system 120. The backend system 120 may allow the document upload process to continue, which includes allowing the document to be uploaded to the backend system 120 and performing any additional processing of the document required by the backend system 120. The additional processing may include updating a user account associated with the document upload application with the contents of the uploaded document.

[0055] In some embodiments, when step 208 is performed by the keyword character recognition engine 126a, step 218 is performed, which includes the document type model 128 providing a trained model to the keyword character recognition engine. The trained model may include mappings between any combination of (1) keywords to document types, (2) keyword positions to document types, (3) keyword positions based on account characters (provided by the account character recognition engine 126b), and (4) keyword positions based on account characters. The keyword character recognition engine 126a can utilize the provided mappings to more efficiently detect keywords in the image data and generate predicted document types based on the detected keywords. The mappings can further increase the confidence in the predicted document types.

[0056] As a non-limiting example, the trained model may include mappings for certain keywords to the "check" document type and the location(s) of those keywords for checks from different issuing banks (i.e., a check from one bank may have the keyword(s) identifying the document type in different locations than a check from another bank). In some embodiments, the trained model may also include a mapping based on account characters, such as a routing number identifying an account associated with a document. The mapping may include information linking the keyword to the account identified by the routing number (e.g., provided by the account character recognition engine 126b).

[0057] System components

[0058] For example, various embodiments may utilize one or more well-known computer systems such as Figure 3 For example, one or more computer systems 300 can be used to implement any of the embodiments discussed herein, as well as combinations and sub-combinations thereof.

[0059] Computer system 300 may include one or more processors (also known as central processing units, or CPUs), such as processor 304. Processor 304 may be connected to a communication infrastructure or bus 306.

[0060] Computer system 300 may also include user input / output device(s) 303 , such as a monitor, keyboard, pointing device, etc., which may communicate with communication infrastructure 306 via user input / output interface(s) 302 .

[0061] One or more of the processors 304 may be a graphics processing unit (GPU). In an embodiment, a GPU may be a processor that is a specialized electronic circuit designed to process mathematically intensive applications. A GPU may have a parallel architecture that efficiently processes large blocks of data in parallel, such as mathematically intensive data commonly used in computer graphics applications, images, videos, and the like.

[0062] The computer system 300 may also include a main or primary memory 308, such as random access memory (RAM). The main memory 308 may include one or more cache levels. The main memory 308 may have stored therein control logic (i.e., computer software) and / or data.

[0063] Computer system 300 may also include one or more secondary storage devices or memories 310. For example, secondary storage 310 may include a hard drive 312 and / or a removable storage device or drive 314. Removable storage drive 314 may be a floppy disk drive, a tape drive, an optical drive, an optical storage device, a tape backup device, and / or any other storage device / drive.

[0064] Removable storage drive 314 can interact with removable storage unit 318. Removable storage unit 318 may include a computer-usable or readable storage device that has computer software (control logic) and / or data stored thereon. Removable storage unit 318 may be a floppy disk, magnetic tape, optical disk, DVD, optical storage disk, and / or any other computer data storage device. Removable storage drive 314 can read from or write to removable storage unit 318.

[0065] Secondary memory 310 may include other means, devices, components, tools, or other methods for allowing computer programs and / or other instructions and / or data to be accessed by computer system 300. For example, such means, devices, components, tools, or other methods may include a removable storage unit 322 and an interface 320. Examples of removable storage unit 322 and interface 320 may include a program cartridge and cartridge interface (such as found in video game devices), a removable memory chip (such as an EPROM or PROM) and an associated slot, a memory stick and a USB port, a memory card and an associated memory card slot, and / or other removable storage units and associated interfaces.

[0066] The computer system 300 may also include a communication or network interface 324. The communication interface 324 may enable the computer system 300 to communicate and interact with any combination of external devices, external networks, external entities, and the like (individually and collectively represented by reference numeral 328). For example, the communication interface 324 may allow the computer system 300 to communicate with an external or remote device 328 via a communication path 326, which may be wired and / or wireless (or a combination thereof) and may include any combination of a LAN, a WAN, the Internet, and the like. Control logic and / or data may be transferred to and from the computer system 300 via the communication path 326.

[0067] The computer system 300 may also be any of a personal digital assistant (PDA), a desktop workstation, a laptop or notebook computer, a netbook, a tablet, a smartphone, a smartwatch or other wearable device, an appliance, part of the Internet of Things, and / or an embedded system, to name a few non-limiting examples, or any combination thereof.

[0068] The computer system 300 can be a client or server accessing or hosting any application and / or data through any delivery paradigm, including but not limited to remote or distributed cloud computing solutions; local or on-site software ("on-site" cloud-based solutions); "as a service" models (e.g., Content as a Service (CaaS), Digital Content as a Service (DCaaS), Software as a Service (SaaS), Management Software as a Service (MSaaS), Platform as a Service (PaaS), Desktop as a Service (DaaS), Framework as a Service (FaaS), Backend as a Service (BaaS), Mobile Backend as a Service (MBaaS), Infrastructure as a Service (IaaS), etc.); and / or hybrid models including any combination of the foregoing examples or other services or delivery paradigms.

[0069] Any applicable data structures, file formats, and schemas in the computer system 300 may be derived from standards including, but not limited to, JavaScript Object Notation (JSON), Extensible Markup Language (XML), Yet Another Markup Language (YAML), Extensible Hypertext Markup Language (XHTML), Wireless Markup Language (WML), message packages, XML User Interface Language (XUL), or any other functionally similar representations, alone or in combination. Alternatively, proprietary data structures, formats, or schemas may be used alone or in combination with known or open standards.

[0070] In some embodiments, a tangible, non-transitory device or article of manufacture that includes a tangible, non-transitory computer-usable or readable medium having stored thereon control logic (software) may also be referred to herein as a computer program product or program storage device. This includes, but is not limited to, computer system 300, main memory 308, secondary memory 310, and removable storage units 318 and 322, as well as tangible articles of manufacture embodying any combination of the foregoing. Such control logic, when executed by one or more data processing devices (such as computer system 300), may cause such data processing devices to operate as described herein.

[0071] Based on the teachings contained in this disclosure, it will be readily apparent to those skilled in the relevant art(s) how to use the Figure 3 It will be apparent that data processing devices, computer systems, and / or computer architectures other than those shown can be used to make and use the embodiments of the present disclosure. In particular, the embodiments can operate using software, hardware, and / or operating system implementations other than those described herein.

[0072] The present invention has been described above with the aid of functional building blocks that illustrate the specific functions and relationships of the present invention. For ease of description, the boundaries of these functional building blocks have been arbitrarily defined herein. Alternative boundaries may be defined as long as the specific functions and relationships are appropriately performed.

[0073] The description of the previous specific embodiments will fully reveal the general nature of the invention so that others can, by applying knowledge within the art, easily modify and / or adapt such specific embodiments to various applications without undue experimentation and without departing from the overall concept of the invention. Therefore, based on the teachings and guidance presented herein, such adaptations and modifications are intended to be within the meaning and range of equivalents of the disclosed embodiments. It should be understood that the wording and terminology herein are for purposes of description and not limitation, so that the terms or wording of this specification will be interpreted by the skilled person in accordance with the teachings and guidance.

[0074] The terms "module" or "unit" referred to in this disclosure may include software, hardware, or a combination thereof in aspects of this disclosure, depending on the context in which the terms are used. For example, software may be machine code, firmware, embedded code, or application software. Also, for example, hardware may be a circuit device, a processor, a special-purpose computer, an integrated circuit, an integrated circuit core, or a combination thereof. In addition, if a module or unit is written into the section of the system or device claim below, the module or unit is considered to include a hardware circuit device for the purpose and scope of the system or device claim.

[0075] In the following descriptions of various aspects, modules or units may be coupled to each other as described or shown. The coupling may be direct or indirect, with no or no intervening elements between the coupled modules or units. The coupling may be through physical contact or through communication between the modules or units.

[0076] The above detailed description and aspects of the disclosed environment 100 are not intended to be exhaustive or to limit the disclosed environment 100 to the precise form disclosed above. For illustrative purposes, although the specific examples of environment 100 are described above, those skilled in the relevant art will recognize that, within the scope of the disclosed environment 100, various equivalent modifications are possible. For example, although the processes and methods are presented in a given order, alternative implementations can perform routines with steps in different orders or adopt systems with processes or methods, and some processes or methods can be deleted, moved, added, subdivided, combined or modified to provide alternatives or sub-combinations. Each of these processes or methods or methods can be implemented in a variety of different ways. Moreover, although processes or methods are sometimes shown as serial executions, these processes or blocks can alternatively be performed or implemented in parallel or can be executed at different times.

[0077] These and other valuable aspects of the various aspects of the present disclosure thus advance the state of the art to at least the next level. While the disclosed aspects have been described as the best mode of implementing environment 100, it should be understood that many alternatives, modifications, and variations will be apparent to those skilled in the art in light of the description herein. Accordingly, it is intended that all such alternatives, modifications, and variations falling within the scope of the included claims be included. All matter herein set forth or shown in the accompanying drawings is to be interpreted in an illustrative and non-limiting sense.

Claims

1. A computer-implemented method for automatically identifying a document type of a document in an image, the method comprising: receiving an image including image data from a mobile device, the image data representing the document; executing, by a first character recognition engine, a first detection process of the image data to detect keywords in the image data; receiving, from a second character recognition engine, a plurality of numeric characters identified in the image data; Based on the keyword, identifying the document type of the document; performing verification of the identified document type as an accepted document type based on the plurality of numeric characters; as well as In response to the verification indicating whether the identified document type is an accepted document type, a notification regarding the identified document type is transmitted to the mobile device.

2. The computer-implemented method of claim 1 , further comprising: In response to the verification indicating that the identified document type is an accepted document type, additional processing of the image data proceeds.

3. The computer-implemented method of claim 1, wherein the image data comprises data representing a Magnetic Ink Character Recognition (MICR) line, and wherein the numeric characters represent a routing number associated with the document.

4. The computer-implemented method of claim 3 , further comprising: The image data is transmitted to the second character recognition engine, wherein the second character recognition engine is configured to identify the routing number from the MICR line.

5. The computer-implemented method of claim 1 , wherein identifying the document type of the document comprises: The image data including the keywords is provided to a trained machine learning model, wherein based on (i) training data, the training data including training image data for a plurality of training documents and training keywords included in the plurality of training documents, and (ii) document types for the plurality of training documents, the trained machine learning model has been trained to generate a mapping between the training data and the document types, so that the trained machine learning model is configured to output the identified document type for the provided image data and keywords.

6. The computer-implemented method of claim 1 , wherein performing the verification comprises: The keyword and the plurality of numeric characters are matched to the identified document type.

7. The computer-implemented method of claim 1 , wherein the document type is associated with the keyword, and identifying the document type based on the keyword comprises: The keyword is compared to a plurality of predefined character strings, wherein the document type is identified based on a match between the keyword and a predefined character string among the plurality of predefined character strings.

8. A system for automatically identifying a document type of a document in an image, the system comprising: Memory; as well as at least one processor coupled to the memory and configured to: receiving an image including image data from a mobile device, the image data representing the document; executing, by a first character recognition engine, a first detection process of the image data to detect keywords in the image data; receiving, from a second character recognition engine, a plurality of numeric characters identified in the image data; Based on the keyword, identifying the document type of the document; performing verification of the identified document type as an accepted document type based on the plurality of numeric characters; as well as In response to the verification indicating whether the identified document type is an accepted document type, a notification regarding the identified document type is transmitted to the mobile device.

9. The system of claim 8, wherein the at least one processor is further configured to: In response to the verification indicating that the identified document type is an accepted document type, additional processing of the image data proceeds.

10. The system of claim 8, wherein the image data comprises data representing a magnetic ink character recognition (MICR) line, and wherein the numeric characters represent a routing number associated with the document.

11. The system of claim 10, wherein the processor is further configured to transmit the image data to the second character recognition engine, wherein the second character recognition engine is configured to identify the routing number from the MICR line and the first character recognition engine is not configured to identify the routing number from the MICR line.

12. The system of claim 8, wherein to identify the document type of the document, the processor is further configured to: The image data including the keywords is provided to a trained machine learning model, wherein based on (i) training data, the training data including training image data for a plurality of training documents and training keywords included in the plurality of training documents, and (ii) document types for the plurality of training documents, the trained machine learning model has been trained to generate a mapping between the training data and the document types, so that the trained machine learning model is configured to output the identified document type for the provided image data and keywords.

13. The system of claim 12, wherein to perform the verification, the processor is further configured to: The keyword and the plurality of numeric characters are matched to the identified document type.

14. The system of claim 8, wherein the document type is associated with the keyword, and to identify the document type based on the keyword, the processor is further configured to: The keyword is compared to a plurality of predefined character strings, wherein the document type is identified based on a match between the keyword and a predefined character string among the plurality of predefined character strings.

15. A non-transitory computer-readable medium storing instructions for automatically identifying a document type of a document in an image, the instructions, when executed by a processor on a mobile device, causing the processor to perform operations comprising: receiving an image including image data from a mobile device, the image data representing the document; executing, by a first character recognition engine, a first detection process on the image data to detect keywords in the image; receiving, from a second character recognition engine, a plurality of numeric characters identified in the image data; Based on the keyword, identifying the document type of the document; performing verification of the identified document type as an accepted document type based on the plurality of numeric characters; as well as In response to the verification indicating whether the identified document type is an accepted document type, a notification regarding the identified document type is transmitted to the mobile device.

16. The non-transitory computer-readable medium of claim 15, the operations further comprising: In response to the verification indicating that the identified document type is an accepted document type, additional processing of the image data proceeds.

17. The non-transitory computer-readable medium of claim 15, wherein the image data comprises data representing a magnetic ink character recognition (MICR) line, and wherein the numeric characters represent a routing number associated with the document.

18. The non-transitory computer readable medium of claim 17, the operations further comprising: The image data is transmitted to the second character recognition engine, wherein the second character recognition engine is configured to identify the routing number from the MICR line and the first character recognition engine is not configured to identify the routing number from the MICR line.

19. The non-transitory computer-readable medium of claim 15, wherein identifying the document type of the document comprises: The image data including the keywords is provided to a trained machine learning model, wherein based on (i) training data, the training data including training image data for a plurality of training documents and training keywords included in the plurality of training documents, and (ii) document types for the plurality of training documents, the trained machine learning model has been trained to generate a mapping between the training data and the document types, so that the trained machine learning model is configured to output the identified document type for the provided image data and keywords.

20. The non-transitory computer-readable medium of claim 15, wherein the document type is associated with the keyword, and identifying the document type based on the keyword comprises: The keyword is compared to a plurality of predefined character strings, wherein the document type is identified based on a match between the keyword and a predefined character string among the plurality of predefined character strings.

Images