Artificial Intelligence (AI)-based document processing and verification

Abstract

Embodiments of the present disclosure relate to document processing and verification based on artificial intelligence (AI). The document processing and verification system based on artificial intelligence (AI) identifies anomalies, such as errors, fraud, and duplications in received documents, and uses machine learning (ML) techniques to enable automatic actions for valid documents. Received documents are processed to determine the probabilities for errors, fraud, and duplications. A verification worklist is generated in which documents are sorted in descending order of probability, and invalid documents with higher probabilities are marked for review, while valid documents with lower probabilities are further processed for execution of automatic actions. Feedback from the review of invalid documents is used to further train a model to determine probabilities.

Description

Technical Field

[0001] The present disclosure generally relates to artificial intelligence (AI)-based document processing and verification. Background Art

[0002] Validation is the documented process of demonstrating that a system or process meets a set of defined requirements. The concept of validation can be applied in different contexts. For example, in addition to more complex validations such as software system validation, documents such as invoices, mechanical or flow diagrams, and labels may also require validation. In addition to conforming to formatting and other requirements, the validation of documents such as invoices may require that the entries therein are accurate. In particular, invoice validation involves a thorough review of the bill to ensure that any discrepancies are highlighted, addressed, and corrected. This can be automated through the accounting system or performed as a manual process. Summary of the Invention

[0003] Embodiments of the present application disclose an AI-based document processing and verification system. The document verification system receives a document package that includes one or more documents or invoices associated with an automated action, processes the documents to automatically mark documents with one or more of errors, fraud, and duplicates, and enables the execution of automated actions for valid documents. Although the example of document processing is discussed herein with respect to invoices, it will be understood that the error and fraud processing or duplicate document identification technology disclosed herein can also be applied to other documents. When a document package including documents such as invoices is received, the invoices therein are processed to detect errors, fraud, and duplicates. Invoices that include errors, fraud, and duplicates are marked for further review, while valid invoices without any errors, fraud, or duplicates are further processed for executing automated actions, such as automatic payments. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The features of the present disclosure are illustrated by way of example in the following drawings, in which like reference numerals denote like elements, and in which:

[0005] Figure 1 A block diagram of an artificial intelligence (AI)-based document processing and verification system according to examples disclosed herein is shown.

[0006] Figure 2 A block diagram of an error and fraud detector according to examples disclosed herein is shown.

[0007] Figure 3 A block diagram of a duplicate document detector according to examples disclosed herein is shown.

[0008] Figure 4 A block diagram of an action optimizer according to examples disclosed herein is shown.

[0009] Figure 5 A flow chart is shown detailing a method of processing a document, such as an invoice, according to examples disclosed herein.

[0010] Figure 6 A flow chart detailing a method of error and fraud detection according to examples disclosed herein is shown.

[0011] Figure 7 A flowchart is shown that details a method of identifying duplicate documents according to examples disclosed herein.

[0012] Figure 8 A flow chart illustrating a method of predicting a time for performing an automatic action according to examples disclosed herein is shown.

[0013] Figure 9A and Figure 9B Two graphs are shown illustrating improvements in document processing provided by an AI-based document processing and verification system according to examples disclosed herein.

[0014] Figure 10 A computer system is shown that can be used to implement an AI-based document processing and verification system according to some examples disclosed herein. DETAILED DESCRIPTION

[0015] For simplicity and illustrative purposes, the present disclosure is described by reference to its examples. In the following description, many specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it is apparent that the present disclosure can be put into practice without being limited to these specific details. In other examples, some methods and structures are not described in detail to avoid unnecessarily confusing the present disclosure. Throughout the present disclosure, the terms "one" and "an" are intended to represent at least one of a particular element. As used herein, the term "include" means including but not limited to, and the term "including" means including but not limited to. The term "based on" means at least partially based on.

[0016] An AI-based document processing and verification system is disclosed. The document verification system receives a document package that includes one or more documents or invoices associated with automated actions, processes the documents to automatically mark documents with one or more of errors, fraud, and duplicates, and enables the execution of automated actions for valid documents. Although the document processing example discussed herein is focused on invoices, it will be appreciated that the error and fraud processing or duplicate document identification techniques disclosed herein can also be applied to other documents. When a document package including documents such as invoices is received, the invoices therein are processed to detect errors, fraud, and duplicates. Invoices containing errors, fraud, and duplicates are flagged for further review, while valid invoices without any errors, fraud, or duplicates are further processed for execution of automated actions, such as automatic payment.

[0017] To identify errors, fraud, or duplicates, the document processing and verification system employs machine learning (ML) based models. For example, a supplier profile is initially constructed from supplier data. Features are extracted from the supplier profile. Any existing features can be adjusted or updated using data associated with each new document package (including newer invoices). The invoices in the document package are accessed and scored by a machine learning based anomaly detection model for error and fraud detection. The anomaly detection model can be trained via unsupervised techniques to detect fraud and errors in different invoices. In one example, the anomaly detection model can include an unsupervised outlier detection model, such as, but not limited to, isolation forests, cluster-based local outlier factors (CBLOFs), autoencoders, etc. Each invoice in the document package is scored by the anomaly detection model, and a probability indicating that the invoice includes one or more of errors and fraud is generated.

[0018] Invoices processed for error and fraud detection are further processed for duplicate identification. Invoices in a data package to be processed for duplicate identification are stored in an invoice index. The index can be queried to obtain suggestions for similar invoices. Similar invoices can be selected from data packages and / or previous invoices in historical data sources. A similarity graph is created based on the recommendations. The similarity graph is decomposed into a set of similar invoices using graph theory algorithms, logical combination, and other techniques. The set can be classified as duplicates by a duplicate detection model based on set-level, document-level / invoice-level, and supplier-level features. In one example, methods such as extreme gradient boosting (XG boosting) and feedforward neural networks can be implemented by the duplicate detection model. In one example, the scores can be rescaled based on other invoice attributes such as invoice amount. A validation worklist is generated for error, fraud, and duplicate detection as invoices are processed. The validation worklist sorts the invoices in descending order of score (i.e., decreasing invalidity), so that invoices with the highest probability of error, fraud, or duplicates appear at the top of the validation worklist.

[0019] Invoices at the top of the verification work list (e.g., the top N invoices, where N is a natural number, N=1, 2, 3, ...) with higher probabilities of error, fraud, and duplication are flagged for further review. In one example, the flagged invoices can be reviewed by human reviewers who can provide feedback on the errors, fraud, and duplications flagged by the AI-based document processing and verification system. The feedback from the human reviewers can be provided as training data to the anomaly detection model and the duplicate detection model. Invoices lower in the verification work list have lower probabilities of error, fraud, or duplication and are therefore determined to be valid. Therefore, they can be forwarded for processing that enables the execution of automatic actions. In one example, the AI-based document processing and verification system can be configured with a threshold probability or include a trained AI model (e.g., a classifier) ​​that can separate valid invoices that can be processed automatically from invalid invoices that require review.

[0020] Valid invoices are provided for enabling automatic actions, which include determining a time for executing the automatic action for each of the invoices, i.e., early or delayed execution of the automatic action. Thus, the AI-based document processing and verification system is configured to predict, for (multiple) specific time periods, payment dates for invoices with due dates within the specified time periods. The AI-based document processing and verification system can generate the following prediction: a given invoice with a due date within a specific time period can be paid early (i.e., on time or before the due date) or delayed (i.e., after the due date). The payment prediction can be generated by a trained neural network (NN) using features extracted from input data, including but not limited to invoice and supplier data, anomaly and duplication probabilities, etc.

[0021] The document processing and verification system disclosed herein provides a technical improvement in the field of document processing and verification systems that enable automated actions. The generation of a verification worklist that sorts documents / invoices according to the probability of errors and duplications alleviates the need to analyze each document to determine errors, fraud, or duplications. Thus, the AI-based document processing and verification system marks a subset of received invoices for review, while the remaining invoices are forwarded for automated actions. Thus, the AI-based document processing and verification system significantly reduces the volume of documents to be reviewed (and therefore also reduces the time for document review). Furthermore, as can be seen from certain results discussed below, sorting invoices according to the probability of errors and duplications in the verification worklist enables the AI-based document processing and verification system to identify nearly 100% of errors and duplications in the top 20-40% of invoices, thereby allowing the remaining 60-80% of invoices to pass for automated payment. This reduction in the volume of documents to be reviewed for validity and the detection of errors, fraud, and duplications in the initial subset of documents to be reviewed make the document processing and verification system faster and more efficient.

[0022] Figure 1 A block diagram of an AI-based document processing and verification system 100 according to examples disclosed herein is shown. The AI-based document processing and verification system 100 receives a document package including one or more documents, which are processed to detect and identify one or more errors, fraud, duplications, etc. Documents free of such errors are further processed to determine the order in which automated actions associated with the documents should be executed. For example, a document package 150 may include documents such as invoices 152 and 154, which are processed to identify errors, fraud, duplications, etc. Such defective / invalid invoices are flagged for further review. Invoices are time-stamped business documents that detail and record transactions between buyers and sellers. If goods or services are purchased on credit, the invoice typically specifies the terms of the transaction and provides information on available payment methods. Invoices free of such errors are processed to determine the order in which automated actions, such as automatic payments, should be executed. The AI-based document processing and verification system 100 includes an input receiver 102, an AI-based fault handler 104, an action optimizer 106, and an action handler 108. The AI-based document processing and verification system 100 may also be coupled to one or more data sources. The data storage 170 can be used to store data generated and used during various verification and automated execution processes. The AI-based document processing and verification system 100 can also be coupled to a historical data source 172, which can be used to store feedback data and historical data for training models used by the AI-based document processing and verification system 100.

[0023] Input receiver 102 receives a document package 150, which includes invoices 152, 154, which may be received in a digital / machine-readable format or may be converted to a digital format by input receiver 102. AI-based fault processor 104 includes an error and fraud detector 142 and a duplicate document detector 144. Error and fraud detector 142 analyzes each of invoices 152, 154, etc., and flags the invoice for potential non-compliance issues, field value errors, fraud, etc. Flagged invoices are prioritized based on a score, which may be a combination of a model score and the invoice amount, to ensure that invoices with a high risk of error and / or fraud and higher-value invoices are verified first. The output of error and fraud detector 142 is an error and fraud detection (EFD) score, which indicates the likelihood that a document / invoice contains anomalies such as errors and / or fraud.

[0024] Duplicate document detector 144 also analyzes each invoice in invoices 152 and 154 and forms a collection of invoices that include elements that may be potential duplicates of each other. The collection of invoices is further scored using a machine learning (ML) model (such as XG boosting, neural networks, etc.) to determine the probability of containing duplicates. The output of AI-based fault processor 104 is a validation worklist 130, which includes all invoices from document bundle 150, ranked by their probability of anomaly. In one example, an aggregated failure score can be calculated for each invoice to determine its position in validation worklist 130. In another example, duplicate document detector 144 can use the anomaly score to identify duplicates within the documents in document bundle 150. Accordingly, invoices with a higher probability of failure, error, and / or duplication are ranked at the top of validation worklist 130, while invoices with lower failure scores are ranked at the bottom of validation worklist 130. AI-based fault processor 104 can be configured with a threshold score below which invoices are considered valid and allowed to be further processed by action optimizer 106. Invoices with scores above a threshold score are flagged for further review. Upon completion of the further review, a subset of the flagged invoices may be deemed valid and allowed to be further processed by the action optimizer 106, while those invoices deemed invalid may be transmitted back to the source providing the document package 150.

[0025] In one example, the AI-based document processing and verification system 100 can be coupled to a historical data source 172, which can be used to store feedback data 176 and other historical data that can be used to train different models used by the AI-based document processing and verification system 100. The AI-based fault processor 104 can receive feedback data 176 from further processing, which feedback data 176 is related to valid and invalid invoices from among the marked invoices. In one example, the verification worklist 130 including the ranked valid and invalid invoices can be manually reviewed, and the reviewer provides the feedback data 176 to the AI-based fault processor 104. The feedback data 176 can be used to train the ML models used by the error and fraud detector 142 and the duplicate document detector 144.

[0026] Valid invoices are provided to the action optimizer 106, which optimizes when automatic payment actions can occur within certain constraints, including time and financial constraints. While some invoices can be paid early, others can be paid later. Furthermore, certain discounts can be applied when invoices meet certain conditions. The action optimizer 106 provides a prediction of the payment time for each of the invoices 152 and 154 in the document package 150, maximizing discounts and cash flow for the entity whose invoices are being processed. For example, automatic payments can be scheduled so as not to violate predetermined lower limits corresponding to financial constraints. The action optimizer 106 outputs a payment worklist 180, which sorts the invoices 152, 154, etc. in a predetermined order of their payment dates, e.g., from the earliest to the latest, or vice versa. The payment worklist 180 is provided to the action processor 108, which can initiate automatic actions based on the order in which they are sorted, such as automatically paying the invoices in the payment worklist 180.

[0027] Figure 2 A block diagram of an error and fraud detector 142 according to an example disclosed herein is shown. The error and fraud detector 142 uses a supplier profile generator 202, a feature extractor and updater 204, and an anomaly detection model 206 to process invoices 152, 154, etc., to output potentially abnormal invoices with scores and reason codes. The supplier profile generator 202 generates supplier profiles 252 to understand historical supplier behavior, i.e., amount distribution, invoice numbering patterns, currency used, etc. In one example, frequencies and tables can be used for categorical data variables, numerical data variables, etc. Thus, each time an invoice is received, historical supplier behavior associated with the invoice can be used to analyze the invoice. Additionally, data extracted from the invoice can be used to update the supplier profile. The feature extractor and updater 204 extracts and updates features 256 from the supplier profile 252. Furthermore, features 256 are updated based on feedback 176 received from further review of the verification worklist 130.

[0028] The updated features can be used by anomaly detection model 206 to rank invoices 152, 154 in document bundle 150 for identification of errors, fraud, etc. Errors can include typographical errors, formatting errors, etc. In one example, anomaly detection model 206 can include an ML model, such as Isolation Forest (CBLOF), trained on data from historical data source 172 for outlier / anomaly identification. Anomaly detection model 206 can output a probability that each invoice 152, 154, etc. is an anomaly. Anomaly probabilities 258 output by anomaly detection model 206 can be made accessible to duplicate document detector 144 for further processing.

[0029] Figure 3A block diagram of a duplicate document detector 144 according to examples disclosed herein is shown. The duplicate document detector 144 identifies duplicate documents / invoices within a document package 150 using a collection generator 302, a feature extractor 304, a duplicate detection model 306, and a document ranker 308. In one example, the duplicate document detector 144 may receive as input outstanding / flipped invoices, anomaly probabilities 258 generated by the error and fraud detector 144, and feedback data 176. The collection generator 302 generates collections 322 by considering sequences defined by the account / supplier. More specifically, the collection generator 302 utilizes an index that is updated with each newly received invoice and processed by the recommendation engine. Recommendations are converted into collections 322 based on associated connections.

[0030] Feature extractor 304 can be trained to automatically extract features 356 for each set in the collection based on collection attributes, invoice attributes, historical profiles, and the like. For example, methods such as MinHash can be used for text feature extraction. Features 356 are used by duplicate detection model 306 to identify duplicate invoices for documents 152, 154 within one or more of document bundle 150 and historical data source 172. More specifically, duplicate detection model 306 scores new sets formed as new invoices are received. The score, i.e., duplicate probability 358, can indicate the presence of duplicate invoices in one or more of document bundle 150 and historical data source 172. Anomaly probability 258 and duplicate probability 358 can be aggregated by document ranker 308 in a predetermined manner to rank invoices 152, 154 in document bundle 150 to generate verification worklist 130. As described above, the verification worklist 130 lists the invoices in descending order of probability (i.e., invalidity) and invoice value, such that invoices associated with higher amounts and higher probabilities of error, fraud, and / or duplication are provided for further processing, while the remaining valid invoices are provided to the action optimizer 106 for scheduling automated actions, such as automated payments. In one example, the top N documents or invoices in the verification worklist 130 may be marked for further review, while the remaining documents / invoices may be provided to the action optimizer 106 for execution of corresponding automated actions.

[0031] In one example, feedback data 176 received from further review of the labeled documents can be provided to the AI-based fault processor 104. The feedback data 176 can be used as labeled data for explicitly training ML models (i.e., one or more of the anomaly detection model 206 and the duplicate detection model 306).

[0032] Figure 4A block diagram of the action optimizer 106 according to examples disclosed herein is shown. The action optimizer 106 includes a date-document selector 402, a feature extractor 404, a risk score predictor 406, and a document-action scheduler 408. The action optimizer 106 is configured to predict the early or delayed execution of automated actions associated with one or more documents 152, 154, etc. Referring to an example where the documents 152, 154 are invoices, the action optimizer 106 predicts the temporal attributes of invoices with due dates within a specific time window. The date-document selector 402 selects, from the document bundle 150, outstanding / paid invoices due within the specific time window, along with payment terms metadata and a score or anomaly probability 258 and duplicate probability 358 for each of the selected invoices as input.

[0033] Feature extractor 404 automatically extracts features 456 from supplier profiles 252, supplier metadata, and invoice attributes. In one example, features 456 may include one or more of features 256 and 356 extracted by error and fraud detector 142 and duplicate document detector 144. However, due to the processing of invoices in calculating anomaly probability 258 and duplicate probability 358, newer features, including at least the calculated probabilities, may be included in features 456. Risk score predictor 406 uses these features to calculate invoice predictions 458. For each invoice, risk score predictor 406 can predict a score for the likelihood of a particular action and the impact of the particular action under given constraints. For example, in the case of invoice payment, risk score predictor 406 can predict the likelihood of early payment and the cost of early payment (i.e., cash flow impact), the likelihood of late payment and the cost of late payment, and the likelihood of receiving a discount and the cost / benefit of the discount. As described above, the impact of an action can also be predicted based on different constraints, such as cash flow predictions for different time periods for each invoice in the payment process. In one example, the risk score predictor 406 may employ a trained neural network (NN) for invoice prediction 458 .

[0034] Invoice forecast 458 and cash flow forecast 460 form the basis for an optimization problem for automated action execution, which involves generating a list of invoices that minimizes both early and late payments while maximizing the use of available cash without violating cash flow constraints. This list of invoices to be paid is generated by document action scheduler 408, which generates a list of invoices (i.e., payment work list 180). Invoices to be paid can be sorted in ascending order of payment date, with invoices due to be paid earlier being placed at the top of payment work list 180 and those that are due to be paid further down the payment work list 180. In one example, document action scheduler 408 can employ integer programming to generate payment work list 180. The generated payment work list 180 is provided to action processor 108 for automated action execution.

[0035] Figure 5 A flowchart 500 is shown, detailing a method for processing documents, such as invoices, according to examples disclosed herein. The method begins at 502, where a document package 150 including documents (e.g., invoices 152, 154, etc.) is received for processing. Hundreds of thousands of invoices may be received daily for processing by the AI-based document processing and verification system 100. At 504, the received documents are processed for error and fraud detection to identify and correct errors and anomalies in the documents before payment. At 506, the documents are further processed to identify duplicate documents to prevent duplicate payments as much as possible. In one example, error and fraud probabilities can be used to determine duplicates at step 506. The output of step 506 is a verification worklist 130, in which the documents, i.e., invoices 152, 154, etc., are arranged in descending order of invalidity (i.e., descending order of error / fraud and duplicate probabilities).

[0036] At least a subset of the documents at the top of the verification work list 130, i.e., the top N documents (where N is a natural number and N=1, 2, 3, ...) are provided for review at 508. In one example, the AI-based document processing and verification system 100 can be configured with a threshold probability for providing invoices for review at 508. In one example, an AI-based model can be trained to select documents from the verification work list 130 for review. At 510, feedback from the review is received and used to train an AI model for error and duplication prediction. At 512, documents at the bottom of the verification work list 130, i.e., with a probability below the threshold probability, are processed to predict the best time period for performing relevant automatic actions such as automatic payment. At 514, the automatic payment action is performed at the time period determined for the document processed at 512.

[0037] Figure 6A flowchart 600 is shown that details a method for error and fraud detection according to an example disclosed herein. The method begins at 602 with the creation of a supplier profile. The supplier profile 252 is created based on outstanding / paid invoices, supplier data (including the individual suppliers that issued the invoices), feedback received from the review of previous verification worklists, and historical invoices. In one example, frequency tables for categorical variables and distribution tables for numerical variables associated with the supplier profile are updated, and calculation tables can be generated for derived / calculated values ​​to generate the supplier profile. At 604, features 256 to be used by the anomaly detection model 206 are extracted from the supplier profile 252. In one example, features can be extracted based on the quantile bins in which the associated variables are located. In addition, feature values ​​can be adjusted / updated at 606 based on the variable cardinality and feedback data 176.

[0038] At 608, newly received invoices, such as invoices 152 and 154 in document package 150, are accessed. At 610, invoices 152 and 154 are scored by anomaly detection model 206 to identify anomalies / errors and fraud. In one example, an outlier detection model, such as an isolation forest, CBLOF, or autoencoder, can be employed for anomaly detection. Additionally, reason codes associated with the scores can be output at 610. Reason codes are associated with features considered to be the primary cause of anomalies in a record. These reason codes are defined by comparing the values ​​of these features in a particular record with historical values ​​of the features for that supplier. If the value of the current record falls outside the expected distribution of feature values, the feature is flagged as a possible cause of an anomaly and is labeled with a corresponding reason code. This can, for example, facilitate manual verification of records. In one example, the score for each invoice in the invoices can be rescaled at 612, or biased using weights based on the weights of one or more invoice attributes, such as, but not limited to, the amount associated with each invoice. At 614, the invoices, along with their associated scores (i.e., anomaly probabilities 258), are provided to duplicate document detector 144 for further processing.

[0039] Figure 7A flowchart 700 is shown, detailing a method for identifying duplicate documents, according to an example disclosed herein. The method begins at 702, where input data is received from the error and fraud detector 142. The input data may include, but is not limited to, data for outstanding / paid invoices, including invoice ID, invoice type, the date the invoice was scored, and corresponding anomaly probability / reason code for each invoice. Additionally, supplier data, such as the supplier profile 252 and feedback data 176, may also be received as input at 702. At 704, the received invoice is loaded into an invoice index. At 706, the invoice index is queried to obtain recommendations for similar invoices received in the document package 150 and / or previous invoices stored in the historical data source 172. At 708, a similarity graph is generated based on the recommendations output from the query executed against the index containing the invoices. At 710, the similarity graph may be decomposed into sets of similar invoices based on connectivity. One or more of graph theory algorithms and logical constructs may be used to generate the similarity graph and decompose the similarity graph into sets of similar invoices. At 712 , similar collections are classified as duplicates based on various features including, but not limited to, collection-level, invoice-level, and supplier-level features employed by the duplicate detection model 306 .

[0040] Figure 8 A flowchart 800 is shown, detailing a method for predicting the payment period of an invoice, according to an example disclosed herein. At 802, input data is obtained, including but not limited to open / closed invoices, supplier profiles, payment terms metadata including current time data, anomaly probability 258, and duplication probability 358. At 804, features are extracted from the received data. At 806, a prediction or risk score is calculated for each of the invoices regarding early payment or delayed payment during a selected period. In one example, a neural network model employing the features extracted from the received data at 804 is used to calculate the risk score. At 808, invoices due for payment within the next N days (where N is a natural number, N=1, 2, 3, etc.) are selected based on the risk score calculation. Additionally, at 810, constraint data, such as available cash flow, is retrieved to identify the order in which invoices should be paid. At 812, a payment worklist 180 is generated as a prioritized list of invoices in which discounts are maximized without missing payment deadlines. In one example, the generation of the payment worklist 180 can be formulated as an integer programming problem, in which invoices are sorted to maximize discounts based on financial and time constraints. The financial constraint data may relate to a cash flow that must be maintained, while the time constraint relates to a selected time period.At 814, the payment work list 180 is provided for automation.

[0041] Figure 9A and Figure 9BTwo graphs are shown illustrating the improvements in document processing provided by the AI-based document processing and verification system 100 according to examples disclosed herein. Figure 9A A bar graph 910 is shown, with the top X percent of invoices in terms of anomalies mapped on the X-axis, and the total percentage of documents or invoices processed by the AI-based document processing and verification system 100 mapped on the Y-axis. As can be seen from point 902 on the bar graph 910, substantially all anomalies are captured in the top 20 percent of processed invoices. Because the verification worklist 130 sorts the invoices in descending order of anomaly probability, the AI-based document processing and verification system 100 can be configured to flag a limited number of invoices at the top of the verification worklist 130 for review, while allowing the remaining invoices for automated processing. Thus, the AI-based document processing and verification system 100 provides an improvement over systems that require every document / invoice to be processed for errors and / or fraud.

[0042] Similarly, Figure 9B The line graph 920 in FIG. 1 shows a graph of the X% of sets versus the true duplicates captured on the Y axis. According to the examples disclosed herein, the duplicate document detector 144 generates sets and captures true duplicates. In one example, the duplicate document detector 144 creates sets to ensure that no sets created by predefined rules are missed, and scores each set based on the likelihood that it contains a true duplicate. As can be seen from point 922 on the line graph 920, 100% of the duplicates are captured in 35-40% of the sets. Similarly, because the verification worklist 130 sorts the invoices in descending order of anomaly probability, the AI-based document processing and verification system 100 can be configured to mark a limited number of invoices at the top of the verification worklist 130 for review, while allowing the remaining invoices for automatic processing. Thus, the AI-based document processing and verification system 100 provides an improvement over processing each document / invoice to identify duplicate documents / invoices.

[0043] Figure 10 The computer system 1000 that can be used to implement the AI-based document processing and verification system 100 according to the examples disclosed herein is illustrated. More particularly, computing machines such as desktop computers, laptop computers, smart phones, tablet computers, and wearable devices that can be used to generate or access data from the AI-based document processing and verification system 100 can have the structure of the computer system 1000. The computer system 1000 may include additional components not shown, and some of the process components described may be removed and / or modified. In another example, the computer system 1000 may be located on an external cloud platform, such as Amazon Web Services, Cloud or internal enterprise cloud computing cluster, or organizational computing resources, etc.

[0044] Computer system 1000 includes processor(s) 1002, such as a central processing unit, an ASIC, or another type of processing circuit; input / output devices 1010, such as a display, a mouse, a keyboard, etc.; a network interface 1004, such as a local area network (LAN), a wireless 802.11x LAN, a 3G, 4G, or 10G mobile WAN, or a WiMax WAN; and processor-readable media 1006. Each of these components can be operatively coupled to a bus 1008. Processor-readable or computer-readable media 1006 can be any suitable medium that participates in providing instructions to processor(s) 1002 for execution. For example, processor-readable media 1006 can be a non-transitory or non-volatile medium, such as a disk or solid-state non-volatile memory, or a volatile medium, such as RAM. The instructions or modules stored on processor-readable media 1006 can include machine-readable instructions 1064 executed by processor(s) 1002, which cause processor(s) 1002 to perform the methods and functions of AI-based document processing and verification system 100.

[0045] The AI-based document processing and verification system 100 can be implemented as software or machine-readable instructions stored on a non-transitory processor-readable medium and executed by one or more processors 1002. For example, the processor-readable medium 1006 can store an operating system 1010, such as MAC OS, MS WINDOWS, UNIX, or LINUX, and code 1014 for the AI-based document processing and verification system 100. The operating system 1010 can be multi-user, multi-processing, multi-tasking, multi-threaded, real-time, etc. For example, during runtime, the operating system 1010 is running and code for the AI-based document processing and verification system 100 is executed by the processor(s) 1002.

[0046] The computer system 1000 may include a data storage device 1010, which may include a non-volatile data storage device. The data storage device 1010 stores any data used by the AI-based document processing and verification system 100. The data storage device 1010 may be used as a data storage device 170 to store various invoices, features, supplier information, forecast values, and other data elements generated and / or used during the operation of the AI-based document processing and verification system 100.

[0047] The network interface 1004 connects the computer system 1000 to internal systems, for example, via a LAN. In addition, the network interface 1004 can connect the computer system 1000 to the Internet. For example, the computer system 1000 can connect to a web browser and other external applications and systems via the network interface 1004.

[0048] What is described and illustrated herein is an example and some of its variations. The terms, descriptions, and figures used herein are for illustrative purposes only and are not intended to be limiting. Many variations are possible within the spirit and scope of this subject matter, which are intended to be defined by the appended claims and their equivalents.

Claims

1. An artificial intelligence (AI)-based document processing and verification system, comprising: at least one processor; a non-transitory processor-readable medium storing machine-readable instructions that cause the processor to: accessing a document package comprising one or more documents; determining whether one or more of errors and fraud exists in the one or more documents, wherein the anomaly detection model outputs an error score for the errors and fraud in the documents; determining whether one or more of the document bundle and the historical data source includes duplicate documents for the one or more documents, wherein a duplicate detection model outputs a duplicate score for use in the determination regarding the duplicate documents; generating a validation worklist comprising the one or more documents scored by the anomaly detection model and the duplicate detection model, wherein the documents are arranged in descending order of probability of having one or more of the error and fraud and the duplicate document; marking invalid documents from the verification worklist for review, wherein the invalid documents comprise the top N documents of the verification worklist, where N is a natural number; training one or more of the anomaly detection model and the duplicate detection model based on feedback received from the review of the top N documents; forecasting one or more of an early payment and a delayed payment for valid documents from the verification worklist, wherein the valid documents include the one or more documents below the top N documents from the verification worklist, and wherein the forecast is subject to a financial constraint and a time constraint; generating a payment run worklist that lists the active documents in order of payment dates, wherein the order of the payment dates is determined based on a prediction of the early payment and the delayed payment for the active documents; and The valid documents are automatically paid in the order of the payment dates.

2. The artificial intelligence (AI)-based document processing and verification system according to claim 1, wherein in order to determine whether there is one or more of an error and fraud in the document, the processor is further configured to: Supplier-level features are updated based on the one or more documents.

3. The artificial intelligence (AI)-based document processing and verification system according to claim 2, wherein in order to update the supplier-level features, the processor is further configured to: updating one or more supplier profiles, wherein the updating of the supplier profiles occurs based on the one or more documents, wherein updating the supplier profiles comprises updating distribution tables for numerical variables and calculation tables for derived features; and The one or more documents are scored using the anomaly detection model, the anomaly detection model employing the supplier-level features updated based on the one or more documents.

4. The artificial intelligence (AI)-based document processing and verification system according to claim 3, wherein the anomaly detection model comprises an unsupervised outlier detection model.

5. The artificial intelligence (AI)-based document processing and verification system according to claim 1 , wherein in order to determine whether the document and the historical data source include duplicate documents, the processor is further configured to: Loading the index of historical documents; and The index of historical documents is updated with the one or more documents from the document set.

6. The artificial intelligence (AI)-based document processing and verification system according to claim 5, wherein in order to determine whether the document and the historical data source include duplicate documents, the processor is further configured to: identifying, from the index, documents that are similar to the one or more of the documents; and A set of similar documents is formed using a similarity graph.

7. The artificial intelligence (AI)-based document processing and verification system according to claim 6, wherein in order to determine whether the document and the historical data source include duplicate documents, the processor is further configured to: The collection is classified as a duplicate based on one or more of collection-level features, document-level features, and supplier-level features.

8. The artificial intelligence (AI)-based document processing and verification system according to claim 1 , wherein in order to predict the early payment and the delayed payment for the valid document, the processor is further configured to: For each payment period: The likelihood of the advance payment and the cost of the advance payment are predicted for each of the valid documents.

9. The artificial intelligence (AI)-based document processing and verification system according to claim 1 , wherein in order to predict the early payment and the delayed payment for the valid document, the processor is further configured to: For each payment period: The likelihood of the delayed payment and the cost of the delayed payment are predicted for each of the valid documents.

10. The artificial intelligence (AI)-based document processing and verification system according to claim 1 , wherein in order to predict the early payment and the delayed payment for the valid document, the processor is further configured to: For each pay period: A likelihood of obtaining a discount and a cost / benefit of the discount are predicted for each of the valid documents.

11. The artificial intelligence (AI)-based document processing and verification system according to claim 1 , wherein predicting one or more of the early payment and the delayed payment causes the processor to further: One or more of the early payment and the delayed payment are predicted within a predetermined lower limit corresponding to the financial constraint.

12. The artificial intelligence (AI)-based document processing and verification system according to claim 1, wherein the document is a check.

13. A method for processing and verifying a document, comprising: receiving a document package comprising one or more documents, wherein the one or more documents are associated with an automated action to be performed; determining an anomaly probability for the one or more documents, wherein the anomaly probability indicates a likelihood that the one or more documents include one or more of an error and fraud by processing the one or more documents using an anomaly detection model; generating a duplicate probability for the one or more documents, wherein the duplicate probability is generated by a duplicate detection model and indicates a likelihood that the document package or historical data source includes a duplicate document of the one or more documents, wherein the duplicate probability includes an anomaly probability; generating a verification work list comprising the one or more documents arranged in descending order of invalidity, wherein the invalidity of the one or more documents is determined as a combination of corresponding probabilities of the anomaly probability and the duplication probability; marking the top N documents in the verification worklist for further review; training one or more of the anomaly detection model and the duplicate detection model based on feedback received from the further review of the top N documents; obtaining a prediction for performing the automatic action on the one or more documents below the top N documents in the verification worklist; as well as The automatic action is performed based on the prediction.

14. The method according to claim 13, wherein determining the anomaly probability by the anomaly detection model further comprises: A supplier profile is generated for the supplier that published the one or more documents.

15. The method of claim 14, wherein determining the anomaly probability comprises: Features are extracted from the supplier profile, wherein the features are used by an anomaly detection model to determine the anomaly probability of the one or more documents.

16. The method of claim 14, wherein generating the probability of duplication for the one or more documents comprises: updating an index using the one or more documents; querying the index to obtain recommendations for other documents similar to the one or more documents; generating a similarity graph based on the recommendations; as well as The similarity graph is decomposed into a set of similar invoices based on connectivity.

17. The method of claim 13, wherein the duplicate detection model comprises at least one of extreme gradient XG boosting and a feed-forward neural network.

18. The method of claim 13, wherein obtaining the prediction for performing the automatic action further comprises: A neural network trained via an unsupervised method is used to predict early performance or delayed performance of an automatic action associated with the one or more documents, wherein the one or more documents are invoices and the automatic action comprises automatic payment of the invoices.

19. A non-transitory processor-readable storage medium comprising machine-readable instructions, the machine-readable instructions causing a processor to: accessing a document package comprising one or more documents; determining whether one or more of errors and fraud exists in the one or more documents, wherein the anomaly detection model outputs an anomaly probability regarding the errors and fraud in the one or more documents; determining whether one or more of the document bundle and the historical data source includes duplicate documents for the one or more documents, wherein a duplicate detection model outputs a duplicate probability for use in the determination regarding the duplicate documents; generating a validation worklist comprising the one or more documents scored by the anomaly detection model and the duplicate detection model, wherein the documents are arranged in descending order of probability of having one or more of the error and fraud and the duplicate document; marking invalid documents from the verification worklist for review, wherein the invalid documents comprise the top N documents of the verification worklist, where N is a natural number; forecasting one or more of an early payment and a delayed payment for valid documents from the verification worklist, wherein the valid documents include the one or more documents below the top N documents from the verification worklist, and wherein the forecast is subject to a financial constraint and a time constraint; generating a payment run worklist that lists the active documents in order of payment dates, wherein the order of the payment dates is determined based on a prediction of the early payment and the delayed payment for the active documents; and The valid documents are automatically paid in the order of the payment dates.

20. The non-transitory processor-readable storage medium of claim 19, wherein the non-transitory processor-readable storage medium comprises further instructions that cause the processor to: A payment run worklist is generated based on the prediction, the payment run worklist listing the one or more documents below the top N documents in the verification worklist, wherein the prediction includes a payment due date, and entries in the payment run worklist are listed in ascending order of payment due date.

Images