Deep Learning Based Document Splitter

Abstract

A system and method are provided for dividing an electronic file into sub-documents. An electronic file is received. Portions of the electronic file are classified using a trained machine learning-based model. The classification represents relative positions of the portions within the sub-documents of the electronic file. The electronic file is divided into sub-documents based on the relative positions of the portions. The sub-documents are output.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to Indian Patent Application No. 202011041647, filed on September 25, 2020, and U.S. Patent Application No. 17 / 075,731, filed on October 21, 2020, the disclosures of which are incorporated herein by reference in their entireties.

[0002] The present invention relates to robotic process automation (RPA), and in particular to a deep learning based document splitter for document processing in RPA. [Background technology]

[0003] Robotic Process Automation (RPA) is a form of process automation that uses software robots to automate workflows. RPA may be implemented to automate repetitive and / or labor-intensive tasks to reduce costs and increase efficiency. One important task in RPA is document processing. Typically, document processing is performed on electronic computer files that contain multiple sub-documents. For example, such electronic computer files may include sub-documents corresponding to invoices, reports, insurance forms, etc. To perform many document processing tasks, such as digitizing documents, it is necessary to split the electronic computer file into multiple sub-documents.

[0004] Conventional approaches for document segmentation of electronic computer files rely on identifying keywords in the electronic computer files, but such reliance on keyword identification by conventional document segmentation approaches has many drawbacks, such as the need for manual identification of keywords, difficulty in finding keywords in electronic computer files, keyword collisions when multiple sub-documents contain the same keyword, and human error in keyword assignment. Summary of the Invention

[0005] According to one or more embodiments, a system and method are provided for splitting an electronic file into sub-documents using a trained machine learning based model. An electronic file is received. Portions of the electronic file are classified using the trained machine learning based model. The classification represents relative positions of the portions within the sub-documents of the electronic file. The electronic file is split into sub-documents based on the relative positions of the portions. The sub-documents are output.

[0006] In one embodiment, the classifications representing relative positions of the parts within a sub-document of the electronic file include a classification representing a first part of the sub-document, a classification representing a last part of the sub-document, and a classification representing a part between the first and last parts of the sub-document. The parts of the electronic file may be classified by mapping features of interest extracted from each of the parts of the electronic file to the classifications. The features of interest may include one or more of a word cloud, a page number, or text related features.

[0007] In one embodiment, misclassified portions are detected from the classified portions using a statistical checker, and the misclassified portions are presented to a user for manual classification.

[0008] In one embodiment, the electronic file is split immediately before each portion classified as being the first portion of a subdocument. The multiple portions of the electronic file may correspond to multiple pages of the electronic file.

[0009] In one embodiment, the trained machine learning based model is a trained deep learning model, which may be based on any of the following architectures: LSTM (long short-term memory), Bi-LSTM (bidirectional LSTM), and seq2seq (sequence-to-sequence).

[0010] In one embodiment, the sub-documents are classified using a classifier.

[0011] These and other advantages of the present invention will become apparent to those skilled in the art from the following detailed description and accompanying drawings. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is an architectural diagram illustrating Robotic Process Automation (RPA) in accordance with one embodiment of the present invention.

[0013] [Diagram 2] FIG. 1 is an architectural diagram illustrating an example of a deployed RPA system according to an embodiment of the present invention.

[0014] [Diagram 3] FIG. 1 is an architectural diagram illustrating a simplified example deployment of an RPA system according to an embodiment of the present invention.

[0015] [Figure 4] 1 illustrates a method for dividing an electronic computer file into sub-documents according to one embodiment of the present invention.

[0016] [Diagram 5] 1 is a block diagram representation of an electronic file, according to one embodiment of the present invention.

[0017] [Figure 6]1 illustrates a method for training a machine learning based model to classify portions of an electronic file, according to an embodiment of the present invention.

[0018] [Figure 7] 1 is a block diagram of a computing system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Robotic Process Automation (RPA) is used to automate workflows and processes. FIG. 1 is an architecture diagram of an RPA system 100 according to one or more embodiments. As shown in FIG. 1, the RPA system 100 includes a designer 102 that enables a developer to design an automated process. More specifically, the designer 102 facilitates the development and deployment of RPA processes and robots to perform activities in the process. The designer 102 may provide solutions for application integration and business process automation for third-party applications, management information technology (IT) tasks, and contact center operations. One commercial example of an embodiment of the designer 102 is UiPath Studio™.

[0020] In designing rule-based process automation, developers control the execution order and relationships between a custom set of steps developed in the process, defined herein as "activities." Each activity may include an action, such as clicking a button, reading a file, writing to a log panel, etc. In some embodiments, processes may be nested or embedded.

[0021] Some types of processes may include, but are not limited to, sequences, flowcharts, finite state machines (FSMs), and / or global exception handlers. Sequences may be particularly suitable for linear processes, allowing flow from one activity to another without perturbing the process. Flowcharts may be particularly suitable for more complex business logic, allowing for the integration of decisions and the connection of activities in more diverse ways with multiple branching logic operators. FSMs may be particularly suitable for large workflows. FSMs may use a finite number of states at run time, which are triggered by conditions (i.e., transitions) or activities. Global exception handlers may be particularly suitable for determining workflow behavior when an execution error occurs or for debugging the process.

[0022] Once a process is developed in Designer 102, the execution of the business process is orchestrated by Conductor 104, which orchestrates one or more Robots 106 that execute the workflow developed in Designer 102. One commercial example of an embodiment of Conductor 104 is UiPath Orchestrator™. Conductor 104 facilitates the creation, monitoring, and management of deployment of resources in an RPA environment. In one example, Conductor 104 is a web application. Conductor 104 may also serve as an integration point with third-party solutions and applications.

[0023] The conductor 104 may manage all the RPA robots 106 by connecting and executing the robots 106 from a centralized point. The conductor 104 may have various functions including, but not limited to, provisioning, deployment, configuration, queuing, monitoring, logging, and / or providing interconnectivity. Provisioning may include creating and maintaining connections between the robots 106 and the conductor 104 (e.g., web applications, etc.). Deployment may include ensuring correct delivery of package versions to the robots 106 assigned for execution. Configuration may include maintaining and delivering robot environment and process configurations. Queuing may include providing management of queues and queue items. Monitoring may include tracking robot identification data and maintaining user permissions. Logging may include storing and indexing logs in a database (e.g., an SQL database, etc.) and / or other storage mechanism (e.g., ElasticSearch, etc., which provides the ability to store and quickly query large data sets). Conductor 104 may provide interconnectivity by acting as a centralized point of communication for third party solutions and / or applications.

[0024] The robot 106 is an execution agent that executes processes embedded in the designer 102. One commercial example of some embodiments of the robot 106 is UiPath Robots™. Types of robots 106 may include, but are not limited to, attended robots 108 and unattended robots 110. Attended robots 108 are triggered by a user or user events and run alongside a human user on the same computing system. Attended robots 108 help human users accomplish various tasks and may be triggered directly by human users and / or user events. For attended robots, the conductor 104 may provide a centralized process deployment and logging medium. In certain embodiments, attended robots 108 can only be started from a “robot tray” in a web application or from a command prompt. Unattended robots 110 run hands-free in a virtual environment and can be used to automate many processes, for example for high volume back-end processes. The unattended robot 110 may be responsible for providing remote execution, monitoring, scheduling, and work queue support. Both attended and unattended robots may automate a variety of systems and applications, including, but not limited to, mainframes, web applications, VMs, enterprise applications (e.g., those produced by SAP®, SalesForce®, Oracle®, etc.), and computing system applications (e.g., desktop and laptop applications, mobile device applications, wearable computer applications, etc.).

[0025] In some embodiments, the robot 106 installs by default as a service managed by the Microsoft Windows Service Control Manager (SCM). As a result, such a robot 106 may open an interactive Windows session under the local system account and have the privileges of a Windows service. In some embodiments, the robot 106 may be installed in user mode under which the robot 106 is installed and has the same rights as the user.

[0026] The robot 106 in some embodiments is divided into several components, each dedicated to a specific task. The robot components in some embodiments include, but are not limited to, an SCM-managed robot service, a user-mode robot service, an executor, an agent, and a command line. The SCM-managed robot service manages and monitors the Windows session and acts as a proxy between the conductor 104 and the execution host (i.e., the computing system on which the robot 106 executes). Such a service is entrusted with and manages the credentials of the robot 106. The console application is launched by the SCM under the local system. The user-mode robot service in some embodiments manages and monitors the Windows session and acts as a proxy between the conductor 104 and the execution host. The user-mode robot service may be entrusted with and manage the credentials of the robot 106. If the SCM-managed robot service is not installed, a Windows application may be launched automatically. An Executor may run a given job under a Windows session (e.g., an Executor may run a workflow) and may be aware of per-monitor dots per inch (DPI) settings. An Agent may be a Windows Presentation Foundation (WPF) application that displays available jobs in a system tray window. An Agent may be a client of this service. An Agent may request to start or stop a job and may change settings. A Command Line is a client of the service. A Command Line is a console application that can request to start a job and wait for its output. Splitting up the robot components can help developers, support users, and allow the computing system to more easily execute, identify, and track what each robot component is doing.Special behavior may be configured for each robot component, for example by setting different firewall rules for executors and services. As a further example, in some embodiments, the executor may be aware of per-monitor DPI settings, so that a workflow may be executed at any DPI, regardless of the configuration of the computing system on which the workflow was created.

[0027] Figure 2 illustrates an RPA system 200 according to one or more embodiments. RPA system 200 may be, or may be a part of, RPA system 100 of Figure 1. It should be noted that the "client side," "server side," or both may include any desired number of computing systems without departing from the scope of the present invention.

[0028] As shown on the client side in this embodiment, computing system 202 includes one or more executors 204, agents 206, and designers 208. In alternative embodiments, designers 208 may not be running on the same computing system 20. Executors 204 execute processes (which may be robotic components as described above), and in some embodiments, multiple business processes may be running simultaneously. In such an example, agent 206 (e.g., a Windows service, etc.) is a single point of contact for managing executors 204.

[0029] In some embodiments, a Robot represents an association between a machine name and a username. A Robot may manage multiple executors simultaneously. In a computing system that supports multiple interactive sessions running simultaneously (e.g., Windows Server 2012, etc.), multiple Robots may run simultaneously (e.g., in a High Density (HD) environment), each in a separate Windows session using a unique username.

[0030] The agent 206 is also responsible for sending the status of the robot (e.g., periodically sending "heartbeat" messages to indicate that the robot is still functioning) and downloading the necessary versions of packages to be executed. In some embodiments, communication between the agent 206 and the conductor 212 is initiated by the agent 206. In an example notification scenario, the agent 206 may open a WebSocket channel that is later used by the conductor 212 to send commands to the robot (e.g., start, stop, etc.).

[0031] As shown on the server side in this embodiment, the presentation layer includes web application 214, Open Data Protocol (OData) Representative State Transfer (REST) ​​Application Programming Interface (API) endpoints 216, and notification and monitoring API 218. The server-side service layer includes API implementation / business logic 220. The server-side persistence layer includes database server 222 and indexer server 224. The conductor 212 includes web application 214, OData REST API endpoints 216, notification and monitoring API 218, and API implementation / business logic 220.

[0032] In various embodiments, most actions performed by a user in the conductor 212 interface (e.g., via browser 210) are performed by calling various APIs. Such actions may include, but are not limited to, starting a job on a robot, adding / removing data in a queue, scheduling a job for touchless execution, etc. The web application 214 is the visual layer of the server platform. In such embodiments, the web application 214 uses Hypertext Markup Language (HTML) and JavaScript (JS). However, any desired markup language, scripting language, or any other format may be used without departing from the scope of the invention. In such embodiments, a user interacts with web pages from the web application 214 via browser 210 to perform various actions to control the conductor 212. For example, a user may create robot groups, assign packages to robots, analyze logs per robot and / or per process, start and stop robots, etc.

[0033] In addition to the web application 214, the conductor 212 also includes a services layer that exposes an OData REST API endpoint 216 (alternatively, other endpoints may be implemented without departing from the scope of the present invention). The REST API is used by both the web application 214 and the agent 206. In this exemplary configuration, the agent 206 is a supervisor of one or more robots on a client computer.

[0034] The REST API in such an embodiment covers configuration, logging, monitoring, and queuing functionality. In some embodiments, a configuration REST endpoint may be used to define and configure application users, permissions, robots, assets, releases, and environments. A logging REST endpoint may be used to log various information, such as errors, explicit messages sent by the robot, and other environment specific information. A deployment REST endpoint may be used by the robot to query the package version that needs to be executed when a start job command is used in the conductor 212. A queuing REST endpoint may be responsible for managing queues and queue items, such as adding data to the queue, retrieving transactions from the queue, and setting the status of transactions. A monitoring REST endpoint may monitor the web application 214 and the agents 206. A notification and monitoring API 218 may be a REST endpoint used to register agents 314, deliver configuration settings to the agents 206, and send and receive notifications from the server and agents 206. In some embodiments, the notification and monitoring API 218 may also use WebSocket communication.

[0035] The server-side persistence layer includes a pair of servers in this exemplary embodiment: a database server 222 (e.g., SQL Server, etc.) and an indexer server 224. The database server 222 in this embodiment stores configurations for robots, robot groups, associated processes, users, roles, schedules, etc. Such information is managed through a web application 214 in some embodiments. The database server 222 may manage queues and queue items. In some embodiments, the database server 222 may store messages logged by the robots (in addition to or instead of the indexer server 224). The indexer server 224, which is optional in some embodiments, stores and indexes information logged by the robots. In certain embodiments, the indexer server 224 may be disabled through a configuration setting. In some embodiments, the indexer server 224 uses ElasticSearch, a full-text search engine from an open source project. Messages logged by the robot (e.g., using activities such as log messages, write lines, etc.) may be sent via a logging REST endpoint to the indexer server 224, where they may be indexed for future use.

[0036] FIG. 3 is an architecture diagram illustrating a simplified example deployment of an RPA system 300 according to one or more embodiments of the present invention. In some embodiments, the RPA system 300 may be or may include the RPA systems 100 and / or 200 of FIGS. 1 and 2, respectively. The RPA system 300 includes multiple client computing systems 302 that execute robots. The computing systems 302 can communicate with a conductor computing system 304 via web applications executed thereon. The conductor computing system 304 in turn communicates with a database server 306 and an optional indexer server 308. With respect to FIGS. 2 and 3, it should be noted that although web applications are used in these embodiments, any suitable client / server software may be used without departing from the scope of the present invention. For example, the conductor may execute a server-side application that communicates with a non-web-based client software application on the client computing system.

[0037] The RPA system 100 of FIG. 1, the RPA system 200 of FIG. 2, and / or the RPA system 300 of FIG. 3 may be implemented to automatically perform various RPA tasks using one or more RPA robots. One important RPA task is document processing. Many document processing tasks, such as document digitization, require splitting an electronic file into various sub-documents in order to perform further downstream document processing on the sub-documents. Each sub-document may correspond to, for example, an invoice, a report, an insurance form, etc. The embodiments described herein provide for splitting an electronic file into sub-documents using a trained machine learning based model. Each part of an electronic file is generally labeled using a trained machine learning based model. This labeling helps determine the inclusion of parts between two consecutive sub-documents. Advantageously, the embodiments described herein provide more accurate document segmentation results without the drawbacks of traditional keyword-based document segmentation approaches, thereby resulting in higher quality downstream document processing.

[0038] 4 illustrates a method 400 for dividing an electronic computer file into sub-documents according to one or more embodiments. Method 400 may be performed by one or more suitable computing devices, such as, for example, computing system 700 of FIG.

[0039] In step 402, an electronic file is received. The electronic file may be a computer file of any suitable format, such as, for example, a portable document format (PDF). The electronic file may be received by loading a previously stored electronic file from a storage device or memory of the computer system, or by receiving an electronic file sent from a remote computer system.

[0040] An electronic file includes multiple parts. In one embodiment, each part of an electronic file corresponds to a page of the electronic file. However, it should be understood that a part of an electronic file may be any suitable part, such as, for example, a paragraph of an electronic file, a section of an electronic file, etc. The parts of an electronic file may be associated with sub-documents of various document types. For example, a part may be associated with a sub-document, such as, for example, an invoice, a report, an insurance form, or any other suitable sub-document.

[0041] 5 illustrates a block diagram representation of an electronic file 500, according to one or more embodiments. The electronic file 500 may be the electronic file received in step 402 of FIG. 4. The electronic file 500 includes pages 502-A, 502-B, 502-C, . . . , 502-X, 502-Y, 502-Z (collectively referred to herein as pages 502). The pages 502 may include any number of pages of the electronic file 500. Although the pages 502 are represented as pages of the electronic file 500, it should be understood that the pages 502 may be any portion of the electronic file 502 (e.g., paragraphs, sections, etc.). Each of the pages 502-A, 502-B, 502-C, ..., 502-X, 502-Y, and 502-Z includes a plurality of words 504-A, 504-B, 504-C, ..., 504-X, 504-Y, and 504-Z (collectively referred to as words 504 in this specification).

[0042] Returning to FIG. 4 , at step 404, the portions of the electronic file are classified using the trained machine learning based model. The classification represents the relative location of the portion within a sub-document of the electronic file. In one embodiment, the classification is in an IOB (inside-outside-begin or inside-other-begin) format and represents the relative location of the portion within the sub-document as either an inside classification, an outside (or other) classification, or a beginning classification. The beginning classification represents the first portion of the sub-document, the outside classification represents the last portion of the sub-document, and the inside classification represents the portion between the first and last portions. However, the classification may include other suitable classifications that represent the relative location of the portion within the sub-document.

[0043] The trained machine learning based model receives an electronic file as input. Each portion of the electronic file may contain any number of words. However, the trained machine learning based model considers only a predetermined number of words (including meta-functions of words). This predetermined number of words is empirically determined during training of the machine learning based model. Thus, for portions that have fewer than the predetermined number of words, such portions are null padded by adding null words until the predetermined number of words is reached. For portions that exceed the predetermined number of words, words are selected for consideration by the trained machine learning based model from the top of the portion down and from the bottom of the portion up until the predetermined number of words is reached. Words may be selected equally from the top and bottom of the portion, or weighted to select more words from the top of the portion or more words from the bottom of the portion. For example, if the predetermined number of words is 250 per page and a particular page has 400 words, 70% of the predetermined 250 words may be selected from the top of the page and 30% of the predetermined 250 words may be selected from the bottom of the page. In this example, 175 words would be selected from the top of the page and 75 words would be selected from the bottom of the page. By selecting words from the top and bottom of the portion, text metadata and / or properties are prioritized over actual words and useful information in the header and footer of the portion (e.g., page, etc.) is preserved. The predetermined number of words may be determined in a prior offline or training phase of the machine learning based model. In one embodiment, the predetermined number of words is determined as the median number of words in the portion of the training dataset.

[0044] The trained machine learning based model extracts features of interest from each part of the electronic file and maps the extracted features of interest for each of the parts to a classification, thereby classifying the parts of the electronic file. The features of interest may be represented in any suitable format. In one embodiment, the features of interest are included as part of a tensor. For example, the tensor may be a three-dimensional tensor that includes a list of the parts as a first dimension, a word for each part as a second dimension, and a set of text-related features for each word as a third dimension.

[0045] The features of interest may be features related to the Document Object Model (DOM). For example, in one embodiment, the features of interest include a word cloud for each portion of the electronic file. The word cloud is a representation of the extent of the portion based on the words in the portion. The word cloud is generated based on word embeddings. Word embeddings are numeric vector representations of the words in the portion extracted using, for example, the global vectors for word representation (GloVe) algorithm. In another embodiment, the features of interest include a number or length of pages. For example, the page number "Page 2 of 5" may be identified in the footer or header of the page to classify the page as inside a subdocument that totals 5 pages. In another embodiment, the features of interest include text-related features. Text-related features include any features related to the text of the portion, such as, for example, the size of the characters of the text (e.g., lowercase, uppercase, etc.), the font or format of the text (e.g., word height, width, font style, length (e.g., determined using a bounding box), distance from the top or bottom of the text), the type of section (e.g., paragraph, table, header), or any other suitable text-related feature. In one embodiment, features of interest extracted from electronic files are normalized before being mapped to classifications by a trained machine learning based model.

[0046] The output of the trained machine learning based model can be a classification vector that identifies a classification for each part of the electronic file. In one embodiment, the classification vector is a one-hot encoded classification vector in the format [inside outside begin], where a vector of 1 indicates a positive classification and a vector of 0 indicates a negative classification. For example, a vector of "1 0 0" for a part indicates that the part is classified as inside, a vector of "0 1 0" indicates that the part is classified as outside, and a vector of "0 0 1" indicates that the part is classified as begin. The classification vector includes a vector for each part in the electronic file. In one example, the classification vector is [

[0001] ,

[0100] ,

[0100] ,

[0010] ], indicating that the first part is classified as begin, the second part is classified as inside, the third part is classified as inside, and the fourth part is classified as outside.

[0047] The trained machine learning based model may be any suitable machine learning based model, such as, for example, a neural network based model, an artificial neural network based model, etc. In one embodiment, the machine learning based model is a deep learning based model. In one embodiment, the machine learning based model may be implemented using a LSTM (Long Short-Term Memory) RNN (Recurrent Neural Network) architecture. The LSTM RNN provides long-term memory controlled by opening and closing input gates, output gates, and / or forget gates. Thus, the LSTM RNN allows for storage and subsequent retrieval of encoded features, thereby providing classification of parts of an electronic file based on encoded features from classification of a previous part of the electronic file. In another embodiment, the machine learning based model may be implemented using a Bi-LSTM (Bi-directional LSTM) RNN architecture. The Bi-LSTM RNN allows for bidirectional communication, providing classification of parts of an electronic file based on encoded features from classification of a previous part and encoded features from classification of a next part in the electronic file. In another embodiment, the machine learning based model may be implemented using a seq2seq (sequence-to-sequence) network architecture.

[0048] The machine learning based model is trained to classify the portions of the electronic file using a training dataset in a prior offline or training phase. In one embodiment, the machine learning based model is trained according to method 600 of FIG. 6, which is described in more detail below. Once trained, in step 404, the trained machine learning based model is applied to unseen data in an online or predictive phase to classify the portions of the electronic file.

[0049] In one embodiment, after classifying each of the multiple parts of the electronic file by the trained machine learning based model, a statistical checker is applied to detect misclassified parts. This detection is performed based on the assumption that the softmax scores for correct predictions in the pre-training stage follow a Gaussian or normal distribution. In the pre-training stage, for each classification, assuming a Gaussian distribution, the minimum, maximum, mean and standard deviation of the distribution are calculated using the observed softmax value. For new predictions (e.g., classification of parts in step 404), the softmax p is obtained from a trained machine learning based model. The threshold or cutoff value is calculated according to Equation (1) as follows: Threshold = CEILING (average - standard deviation) Formula (1) where CEILING is rounding the value up to the nearest integer, and the mean and standard deviation are calculated in a pre-training phase. The statistical checker is the softmax p and provides guidance based on the calculated threshold. The guidance is based on the softmax p The guidance may be binary such that if the value of is less than the calculated threshold, the predicted classification by the trained machine learning based model is discarded, otherwise the predicted classification is valid. If the statistical checker identifies a misclassified portion, an electronic file containing the misclassified portion may be presented to a user for manual classification.

[0050] At step 406, the electronic file is divided into sub-documents based on the relative positions of the parts. Each sub-document includes a sequence of one or more parts. In one embodiment, the electronic file is divided just before each part classified as being the beginning part of a sub-document. For example, the electronic file may be divided just before each part having a beginning classification. Thus, the electronic file may be divided, for example, by extracting the sequence of parts between each divided part from the electronic file as a sub-document.

[0051] It should be understood that the sequence of parts for each sub-document need not include a part classified as an beginning, one or more parts classified as interior, and a part classified as exterior. For example, a sub-document containing a single part may have one part classified as an beginning. In another example, a sub-document containing a sequence of two parts may have one part classified as an beginning and the other part classified as exterior.

[0052] The sub-document is output at step 408. The sub-document may be output, for example, by causing the sub-document to be displayed on a display device of the computer system (e.g., display 710 of FIG. 7) or by storing the sub-document in a memory or storage device of the computer system (e.g., memory 706 of FIG. 7).

[0053] In one embodiment, the sub-documents are output for further document processing to perform the RPA task. In one example, the sub-documents are output to a classifier for classifying the sub-documents according to document type, such as a bill, a report, an insurance form, or any other suitable document type. The classifier can be any suitable classifier. In one example, the classifier is a machine learning based classifier.

[0054] FIG. 6 illustrates a method 600 for training a machine learning based model to classify portions of an electronic file, according to one or more embodiments. The steps of method 600 are performed in an offline or training phase. Once trained, the trained machine learning based model is applied in an online or prediction phase to classify portions of an electronic file. In one embodiment, the trained machine learning based model trained according to method 600 may be applied in a prediction phase at step 404 of FIG. 4 to classify portions of an electronic file. It should be understood that features and embodiments described with respect to application of the trained machine learning based model (e.g., in the prediction phase of method 400 of FIG. 4) may also be applicable for training the machine learning based model (e.g., in the training phase of method 600). Method 600 may be performed by one or more suitable computing systems, such as computing system 700 of FIG. 7.

[0055] At step 602, a training dataset is received. The training dataset includes one or more electronic training files, each of which includes one or more training sub-documents. Each training sub-document is identified within the training dataset, enabling a machine learning based model to infer relative positions of parts within the sub-documents without requiring manual annotation.

[0056] At step 604, features of interest are extracted from the training dataset, which may include the features of interest described above with respect to step 404 of Figure 4. For example, the features of interest may include word clouds, page counts or lengths, or text-related features for portions of the training subdocuments.

[0057] At step 606, a machine learning based model is trained to classify the portions of the training dataset based on the extracted features of interest. The machine learning based model may be a deep learning based model or any other suitable machine learning based model. In one embodiment, the machine learning based model may be implemented using an LSTM RNN, a Bi-LSTM RNN, or a seq2seq network architecture. During training, the machine learning based model learns a mapping between the extracted features of interest and a classification. In one embodiment, the classification is in IOB format, but may be any other suitable classification that represents the relative position of the portions within the subdocument.

[0058] At step 608, the trained machine learning based model is output. The trained machine learning based model may be output, for example, by storing the trained machine learning based model in a memory or storage device of the computer system (e.g., memory 706 of FIG. 7). The trained machine learning based model may then be retrieved from the memory to classify portions of an electronic file during an online or predictive phase, such as at step 404 of FIG. 4.

[0059] The embodiments described herein have been experimentally validated using an available dataset of approximately 52,000 pages with a total of 540 subdocuments, and the experimental validation results showed a training accuracy rate of 88% and a validation accuracy rate of 83%.

[0060] Advantageously, the embodiments described herein provide for splitting electronic files into sub-documents using a trained machine learning based model without requiring prior annotation or partitioning of a training dataset to train the machine learning based model. Once trained, no retraining of the trained machine learning based model is required before it is applied in the prediction stage. The embodiments described herein provide a cost-effective solution for splitting electronic files without requiring manual review by a user, and also avoids the shortcomings of traditional keyword-based document segmentation approaches.

[0061] FIG. 7 is a block diagram illustrating a computing system 700 configured to perform the methods, workflows, and processes described herein, including the methods illustrated in FIG. 4 and FIG. 6, according to an embodiment of the present invention. In some embodiments, the computing system 700 may be one or more of the computing systems illustrated and / or described in this application. The computing system 700 includes a bus 702 or other communication mechanism for communicating information and a processor 704 coupled to the bus 702 for processing information. The processor 704 may be any type of general-purpose or special-purpose processor, including a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a graphics processing unit (GPU), examples thereof, and / or any combination thereof. The processor 704 may also have multiple processing cores, at least some of which may be configured to perform a particular function. In some embodiments, multiple parallel processing may be used.

[0062] The computing system 700 further includes a memory 706 for storing information and instructions executed by the processor 704. The memory 706 may be comprised of any combination of random access memory (RAM), read-only memory (ROM), flash memory, cache, static storage devices such as magnetic or optical disks, or any other type of non-transitory computer readable medium, or any combination thereof. The non-transitory computer readable medium may be any available medium that can be accessed by the processor 704 and may include volatile media, non-volatile media, or both. The media may be removable, non-removable, or both.

[0063] Additionally, the computing system 700 includes a communications device 708, such as a transceiver, to provide access to a communications network via wireless and / or wired connections according to any currently existing or future implemented communications standards and / or protocols.

[0064] The processor 704 is further connected via bus 702 to a display 710 suitable for displaying information to a user. The display 710 may also be configured as a touch display and / or any suitable tactile I / O device.

[0065] A keyboard 712 and cursor control device 714, e.g., a computer mouse, touchpad, etc., are further connected to bus 702 to allow a user to interface with the computing system. However, in certain embodiments, a physical keyboard and mouse may not be present and the user may interact with the device solely through the display 710 and / or a touchpad (not shown). Any type and combination of input devices may be used as a matter of design choice. In certain embodiments, no physical input devices and / or displays are present. For example, a user may interact with computing system 700 remotely via another computing system that communicates with computing system 700, or computing system 700 may operate autonomously.

[0066] The memory 706 stores software modules that, when executed by the processor 704, provide functionality, including an operating system 716 for the computing system 700, and one or more additional functional modules 718 configured to perform all or a portion of the processes described herein or derivatives thereof.

[0067] Those skilled in the art will appreciate that the "system" may be embodied as a server, an embedded computing system, a personal computer, a console, a personal digital assistant (PDA), a mobile phone, a tablet computing device, a quantum computing system, any other suitable computing device, or a combination of devices, without departing from the scope of the present invention. The depiction of the above functions as being performed by a "system" is in no way intended to limit the scope of the present invention, but is intended to illustrate one example of many embodiments of the present invention. Indeed, the methods, systems, and apparatus disclosed herein may be implemented in a localized and distributed fashion consistent with computing technologies, including cloud computing systems.

[0068] It should be noted that some of the system functions described herein are illustrated as modules to better emphasize implementation independence. For example, the modules may be implemented as hardware circuits, including custom very large scale integrated (VLSI) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. The modules may be implemented in programmable hardware devices, such as field programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, and the like. The modules may be implemented at least in part in software for execution by various types of processors. For example, an identified unit of executable code may include one or more physical or logical blocks of computer instructions, which may be organized as, for example, an object, procedure, or function. Notwithstanding this, the executable files of the identified modules need not be physically located together, but may include disparate instructions stored in various locations that, when logically combined, comprise the modules and achieve the above-mentioned purpose of the modules. Furthermore, the modules may be stored on a computer readable medium without departing from the scope of the present invention, which may be, for example, a hard disk drive, a flash device, a RAM, a tape, and / or other such non-transitory computer readable medium used to store data. Indeed, a module of executable code may be a single instruction or many instructions, and may be distributed across several different code segments, among different programs, and across several memory devices. Similarly, operational data may be identified and depicted herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be organized as a single data set, or may be distributed in different locations, including different storage devices, or may exist, at least in part, simply as electronic signals on a system or network.

[0069] The above merely illustrates the principles of the present disclosure. Thus, those skilled in the art will understand that, although not explicitly described or shown herein, various configurations may be devised that embody the principles of the present disclosure and fall within its spirit and scope. Furthermore, all examples and conditional language described herein are intended primarily for educational purposes only to help the reader understand the principles of the present disclosure and the concepts provided by the inventors to develop the present technology, and should be interpreted as not being limited to such specifically described examples and conditions. Furthermore, all descriptions herein that describe the principles, aspects, and embodiments of the present disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Furthermore, such equivalents are intended to include both currently known equivalents and equivalents developed in the future.

Claims

1. 1. A computer-implemented method comprising: The method comprises: classifying a plurality of portions of an electronic file using a trained machine learning based model with classifications representative of relative positions of the plurality of portions within sub-documents of the electronic file; Dividing the electronic file into the sub-documents based on the relative positions of the portions; outputting the sub-document; The method according to claim 1, further comprising:

2. 2. The method of claim 1, wherein classifications representing relative positions of the plurality of portions within the subdocument of the electronic file include a classification representing a first portion of the subdocument, a classification representing a last portion of the subdocument, and a classification representing a portion between the first and last portions of the subdocument.

3. Using a trained machine learning-based model to classify multiple parts of an electronic file Mapping features of interest extracted from each of the plurality of portions of the electronic file to the classification. Including, The method of claim 1 , wherein the features of interest include one or more of a word cloud, a page count, or text-related features.

4. Using a trained machine learning-based model to classify multiple parts of an electronic file detecting misclassified portions from the plurality of classified portions using a statistical checker; and presenting the misclassified portions to a user for manual classification; The method of claim 1 further comprising:

5. Dividing the electronic file into the sub-documents based on the relative positions of the portions includes: Dividing the electronic file immediately before each portion classified as being the first portion of a subdocument. The method of claim 1 , comprising:

6. The method of claim 1 , wherein the plurality of portions of the electronic file correspond to a plurality of pages of the electronic file.

7. 2. The method of claim 1 , wherein the trained machine learning based model comprises a trained deep learning model.

8. 2. The method of claim 1, wherein the trained machine learning based model is based on one of the following architectures: LSTM (Long Short-Term Memory), Bi-LSTM (Bi-directional LSTM), and seq2seq (sequence-to-sequence) architectures.

9. classifying the subdocuments using a classifier The method of claim 1 further comprising:

10. a memory for storing computer instructions; at least one processor configured to execute the computer instructions; Equipped with The computer instructions include: classifying a plurality of portions of an electronic file using a trained machine learning based model with classifications representative of relative positions of the plurality of portions within sub-documents of the electronic file; Dividing the electronic file into the sub-documents based on the relative positions of the portions; outputting the sub-document; 5. An apparatus configured to cause the at least one processor to execute:

11. 11. The apparatus of claim 10, wherein the classifications representing the relative positions of the portions within the subdocument of the electronic file include a classification representing a first portion of the subdocument, a classification representing a last portion of the subdocument, and a classification representing a portion between the first and last portions of the subdocument.

12. Using a trained machine learning-based model to classify multiple parts of an electronic file Mapping features of interest extracted from each of the plurality of portions of the electronic file to the classification. Including, The apparatus of claim 10 , wherein the features of interest include one or more of a word cloud, a page count, or text-related features.

13. Using a trained machine learning-based model to classify multiple parts of an electronic file detecting misclassified portions from the plurality of classified portions using a statistical checker; and presenting the misclassified portions to a user for manual classification; The apparatus of claim 10 further comprising:

14. Dividing the electronic file into the sub-documents based on the relative positions of the portions includes: Dividing the electronic file immediately before each portion classified as being the first portion of a subdocument. The apparatus of claim 10, comprising:

15. A computer program stored on a non-transitory computer readable medium, comprising: The computer program comprises: classifying a plurality of portions of an electronic file using a trained machine learning based model with classifications representative of relative positions of the plurality of portions within sub-documents of the electronic file; Dividing the electronic file into the sub-documents based on the relative positions of the portions; outputting the sub-document; A computer program product configured to cause at least one processor to execute the

16. 16. The computer program product of claim 15, wherein the classifications representing relative positions of the plurality of portions within the subdocument of the electronic file include a classification representing a first portion of the subdocument, a classification representing a last portion of the subdocument, and a classification representing a portion between the first and last portions of the subdocument.

17. 16. The computer program product of claim 15, wherein the plurality of portions of the electronic file correspond to a plurality of pages of the electronic file.

18. 16. The computer program product of claim 15, wherein the trained machine learning based model comprises a trained deep learning model.

19. 16. The computer program product of claim 15, wherein the trained machine learning based model is based on one of a long short-term memory (LSTM) architecture, a bidirectional LSTM (Bi-LSTM) architecture, and a sequence-to-sequence (seq2seq) architecture.

20. classifying the subdocuments using a classifier 16. The computer program of claim 15, further comprising:

Images