Automatic Representation and Extraction of Tabular Data in Portable Document Formats Using Graph Neural Networks

Abstract

A method is disclosed for automated depiction and extraction of portable document format (PDF) tabular data. The method includes one or more processors extracting metadata corresponding to the text-based portable document format (PDF) tabular data, the metadata being associated with characters and borders of the tabular data. The method further includes generating a graph structure corresponding to the text-based PDF tabular data based at least in part on the metadata. The method further includes generating a vector representation of the graph structure. The method further includes constructing a tree structure corresponding to the tabular data based at least in part on the vector representation.

Description

[Technical Field]

[0001] The present invention relates generally to the field of machine learning, and more particularly to the delineation and extraction of tabular data in portable document formats. [Background technology]

[0002] Many learning tasks require working with graph data, which contains rich relational information between elements. Graph neural networks (GNNs) are connectionist models that capture graph dependencies through message passing between nodes in the graph. Unlike standard neural networks, graph neural networks maintain state that can represent information from neighbors at any depth. While primitive GNNs have proven difficult to train due to their fixed points, recent advances in network architectures, optimization techniques, and parallel computing have enabled successful learning using them. In recent years, systems based on variants of graph neural networks, such as graph convolutional networks (GCNs), graph attention networks (GATs), and gated graph neural networks (GGNNs), have shown breakthrough performance in many of the tasks mentioned above.

[0003] Graph Attention Networks (GATs) are novel neural network architectures that operate on graph-structured data and utilize masked self-attention layers. By stacking layers that allow nodes to attend to neighboring features, it becomes possible to (e.g., implicitly) specify different weights for different neighboring nodes without requiring any kind of expensive matrix operations (e.g., matrix inversion) or relying on prior knowledge of the graph structure.

[0004] A Delaunay triangulation (also known as a Delone triangulation) for a given set of discrete points "P" in the plane is a triangulation DT(P) such that no point in "P" is within the circumcenter of any triangle in "DT(P)". The Delaunay triangulation of a discrete set of points "P" in general positions corresponds to the dual graph of the Voronoi diagram for "P". The circumcenters of the Delaunay triangles are the vertices of the Voronoi diagram. In the 2D case, the Voronoi vertices are connected via edges that can be obtained from the adjacency relations of the Delaunay triangles. Summary of the Invention

[0005] Aspects of the present invention disclose methods, computer program products, and systems for automated rendering and extraction of tabular data in a Portable Document Format (PDF). The method includes one or more processors extracting metadata corresponding to the tabular data in a text-based Portable Document Format (PDF), the metadata being associated with characters and borders of the tabular data. The method further includes one or more processors generating a graph structure corresponding to the tabular data in the text-based PDF based at least in part on the metadata. The method further includes one or more processors generating a vector representation of the graph structure. The method further includes one or more processors constructing a tree structure corresponding to the tabular data based at least in part on the vector representation. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a functional block diagram of a data processing environment in accordance with an embodiment of the present invention; [Figure 2]2 is a flowchart illustrating the operational steps of a program within the data processing environment of FIG. 1 for automated rendering and extraction of tabular data in Portable Document Format (PDF) in accordance with an embodiment of the present invention. [Figure 3A] FIG. 10 illustrates an example of a table in a text-based PDF, according to an embodiment of the present invention. [Figure 3B] FIG. 10 is an illustration of an exploded view of a table in a text-based PDF document, according to an embodiment of the present invention. [Figure 3C] FIG. 1 illustrates an example tree structure corresponding to a table in a text-based PDF document, according to an embodiment of the present invention. [Figure 4] FIG. 2 is a block diagram of the components of FIG. 1 in accordance with an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0007] Embodiments of the present invention enable automated depiction and extraction of tabular data in Portable Document Format (PDF). Embodiments of the present invention convert tabular data in a text-based PDF format into a structured format that formulates the tabular data PDF file as a graph in which each character is treated as a node. Embodiments of the present invention encode the nodes of the generated graph into a vector representation. Additional embodiments of the present invention create a tree structure corresponding to the tabular data from one or more node embeddings and adjacency relationships. Yet another embodiment of the present invention assigns attributes to leaves of the generated tree structure.

[0008] Over 3 trillion documents are available in Portable Document Format (PDF), and these documents can contain many tables that summarize information more effectively than can be conveyed in natural language. PDF documents can be displayed on a screen or printed on paper at any time, but they are not machine-readable. As a result, efforts have been made to make these documents machine-readable and to enable automatic processing by natural language processing algorithms.

[0009] Some embodiments of the present invention recognize that rule-based approaches for understanding tables in PDF documents are limited by the capabilities of the rule generator, are not trainable, and as a result, cannot be automatically optimized for new document domains. Furthermore, machine learning approaches require converting tabular regions of PDF documents into images and analyzing them with computer vision models, which face issues related to resolution and graphics processing unit (GPU) memory and do not utilize the valuable metadata of PDF document files. Various embodiments of the present invention ameliorate these issues by utilizing the metadata of PDF documents to formulate a graph of the tabular data and encode the nodes and edges of the graph. Furthermore, embodiments of the present invention utilize the encoded graph features to reconstruct a tree structure of the tabular data and determine the attributes of the leaves of the tree structure.

[0010] Embodiments of the present invention recognize that computing systems experience GPU memory challenges while processing tabular data through computer vision models. Various embodiments of the present invention may operate to increase the efficiency of computing systems by utilizing a graph neural network (GNN)-based solution to convert tabular data in a text-based PDF format into a structured format using metadata corresponding to the tabular data. As a result, the amount of processing resources utilized is reduced by eliminating the GPU memory challenges associated with converting tabular data in PDF files into images and extracting information from the converted images.

[0011] Implementation of embodiments of the present invention may take a variety of forms, and details of exemplary implementations are described below with reference to the figures.

[0012] The present invention will now be described in detail with reference to the figures. Figure 1 is a functional block diagram illustrating a distributed data processing environment, generally designated 100, in accordance with one embodiment of the present invention. Figure 1 is intended only to provide an illustration of one implementation and is not intended to imply any limitation with respect to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made by one of ordinary skill in the art without departing from the scope of the present invention, as set forth in the claims.

[0013] The present invention can include various accessible data sources, such as databases 144, that can contain personal data, content, or information that a user does not want processed. Personal data includes user information, such as personally identifiable information or sensitive personal information, as well as tracking or geolocation information. Processing refers to any automated or non-automated operation or set of operations performed on personal data, such as collection, recording, organization, structuring, storage, adaptation, alteration, retrieval, consultation, use, disclosure by transmission, dissemination, or otherwise making available, combination, restriction, erasure, or destruction. The table program 200 enables the authorized and secure processing of personal data. The table program 200 provides informed consent, notifies the user of the collection of personal data, and allows the user to opt in or out of the processing of their personal data. Consent can take several forms. Opt-in consent can require the user to take an affirmative action before their personal data is processed. Alternatively, opt-out consent can require the user to take an affirmative action to prevent the processing of their personal data before their personal data is processed. The tabulation program 200 provides information about the personal data and the nature of the processing (e.g. type, scope, purpose, duration, etc.). The tabulation program 200 provides the user with a copy of the stored personal data. The tabulation program 200 allows for the correction or completion of inaccurate or incomplete personal data. The tabulation program 200 allows for the immediate deletion of personal data.

[0014] Distributed data processing environment 100 includes servers 140 and client devices 120, all interconnected via network 110. Network 110 may be, for example, a telecommunications network, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN) such as the Internet, or a combination of the three, and may include wired, wireless, or fiber optic connections. Network 110 may include one or more wired or wireless networks, or both, capable of receiving and transmitting data, voice, or video signals, including multimedia signals, including voice, data, and video information, or combinations thereof. In general, network 110 may be any combination of connections and protocols supporting communication between servers 140 and client devices 120, as well as other computing devices (not shown), in distributed data processing environment 100.

[0015] Client device 120 may be one or more of a laptop computer, a tablet computer, a smartphone, a smart watch, a smart speaker, a virtual assistant, or any programmable electronic device capable of communicating with various components and devices in distributed data processing environment 100 over network 110. Generally, client device 120 represents one or more programmable electronic devices or combinations of programmable electronic devices capable of executing machine-readable program instructions and communicating with other computing devices (not shown) in distributed data processing environment 100 over a network, such as network 110. According to embodiments of the present invention, client device 120 may include components such as those shown and described in further detail with respect to FIG.

[0016] Client device 120 includes a user interface 122 and applications 124. In various embodiments of the present invention, a user interface is a program that provides an interface between a device user and multiple applications residing on the client device. A user interface, such as user interface 122, refers to the information (graphical, textual, audio, etc.) that a program presents to the user and the control sequences the user employs to control the program. Various types of user interfaces exist. In one embodiment, user interface 122 is a graphical user interface. A graphical user interface (GUI) is a type of user interface that allows a user to interact with electronic devices, such as a computer keyboard and mouse, through graphical icons and visual indicators, such as secondary notation, as opposed to text-based interfaces, typed command labels, or text navigation. In computing, GUIs were introduced in response to the recognition of the steep learning curve of command-line interfaces, which require typing commands on a keyboard. Actions in a GUI are often performed through direct manipulation of graphical elements. In another embodiment, user interface 122 is a script or application programming interface (API).

[0017] The application 124 is a computer program designed to run on the client device 120. Applications often serve to provide users with similar services accessed on a personal computer (e.g., a web browser, music playback, email program, or other media). In one embodiment, the application 124 is mobile application software. For example, mobile application software, or "app," is a computer program designed to run on smartphones, tablet computers, and other mobile devices. In another embodiment, the application 124 is a web user interface (WUI) that can display text, documents, web browser windows, user options, application interfaces, and instructions for operation, and can include information (e.g., graphics, text, and audio) that the program provides to the user and control sequences employed by the user to control the program. In another embodiment, the application 124 is a client-side application for the table program 200.

[0018] In various embodiments of the present invention, server 140 may be a desktop computer, a computer server, or any other computer system known in the art. Generally, server 140 represents any electronic device or combination of electronic devices capable of executing computer-readable program instructions. According to embodiments of the present invention, server 140 may include components shown and described in further detail with respect to FIG.

[0019] Server 140 may be a standalone computing device, an administrative server, a web server, a mobile computing device, or any other electronic device or computing system capable of receiving, transmitting, and processing data. In one embodiment, server 140 may represent a server computing system utilizing multiple computers as a server system, such as in a cloud computing environment. In another embodiment, server 140 may be a laptop computer, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a personal digital assistant (PDA), a smartphone, or any programmable electronic device capable of communicating with client devices 120 and other computing devices (not shown) in distributed data processing environment 100 via network 110. In another embodiment, server 140 represents a computer system utilizing clustered computers and components (e.g., database server computers, application server computers, etc.) that function as a single pool of seamless resources when accessed within distributed data processing environment 100.

[0020] Server 140 includes storage device 142, database 144, and table program 200. Storage device 142 may be implemented with any type of storage device, such as persistent storage 405, capable of storing data that can be accessed and utilized by client device 120 and server 140, such as a database server, hard disk drive, or flash memory. In one embodiment, storage device 142 may represent multiple storage devices within server 140. In various embodiments of the present invention, storage device 142 stores multiple types of data, which may include database 144. Database 144 may also represent one or more organized collections of data that are stored and accessed by server 140. For example, database 144 may include PDF files, metadata for PDF files, tabular data, etc. In one embodiment, data processing environment 100 may include additional servers (not shown) that host additional information accessible via network 110.

[0021] In general, the table program 200 can convert tabular data in text-based PDF format into a structured format while avoiding heuristic analysis of PDF data (e.g., words, lines, text boxes, etc.), thereby enabling end-to-end training and processing. In one embodiment, the table program 200 generates a tree structure of tables for documents in the database 144. For example, the table program 200 generates a graph of the table for the PDF document using PDF information (e.g., PDF table metadata) and a Delaunay triangulation. In this example, the table program 200 uses a machine learning algorithm (e.g., an attention GNN) to encode the nodes and edges of the graph into a feature space. The table program 200 uses the encoded graph features to reconstruct the table tree structure, and the table program 200 predicts the sequence of tree operations. Furthermore, the table program 200 predicts the attributes of the leaves (e.g., table cells) of the tree structure. As a result, the PDF information can be utilized by modeling the PDF data as a graph and using a machine learning algorithm to derive insights from the graph. The machine learning algorithms of the table program 200 can include sequential tree builder and cell builder models that can utilize state information nodes that allow state history to be taken into account when making predictions.

[0022] 2 is a flowchart illustrating the operational steps of table program 200, a program that converts tabular data in text-based PDF format into a structured format, according to an embodiment of the present invention. In one embodiment, table program 200 starts in response to a user connecting client device 120 to table program 200 through network 110. For example, table program 200 starts in response to a user registering (e.g., opting in to) their laptop (e.g., client device 120) with table program 200 via a WLAN (e.g., network 110). In another embodiment, table program 200 is a background application that continuously monitors client device 120. For example, table program 200 is a client-side application (e.g., application 124) that starts upon startup of a user's laptop (e.g., client device 120) and monitors the laptop to detect when the user accesses a PDF file containing tabular data (e.g., a table).

[0023] In step 202, the table program 200 extracts metadata for the table of the document. In one embodiment, the table program 200 extracts metadata corresponding to the tabular data of the PDF document from the database 144. For example, the table program 200 extracts information (e.g., metadata) corresponding to a table (e.g., tabular data) from a text-based PDF document. In this example, the table program 200 identifies information including character position and font information and table row positions. Additionally, the table program 200 can utilize PDF analysis to extract metadata for the text-based PDF document, thereby eliminating user heuristics for analyzing text-based PDF documents.

[0024] FIG. 3A shows table 310, an exemplary diagram of a table instance in a text-based PDF document converted by table program 200, according to an embodiment of the present invention. Table 310 includes lines 312, 314, 316, a header 313, and a body 315. Lines 312, 314, and 316 (hereinafter, lines) are the boundaries of table 310. Header 313 corresponds to the row of tabular data table 310 between lines 312 and 314, which is used to represent each column in body 315. Body 315 is the main portion of table 310 and includes categorized numerical information related to row and column captions. In the exemplary embodiment shown in FIG. 3A, table program 200 receives table 310 from client device 120 and stores table 310 in database 144.

[0025] FIG. 3B shows processed table 320, which is an exemplary diagram of an exploded view of table 310 after table program 200 begins extracting metadata, according to an exemplary embodiment of the present invention. Table 320 includes lines 312, 314, and bounding boxes 322. Bounding boxes 322 are bounding boxes that indicate characters identified during metadata extraction. In the exemplary embodiment of FIG. 3B, table program 200 includes one or more instances of bounding boxes 322 corresponding to each character in table 310 of FIG. 3A, as shown in FIG. 3B. In this exemplary embodiment, table program 200 collects font and position information for each instance of bounding box 322 in table 310, as well as position information for the lines in table 310.

[0026] In step 204, the table program 200 generates a graph corresponding to the table. In one embodiment, the table program 200 generates a graph corresponding to the tabular data of a PDF document in the database 144. For example, the table program 200 uses a triangulation technique (e.g., a Delaunay triangulation) and position information (e.g., metadata) of each character in a table (e.g., tabular data) of a text-based PDF document to generate a graph structure corresponding to the characters in the table. In this example, the table program 200 computes "DT(P)," which is the Delaunay triangulation "DT" of a set of discrete points (e.g., each character in the table) "P" such that no point in the set of "P" is within the circumcenter of any triangle in "DT(P)." A Delaunay triangulation of a set of discrete points "P" in general positions corresponds to the dual graph of the Voronoi diagram for "P," where the circumcenters of the Delaunay triangles are vertices "V" of the Voronoi diagram, and in two dimensions (2D), the Voronoi vertices "V" are connected via edges "E," which can be derived from the adjacency relationships of the Delaunay triangles. Furthermore, the table program 200 generates a graph structure corresponding to the table using the vertices "V" and edges "E" (e.g., graph (V,E)), which represent the location and adjacency relationships of each of the characters in the table. Referring now to FIG. 3B , in an exemplary embodiment, the table program 200 generates a graph structure based on the adjacency relationships of one or more instances of the bounding box 322 for the characters in the table 320, using one or more instances of the bounding box 322 as the circumcenters of the Delaunay triangles (e.g., vertices "V").

[0027] In step 206, the table program 200 embeds the nodes and edges of the graph. In one embodiment, the table program 200 uses a machine learning algorithm to generate a vector representation of the generated graph. For example, the table program 200 uses an attention graph neural network (e.g., a machine learning algorithm) to encode the nodes and edges of the generated graph structure (e.g., graph(V,E)) into a feature space. In this example, the table program 200 generates a set of feature vectors corresponding to each node “V” (e.g., character) and each edge “E” of each set of generated graph structures. Furthermore, the table program 200 uses the table metadata and “DT” information to generate a set of feature vectors corresponding to nodes in the set of nodes “V,” which may include features such as character, position information (e.g., bounding box), font features (e.g., bold, italic, etc.), etc. Additionally, the table program 200 utilizes the metadata of the table and the information in "DT" to generate a set of feature vectors corresponding to edges in the set of edges "E", which may include features such as a start node (e.g., "V0"), an end node (e.g., "V1"), an edge category, etc. Additionally, the table program 200 utilizes the identified lines in the table to determine whether edges in "DT" intersect to classify edge "E".

[0028] In another embodiment, the table program 200 trains a machine learning algorithm to build a tree structure that corresponds to the graph of the tabular data of the generated PDF document in the database 144. For example, the table program 200 trains a machine learning algorithm (e.g., an attention graph neural network, a graph neural network, etc.) to create a tree representation (e.g., a tree structure) of the table (e.g., the tabular data) using node embeddings and adjacency relationships in the node embeddings. In this example, the machine learning algorithm may include two or more models (e.g., a tree builder network, a tree builder model, a cell builder model, etc.) that the table program 200 sequentially constructs, thereby enabling a first model (e.g., the cell builder model) to use state information of a second model (e.g., the tree builder model nodes) to predict the content of cells (e.g., leaves) of the tree representation (i.e., jointly training the tree builder model and the cell builder model to reconstruct a structured representation of the tabular data of the PDF document).

[0029] Additionally, the table program 200 can train a tree builder model with a sequence loss of actions, where the actions include tasks such as insert header, insert body, insert row, insert cell, move to parent, exit, etc., when building a tree representation. Additionally, the table program 200 can train a cell builder model with a weighted sum of the sequence loss of actions, the sequence loss of cell content, and the column / row span regression loss to assign cell attributes, where the cell attributes can include a value corresponding to the amount of column span, a value corresponding to the amount of row span, and the cell content. Additionally, training the table 200 of a machine learning algorithm can include two or more attention mechanisms correlated with two or more models, where the attention mechanisms comprise a neural network (e.g., a machine learning algorithm) capable of focusing on a subset of the neural network's inputs (e.g., features).

[0030] In step 208, the table program 200 rebuilds the tree structure of the table. In one embodiment, the table program 200 generates a tree structure corresponding to the generated graph of the tabular data of the PDF document in the database 144. For example, the table program 200 builds a tree-structured representation (e.g., tree representation) of the tabular data (e.g., a table) of the PDF document. In this example, the table program 200 uses the output of a machine learning algorithm (e.g., a tree builder model, a neural network, an AGNN, etc.) to determine which action (e.g., insert header, insert body, insert row, insert cell, etc.) to select when building the tree-structured representation. Additionally, the table program 200 uses the adjacency relationships (e.g., edges) of the generated graph (e.g., graph [V, E]) to determine branch assignments in the tree-structured representation between parent and child nodes.

[0031] FIG. 3C shows structure 330, an exemplary diagram of a table instance in a text-based PDF document that table program 200 converts into a tree structure in accordance with an exemplary embodiment of the present invention. Structure 330 includes table node 331, header node 332, body node 333, multiple instances of row node 334, multiple instances of cell node 335, and multiple instances of branch 336. Table node 331 is the root node corresponding to table 310 in FIG. 3A. Header node 332 is the parent node corresponding to header 313 in FIG. 3A. Row node 334 is a child node corresponding to tabular data (e.g., rows) in table 310 and may include one or more instances of leaves (e.g., cell node 335). Cell node 335 is a leaf node that may include the content (e.g., characters, values, etc.) of table 310 in FIG. 3A. Branch 336 is an element of structure 330 that indicates the adjacency relationship between nodes. Additionally, structure 330 includes a number of ellipsis symbols indicating that one or more elements (nodes, rows, cells, etc.) of the tree structure are missing (i.e., included but not represented). In the exemplary embodiment with reference to Figure 3A, table program 200 utilizes a machine learning algorithm to select actions for reconstructing table 310 in the form of structure 330 (i.e., forming a tree structure).

[0032] In step 210, the table program 200 assigns attributes to the tree structure. In one embodiment, the table program 200 assigns attributes to the generated tree structure corresponding to the tabular data of the PDF document in the database 144. For example, the table program 200 determines cell attributes for one or more leaves (e.g., cells) of a tree-structured representation (e.g., tree representation) of the tabular data (e.g., table) of the PDF document. In this example, the table program 200 utilizes the output of a machine learning algorithm (e.g., a cell builder model, a neural network, an AGNN, etc.) to determine attributes (e.g., column span, row span, content, etc.) to assign when filling the leaves of the tree-structured representation.

[0033] Referring to FIG. 3C , in an exemplary embodiment, the table program 200 utilizes a machine learning algorithm to define attributes of the cell node 335. In this example, the table program 200 assigns character and / or integer values ​​corresponding to the cell attributes (e.g., column span, row span, content, etc.) based on the output of the machine learning algorithm. Furthermore, the column span is the number of columns in the processed table 320 of FIG. 3B that are covered by the content of the collection of one or more nodes in FIG. 3B . The row span is the number of rows in the processed table 320 of FIG. 3B that are covered by the content of the collection of one or more nodes in FIG. 3B . The content is the collection of characters / values ​​of the tabular data of the PDF document in the database 144 (e.g., one or more nodes in the processed table 320). In one scenario, if the content is a collection of nodes including "Mean" in FIG. 3B, the table program 200 assigns the cell attributes for cell node 335 as follows: column span: (1); row span: (1); and content: "Mean".

[0034] 4 illustrates a block diagram of components of client device 120 and server 140 in accordance with an exemplary embodiment of the present invention. It should be understood that FIG. 4 is intended to be illustrative of one implementation and is not intended to imply any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environments may be made.

[0035] 4 includes a processor 401, a cache 403, a memory 402, persistent storage 405, a communication unit 407, an input / output (I / O) interface 406, and a communication fabric 404. The communication fabric 404 provides communication between the cache 403, the memory 402, the persistent storage 405, the communication unit 407, and the input / output (I / O) interface 406. The communication fabric 404 can be implemented with any architecture designed to route data and / or control information between processors (such as microprocessors, communication and network processors), system memory, peripherals, and any other hardware components in the system. For example, the communication fabric 404 can be implemented with one or more buses or crossbar switches.

[0036] Memory 402 and persistent storage 405 are computer-readable storage media. In this embodiment, memory 402 includes random access memory (RAM). In general, memory 402 may include any suitable volatile or non-volatile computer-readable storage medium. Cache 403 is high-speed memory that improves the performance of processor 401 by retaining recently accessed data and data near recently accessed data from memory 402.

[0037] Program instructions and data (e.g., software and data 410) used to implement embodiments of the present invention may be stored in persistent storage 405 and memory 402 for execution by one or more of the respective processors 401 via cache 403. In an embodiment, persistent storage 405 includes a magnetic hard disk drive. Alternatively, or in addition to a magnetic hard disk drive, persistent storage 405 may include a solid-state hard drive, a semiconductor storage device, read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, or any other computer-readable storage medium capable of storing program instructions or digital information.

[0038] The media used by persistent storage 405 may also be removable. For example, a removable hard drive may be used for persistent storage 405. Other examples include optical and magnetic disks, thumb drives, and smart cards inserted into a drive for transfer to another computer-readable storage medium that is also part of persistent storage 405. Software and data 410 may be stored in persistent storage 405 for access and / or execution by one or more of the respective processors 401 via cache 403. For client device 120, software and data 410 includes data for user interface 122 and applications 124. For server 140, software and data 410 includes data for storage device 142 and table program 200.

[0039] In these examples, communications unit 407 provides for communication with other data processing systems or devices. In these examples, communications unit 407 includes one or more network interface cards. Communications unit 407 may provide communications through the use of either or both physical and wireless communications links. Program instructions and data (e.g., software and data 410) used to implement embodiments of the present invention may be downloaded to persistent storage 405 through communications unit 407.

[0040] The I / O interface 406 allows for the input and output of data by other devices that may be connected to each computer system. For example, the I / O interface 406 may provide a connection to external devices 408, such as a keyboard, keypad, touch screen, or any other suitable input device, or a combination thereof. The external devices 408 may also include portable computer-readable storage media, such as thumb drives, portable optical or magnetic disks, and memory cards. Program instructions and data (e.g., software and data 410) used to implement embodiments of the present invention may be stored on such portable computer-readable storage media and loaded into persistent storage 405 via the I / O interface 406. The I / O interface 406 is also connected to a display 409.

[0041] Display 409 provides a mechanism for displaying data to a user and may be, for example, a computer monitor.

[0042] The programs described herein are identified based on the applications for which they are implemented in particular embodiments of the invention. However, it should be understood that any specific program names herein are used merely for convenience, and therefore the invention should not be limited to use with only any particular application identified and / or implied by such names.

[0043] The present invention may be embodied as a system, method, or computer program product, or a combination thereof, at any possible level of technical detail. The computer program product may include a computer-readable storage medium or media having computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0044] A computer-readable storage medium may be any tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disks (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves having instructions recorded thereon, and any suitable combination of the above. As used herein, computer-readable storage media is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses through fiber optic cable), or electrical signals sent through wires.

[0045] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network can include copper cables, optical fibers, wireless networks, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0046] The computer-readable program instructions for carrying out the operations of the present invention may be source or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or object-oriented programming languages ​​such as Smalltalk, C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may run entirely on the user's computer, partially on the user's computer as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer readable program instructions to individualize the electronic circuitry by utilizing state information in the computer readable program instructions to implement aspects of the present invention.

[0047] Aspects of the present invention will be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0048] These computer-readable program instructions can be provided to a computer processor or other programmable data processing apparatus to produce a machine, whereby the instructions, executed by the computer processor or other programmable data processing apparatus, create means for performing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer program instructions, which can direct a computer, programmable data processing apparatus or other device, or combination thereof, to function in a particular manner, can also be stored in a computer-readable medium, whereby the computer-readable storage medium having instructions stored thereon comprises an article of manufacture containing instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0049] The computer-readable program instructions may be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to generate a computer-implemented process, such that the instructions executing on the computer or other programmable apparatus, or other device, perform the functions / operations specified in one or more blocks of the flowcharts or block diagrams, or both.

[0050] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the figures. For example, depending on the functionality involved, two blocks shown in succession may actually be executed concurrently, substantially concurrently, in a partially or fully overlapping manner, or the blocks may sometimes be executed in reverse order. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a special-purpose hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and computer instructions.

[0051] The descriptions of various embodiments of the present disclosure have been presented for illustrative purposes, but they are not intended to be exhaustive or to be limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terms used herein have been selected to best explain the principles, practicality, or technical improvements over commercially available technology of the embodiments, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A computer-implemented method for automated rendering and extraction of tabular data in Portable Document Format (PDF), comprising: extracting, by one or more processors, metadata corresponding to tabular data in a text-based Portable Document Format (PDF), the metadata being associated with characters and borders of the tabular data; generating, by one or more processors, a graph structure corresponding to the tabular data of the text-based PDF based at least in part on the metadata; generating, by one or more processors, a vector representation of the graph structure; constructing, by one or more processors, a tree structure corresponding to the tabular data based at least in part on the vector representation; A method comprising:

2. 10. The method of claim 1, further comprising: assigning, by one or more processors, attributes to one or more leaves of the tree structure based at least in part on the tabular data of the text-based PDF.

3. Extracting the metadata corresponding to the tabular data of the text-based PDF includes: identifying, by one or more processors, character information corresponding to each character of the tabular data of the text-based PDF, the character information including a position of each character of the tabular data of the text-based PDF; identifying, by one or more processors, border information corresponding to each boundary line of the tabular data of the text-based PDF, the border information including a location of each boundary line of the tabular data of the text-based PDF; The method of claim 1 further comprising:

4. generating the graph structure corresponding to the tabular data of the text-based PDF, generating, by one or more processors, a node corresponding to each character of the tabular data of the text-based PDF based at least in part on character position information of the metadata; identifying, by one or more processors, a plurality of vertices associated with a respective set of nodes based at least in part on triangulation techniques, each generated node of the set of nodes corresponding to a character of the tabular data; determining, by one or more processors, a plurality of edges associated with the plurality of vertices, the plurality of edges representing adjacency relationships between the respective sets of nodes; The method of claim 1 further comprising:

5. Generating the vector representation of the graph structure comprises: generating, by one or more processors, a first set of feature vectors corresponding to a node of a set of nodes of the graph structure based at least in part on the metadata corresponding to the tabular data, the node corresponding to a character of the tabular data; generating, by one or more processors, a second set of feature vectors corresponding to an edge of the set of edges of the graph structure based at least in part on the metadata corresponding to the tabular data; encoding, by one or more processors, the set of nodes and the set of edges of the graph structure into a feature space using a machine learning algorithm that is an attention graph neural network; The method of claim 1 further comprising:

6. Constructing the tree structure corresponding to the tabular data based at least in part on the vector representation includes: selecting, by one or more processors, actions that constitute parent and child nodes of the tree structure, the actions corresponding to elements of the tabular data; determining, by one or more processors, branch assignments between the constructed parent and child nodes of the tree structure; The method of claim 1 further comprising:

7. A computer program product causing one or more processors to perform the method of any of claims 1 to 6.

8. A computer readable storage medium having stored thereon the computer program of claim 7.

9. one or more computer processors; one or more computer-readable storage media; program instructions stored on the computer-readable storage medium that cause at least one of the one or more processors to perform the method of any of claims 1 to 6; 1. A computer system comprising:

Images