Document Parsing via Dynamic Programming Global Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current document parsing and interpretation methods are inefficient and costly, particularly for long or multi-page documents, as they rely on independent item matching and segmentation, which can lead to ambiguous local decisions and are impractical for handling complex interactions between items.
Innovation Solution
A method using dynamic programming to optimize a global objective function for parsing documents by generating multiple overlapping candidates and selecting a subset that maximizes the objective, avoiding premature local decisions and accounting for interactions between items.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If independent item matching is used to parse documents, then processing speed is improved, but accuracy deteriorates due to ambiguous local decisions and inability to handle complex interactions between items
Solution Approach 1:
The patent segments the document parsing problem into two distinct phases: (1) generating multiple candidate items with potential overlaps, and (2) selecting a non-overlapping subset that optimizes a global objective function. This segmentation allows independent candidate generation (maintaining speed) followed by global optimization (improving accuracy), resolving the contradiction between processing speed and parsing accuracy.
Solution Approach 2:
The patent introduces dynamic programming to selectively resolve overlaps between candidates based on their contribution to the global objective function. Rather than statically fixing item boundaries or dynamically adjusting them in a greedy manner, the system dynamically determines which candidates to include or exclude based on global optimization criteria, thereby improving accuracy while maintaining computational efficiency.
2Measurement precision
If global optimization is used to account for interactions between items, then accuracy is improved, but computational complexity increases making it infeasible for long or multi-page documents
Solution Approach 1:
The patent segments the global optimization problem by first generating candidates independently and then applying dynamic programming to select the optimal non-overlapping subset. This two-stage segmentation transforms an intractable global optimization problem into manageable sub-problems, reducing computational complexity while preserving global optimality for long and multi-page documents.
Solution Approach 2:
The patent generates multiple overlapping candidates (excessive action) and then selects a non-overlapping subset through dynamic programming. This approach of generating more candidates than ultimately needed allows the system to explore multiple possibilities and select the optimal combination, achieving high accuracy without prohibitive computational cost by leveraging the efficiency of dynamic programming.
3Measurement precision
If manual parsing is performed to ensure accuracy, then parsing precision is improved, but productivity deteriorates due to significant time and cost requirements
Solution Approach 1:
The patent enables the system to automatically perform global optimization without manual intervention. By using dynamic programming to select the optimal non-overlapping subset of candidates, the system self-services the complex task of resolving item interactions and boundary ambiguities, achieving parsing precision comparable to manual methods while maintaining high processing throughput.
Solution Approach 2:
The patent replaces manual mechanical parsing with an automated computational system using dynamic programming. This substitution eliminates the need for human operators to manually resolve item overlaps and interactions, thereby maintaining high parsing precision while dramatically improving productivity and reducing processing time and costs.
Data Source
AI summary
A method is provided for parsing a document having a plurality of lines on which items are listed spanning one or more lines. It includes: obtaining a plurality of candidates, representing hypothetical items within the document, each candidate spanning one or more lines and having a local cost representing a confidence in a quality of the candidate compared to a model; determining labeling costs for intervals of the document defined between pairs of lines, each interval containing candidates therein, each labeling cost reflecting a configuration of the candidates within the interval; identifying a best labeling for each interval based on the labeling costs determined for that interval, the best labeling corresponding to one of the configurations of the candidates within the interval; defining a global objective function; and selecting a subset of the candidates such that the global objective function is optimized, based on the identified best labelings.