Universal Pattern Extraction for Multi-Format Document Phrases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing information extraction methods are inconvenient for extracting phrases from documents with various formats, as they require selecting sample documents for each format, making it labor-intensive and inefficient.
Innovation Solution
An information extraction system that determines patterns in each document to extract phrases, regardless of format, using an input word list, and selects candidate phrases based on predetermined conditions and scores, improving reliability and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sample documents are selected for each document format to extract phrases, then extraction accuracy for specific formats is improved, but operation complexity and time consumption increase significantly
Solution Approach 1:
The patent creates a single unified pattern that can be applied across multiple document formats (tables, lists, hierarchies) to extract phrases. Instead of creating format-specific sample documents, the system uses one universal pattern with format-specific extraction rules, eliminating the need for repeated manual sample selection while maintaining extraction accuracy across diverse formats.
Solution Approach 2:
The patent segments the phrase extraction process into two independent components: (1) a universal pattern creation phase that identifies common phrase structures, and (2) format-specific extraction phase that applies the pattern to different document formats using predetermined extraction rules. This segmentation allows the system to handle various formats without requiring separate sample documents for each.
2Reliability
If multiple sample documents are selected for each format to ensure comprehensive coverage, then extraction reliability is improved, but device complexity and operational burden increase
Solution Approach 1:
The system uses a single universal pattern that serves all document formats, reducing system complexity while maintaining reliability through format-specific extraction rules. The unified pattern is combined with predetermined extraction rules that adapt to different formats (tables, lists, hierarchies), achieving comprehensive coverage without requiring multiple format-specific patterns or complex sample selection mechanisms.
Solution Approach 2:
The patent introduces predetermined extraction rules as an intermediary layer between the universal pattern and the diverse document formats. These extraction rules act as mediators that translate the universal pattern into format-specific extraction logic, simplifying the overall system architecture while ensuring reliable extraction across different formats without requiring direct customization for each format.
3Reliability
If manual classification and registration of phrases is performed to create high-reliability dictionaries, then dictionary quality is improved, but productivity decreases due to manual labor requirements
Solution Approach 1:
The system enables automatic dictionary creation by having the information processing device autonomously specify patterns, extract phrases, and register them in the dictionary without requiring manual classification and registration work. The automated pattern specification and phrase extraction processes maintain high dictionary quality while dramatically improving creation efficiency by eliminating manual labor.
Solution Approach 2:
The patent replaces the manual mechanical process of phrase classification and dictionary registration with an automated information processing system. The system uses algorithmic pattern matching and automated extraction to create dictionaries, substituting human manual work with computational processes that maintain or improve dictionary quality while significantly increasing productivity.
4Ease of operation
If document formats are standardized to a single format for extraction, then extraction process simplicity is improved, but adaptability to real-world diverse documents deteriorates
Solution Approach 1:
The patent creates a universal pattern that can be applied across multiple document formats (tables, lists, hierarchies) to extract phrases. Instead of requiring documents to be standardized to a single format, the system uses one universal pattern with format-specific extraction rules, maintaining extraction simplicity while achieving broad format compatibility.
Solution Approach 2:
The patent applies local quality by having the extraction system adapt its behavior based on the local characteristics of each document format. The universal pattern is combined with format-specific extraction rules that adjust to local format features (table structures, list formats, hierarchy levels), allowing simple unified pattern processing while maintaining adaptability to diverse document formats.
Data Source
AI summary
According to the present invention, phrases of the same kind can be extracted from a plurality of documents having various formats. A storage device stores a plurality of documents that have various formats. A pattern candidate creating unit receives a list of input words that are selected as samples among phrases that are to be included in a dictionary. The pattern candidate creating unit selects one document, determines forward and backward character strings of input words in the selected document as candidates of patterns, and stores the forward and backward character strings as a pattern candidate. The pattern candidate creating unit executes the above processes for each of the documents. A phrase candidate creating unit extracts phrases interposed between patterns included in the pattern candidate as candidates of phrases to be output, and stores the extracted phrases as a phrase candidate. A phrase selecting unit outputs a candidate of a phrase satisfying a predetermined condition among candidates of phrases included in the phrase candidate as an output word to an output device.