Machine-Learned Information Extraction Across Changing Web Templates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing information extraction systems are website-specific and require manual updates due to template variations across different websites, making them inefficient and time-consuming.
Innovation Solution
A scalable deep learning neural network architecture that processes templatic web pages without relying on HTML structure, capable of generalizing across multiple websites and adapting to changes in structure and composition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manually-created extraction rules (e.g., xPath rules) or machine-learned extraction functions customized for a specific website are used, then information extraction accuracy for that website is improved, but the system cannot work on other websites and requires updates every time the website structure changes
Solution Approach 1:
The patent applies universality by training a machine-learned model on a diverse corpus of web pages from multiple websites with different templates and structures. This enables a single model to perform information extraction across various websites without requiring website-specific customization, thus achieving both accuracy and cross-website applicability simultaneously
Solution Approach 2:
The patent changes the approach from static, manually-created extraction rules to dynamic, machine-learned models that automatically adapt to different website structures. The model learns patterns from training data and can generalize to new websites, effectively changing the parameter of extraction methodology from rigid rules to flexible learned representations
2Measurement precision
If manually-created extraction rules are used for each website, then extraction accuracy for that specific website is improved, but the time required to update and retrain the system increases
Solution Approach 1:
The patent implements self-service by using the system's own extracted information to automatically train and update the machine-learned model. The model continuously learns from new web page examples without requiring manual rule creation or extensive human intervention, thus maintaining accuracy while reducing update time
Solution Approach 2:
The patent applies preliminary action by pre-training the model on a large corpus of web pages before deployment. This preliminary training equips the model with general extraction capabilities that can be quickly adapted to new websites without requiring time-consuming manual rule creation for each new target
3Reliability
If website-specific extraction functions are used, then extraction reliability for a specific website is improved, but the scalability to accommodate new or updated templates from various sources is reduced
Solution Approach 1:
The patent changes the extraction approach from static website-specific functions to a dynamic machine-learned model that processes web pages as sequences of tokens. This parameter change enables the system to maintain reliability on known websites while scaling to new templates through automatic learning from training data, eliminating the need for manual adaptation to each new website structure
Data Source
AI summary
Techniques for automatic intelligent information extraction from an electronic document are disclosed. In one embodiment, a computerized method is disclosed comprising training a label prediction model to generate a set of label predictions, obtaining an electronic document, analyzing the electronic document and determining a set of features for each of a set of information items identified in the electronic document, obtaining model output from the label prediction model for each information item, the model output comprising, for a respective information item, a set of probabilities corresponding to a set of information classes, and generating an information extraction comprising a set of labels corresponding to the set of information items.