Automatic Wrapper Induction Using Target Strings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data mining, constructing wrappers for information extraction from various domains is tedious and error-prone, requiring extensive manual effort and labeled training data, especially when dealing with partially- or un-structured documents.
Innovation Solution
An automatic wrapper induction system that bootstraps wrappers from a minimal amount of training data by using a search engine to find target strings across domains, iteratively learning and applying wrappers to reduce the need for manual labeling, assuming domain overlap and using uniform data like UPCs or product names.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If wrappers are hand-coded for each information source, then extraction accuracy is improved, but the complexity and time required for wrapper construction increases significantly
Solution Approach 1:
The system enables automatic wrapper induction where the wrapper construction process serves itself by using extracted target strings as training data for subsequent domains, eliminating the need for manual wrapper coding for each domain while maintaining extraction accuracy through iterative learning
Solution Approach 2:
The wrapper induction system is designed to be universally applicable across multiple domains and information sources. A single induction process can learn wrappers for different domains by leveraging target strings extracted from any domain, making the system multi-functional rather than domain-specific
2Measurement precision
If labeled training data is provided for every domain, then wrapper learning accuracy is improved, but the time and effort required for data preparation increases
Solution Approach 1:
The system performs preliminary extraction of target strings from any domain before actual wrapper induction. These pre-extracted target strings serve as ready-to-use training data, eliminating the need for time-consuming manual labeling while maintaining learning accuracy
Solution Approach 2:
The system discards the need for manual labeled training data by recovering and reusing target strings automatically extracted from domain content. These recovered target strings serve as effective training data, replacing manual labeling efforts while preserving learning effectiveness
3Adaptability or versatility
If manual wrapper construction is used, then adaptability to specific domains is improved, but productivity and scalability decrease
Solution Approach 1:
The automatic wrapper induction system serves itself by using target strings extracted from any domain to train wrappers for that domain and other domains. This self-service mechanism enables rapid adaptation to new domains without manual intervention, simultaneously improving productivity and maintaining domain-specific adaptability
Solution Approach 2:
The system achieves universal applicability across domains while maintaining domain-specific adaptability. The same induction process can adapt to any domain by using its target strings, enabling scalable deployment without sacrificing the ability to handle domain-specific variations
Data Source
AI summary
Wrappers are induced for multiple domains where, for a given target string having relatively universal distribution across domains of interest, a first wrapper may be defined and trained for a particular domain. Target strings extracted from that domain may be used to search for documents in other domains. New wrappers may be learned for other domains also containing the target strings. Further, a first wrapper may be learned for a given domain using a limited amount of training data from that single domain. The first wrapper is then applied to all pages in the domain to extract the relevant information. A few of the new words extracted are then searched against the document collection to obtain a list of domains that contain the extracted words. The updated information may be used as training data to learn new wrappers on those domains.