Web List to Relational Table Extraction via Dynamic Field Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Converting lists into relational database tables is challenging due to unclear delineation of columns or cells, missing information, and inconsistent delimiter usage, making it difficult to extract structured data effectively.
Innovation Solution
A computer-implemented method that determines the number of fields for each item in a list, merges or inserts null fields to align with the number of columns, and writes information into a relational database table, using field quality scores and cost functions to ensure accurate data alignment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If lists are converted into relational database tables using conventional methods, then structured data can be extracted, but the process fails when delimiters are missing or inconsistently applied
Solution Approach 1:
The patent introduces an intermediary alignment process between list items and database columns. Instead of directly mapping delimiters to columns, the system uses field quality scores and cost functions as intermediaries to evaluate and align fields with appropriate columns, resolving the contradiction when delimiters are missing or inconsistent
Solution Approach 2:
The patent changes the parameter of field evaluation from simple delimiter detection to a composite scoring system involving field quality scores and cost functions. This parameter change enables accurate data extraction even when traditional delimiter-based methods fail
2Stability of the object's composition
If fields are strictly aligned with columns using fixed delimiters, then data structure is maintained, but missing information cannot be handled
Solution Approach 1:
The patent makes the field-column alignment dynamic rather than static. The alignment process adapts to each list item by calculating field quality scores and costs individually, allowing the system to handle varying numbers of fields per item while maintaining overall structural stability through null field insertion
Solution Approach 2:
The patent prepares for missing information in advance by introducing null fields as a cushioning mechanism. When a list item has fewer fields than columns, null fields are inserted to maintain structural stability without losing information about what data is present or absent
3Ease of manufacture
If delimiters are used to separate fields, then parsing is simplified, but delimiter abuse causes misalignment
Solution Approach 1:
The patent introduces field quality scores and cost functions as intermediary evaluation layers between delimiters and final field identification. This intermediary system filters out delimiter abuse by evaluating the actual semantic quality of potential fields rather than relying solely on delimiter positions
Solution Approach 2:
The patent implements a feedback mechanism where field quality scores and alignment costs are calculated and used to adjust the field identification process. If delimiter-based parsing produces misaligned fields, the feedback from quality scores identifies and corrects these errors by selecting alternative field boundaries
Data Source
AI summary
List information can be extracted into database tables. A number of fields are independently determined for items in list. A number of database table columns are determined from most common number of list item fields. New fields are determined for items with more fields than database columns. Null fields are inserted into items with fewer fields than database columns. Information from items having the same number of fields as database columns is written to database table rows. Information from each field is written to a corresponding database table column. Streaks of poorly matching cells in a database table row are determined. Streak cells are merged and new cells are determined. Null cells are inserted if number of new cells is less than number of cells in the streak. Information from the new cells is written to the table row and columns that define the streak.