Semi-structured Data Mapping Using Historical Context
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Interpreting semi-structured data into structured data is challenging due to its non-conformity with formal data models, leading to ambiguity in identifying semantic elements and hierarchies.
Innovation Solution
A computing system processes semi-structured data by receiving and mapping values to attributes in a structured data collection using historical data, additional user data, and external system data, converting semi-structured data into natural language text, and embedding keys to resolve ambiguities and classify terms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If highly accurate rules with predefined patterns are used to interpret semi-structured data, then mapping accuracy to structured data collection is improved, but the complexity of interpreting non-conforming data increases
Solution Approach 1:
The system performs preliminary actions by obtaining historical data associated with attributes of the structured data collection before interpreting the semi-structured data. This historical data is used to pre-establish mapping patterns and relationships, so that when semi-structured data is received, the interpretation can leverage these pre-computed mappings to improve accuracy without increasing real-time interpretation complexity
Solution Approach 2:
The system introduces historical data as an intermediary between the semi-structured data and the structured data collection. This historical data acts as a mediator that contains pre-established mappings and patterns, allowing the system to translate semi-structured data more accurately by referencing this intermediate layer rather than directly interpreting against the structured collection
2Measurement precision
If multiple data sources including historical data and user data are used to map keys to attributes, then classification accuracy is improved, but the time required for data processing increases
Solution Approach 1:
The system obtains historical data and user data in advance before the actual mapping operation. This preliminary data gathering allows the system to pre-process and organize reference information, so that during the actual mapping of keys to attributes, the system can quickly query pre-organized data rather than processing everything from scratch, thereby reducing real-time processing time while maintaining high classification accuracy
Solution Approach 2:
The system dynamically adjusts which data sources are used for mapping based on the specific context and requirements. Rather than always using all available data sources (historical data, user data, external system data), the system can selectively apply the most relevant data sources for each mapping operation, optimizing the balance between classification accuracy and processing time
Data Source
AI summary
Techniques for processing semi-structured data include receiving semi-structured data, a value of the semi-structured data being associated with a key; identifying a structured data collection, the structured data collection being associated with a category and having a plurality of attributes; obtaining at least one of historical data associated with the plurality of attributes or additional data associated with a user of the computing system; mapping the key to an attribute of the plurality of attributes based on at least one of the historical data or the additional data; and storing the value in a cell of a data record of the structured data collection, the cell being associated with the attribute that was mapped to the key.