Field Type Determination in Tabular Data Using Embedding Knowledge Bases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Tabular data in telecommunications systems often contains inconsistencies, making it challenging to determine field types accurately, especially in contexts where field type information is missing, unclear, or incorrect.
Innovation Solution
An apparatus and method that generate an embedding knowledge base from training tabular data to determine field types in new tabular data by using convolutional neural network encoders to process field entries and their context, maximizing positive context and minimizing negative context through log-likelihood processing, and applying cosine similarities for field type identification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional methods are used to determine field types in tabular data, then manual annotation is required, but this process is time-consuming and error-prone due to data inconsistencies
Solution Approach 1:
The system enables self-service by allowing the tabular data to automatically determine its own field types through vector representation and similarity comparison, eliminating the need for manual annotation while maintaining high accuracy through contextual analysis
Solution Approach 2:
The patent replaces the mechanical manual annotation process with an automated computational system that uses convolutional neural networks to generate vector representations and determine field types through algorithmic comparison against known field type profiles
2Reliability
If manual field type annotation is performed, then field type information can be obtained, but human errors and inconsistencies reduce reliability
Solution Approach 1:
The patent introduces vector representations as an intermediary between the raw tabular data and field type determination. This intermediary layer captures contextual information in a standardized format, enabling reliable automated comparison while managing system complexity through modular processing
3Productivity
If automated field type determination is implemented, then processing efficiency improves, but accuracy may suffer due to data inconsistencies
Solution Approach 1:
The system performs preliminary action by pre-processing the tabular data into vector representations that capture contextual information before field type determination. This preprocessing step enhances the quality of input data for automated analysis, improving accuracy while maintaining high processing efficiency
Solution Approach 2:
The patent incorporates feedback mechanisms where the system learns from the tabular data patterns and refines its field type determination process. The vector representation approach allows the system to capture contextual feedback from surrounding data points, improving accuracy in the presence of data inconsistencies
Data Source
AI summary
Various example embodiments for supporting determination of field types in tabular data are presented. Various example embodiments for supporting determination of field types in tabular data are configured to provide improvements in computer performance for supporting determination of field types in tabular data. Various example embodiments for supporting determination of field types in tabular data are configured to generate an embedding knowledge base based on training tabular data and to process new tabular data based on the embedding knowledge base in order to determine field types of fields included in the new tabular data.