Bidirectional Encoder Models for Structured Data Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing predictive data analysis systems face challenges in efficiently and reliably processing structured data, particularly in learning cross-entry relationships and training models without labeled training data, which limits their efficiency and accuracy.
Innovation Solution
The use of per-field tokenized values and bidirectional encoder deep learning models, such as BERT, to generate encoded representations of structured data fields, enabling the learning of cross-entry relationships and model training without labeled data through masked tokenization and optimization techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional predictive data analysis systems process structured data, then basic predictions can be made, but the systems cannot efficiently learn cross-entry relationships and require extensive labeled training data
Solution Approach 1:
The patent segments structured data into discrete tokenized representations at multiple levels (field-level tokens, entry-level tokens, relationship tokens). This segmentation enables the model to process and learn cross-entry relationships by breaking down complex data structures into manageable units that can be independently encoded and recombined, reducing the need for extensive labeled training data while improving model accuracy.
Solution Approach 2:
The patent introduces a new dimensional representation by encoding data at multiple hierarchical levels (field dimension, entry dimension, relationship dimension). This multi-dimensional tokenization approach transforms traditional flat structured data into a layered representation that captures cross-entry relationships, allowing the model to learn patterns without requiring proportionally more labeled data.
2Reliability
If more labeled training data is used to improve model accuracy, then prediction reliability increases, but training time and computational resources increase
Solution Approach 1:
The patent performs preliminary tokenization and encoding of structured data fields into standardized representations before the main training process. By pre-processing data into field-level tokens, entry-level tokens, and relationship tokens, the system prepares the data in a format that accelerates subsequent training, reducing the overall training time while maintaining prediction accuracy.
Solution Approach 2:
The patent changes the parameter representation of structured data by introducing tokenized embeddings for fields, entries, and relationships. This parameter transformation converts traditional structured data formats into a neural network-friendly representation that enables faster convergence during training, reducing training time while preserving or improving prediction accuracy.
3Reliability
If detailed tokenized representations are generated for each structured data field, then model accuracy improves, but storage requirements increase
Solution Approach 1:
The patent creates universal tokenized representations that serve multiple functions: field-level tokens encode data field characteristics, entry-level tokens encode record-level information, and relationship tokens encode cross-entry relationships. These multi-functional tokens reduce storage requirements by eliminating redundant data while maintaining the detailed representations needed for high model accuracy.
Solution Approach 2:
The patent uses tokenized copies of structured data fields instead of storing the original detailed data structures. By creating compressed token representations that capture essential information, the system reduces storage requirements while preserving the detailed representations needed for accurate predictions. The tokens act as efficient copies that maintain information density with reduced space requirements.
Data Source
AI summary
There is a need for more effective and efficient predictive data analysis based at least in part on structured data. This need can be addressed by, for example, solutions for performing predictive data analysis using bidirectional encoder deep learning models that are configured to process structured data attributes. In one example, a method includes identifying a group of training structured data fields; generating a group of per-field tokenized values for each training structured data field; generating a bidirectional encoder deep learning model based at least in part on each group of per-field tokenized values for a training structured data field; and performing one or more prediction-based actions based at least in part on the trained bidirectional encoder deep learning model.