Tokenized Database Predictive Model for Efficient Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning methods are limited by data storage structures, leading to incomplete training datasets, opaque model architectures, and resource-intensive training processes, making them inefficient for adapting to new problems and deploying on low-performance devices.
Innovation Solution
A computer-implemented method that tokenizes input and output data sets, stores them in a non-redundant token list, and applies database operations to identify correlated input-output tuples, creating a predictive model that can generate new outputs based on strongly correlated tokens, allowing for generic and efficient predictive modeling across different problems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional data storage structures are used for machine learning, then the system can process data, but the training dataset becomes incomplete and the model becomes opaque
Solution Approach 1:
The patent segments data into discrete tokens and organizes them in a normalized database structure (6th normal form), separating tokens from their occurrences in data records. This segmentation enables complete data utilization while maintaining model transparency through the token list and IO tuple structures.
Solution Approach 2:
The patent introduces a new dimensional organization of data by creating a token list that indexes all unique tokens across the dataset, and IO tuples that capture input-output relationships. This dimensional transformation allows the system to work with complete datasets rather than sampled subsets, improving both reliability and transparency.
2Adaptability or versatility
If problem-specific algorithms and data structures are used, then the model can be trained, but adaptation to new problems becomes complex and time-consuming
Solution Approach 1:
The patent creates a universal token-based data structure and database-oriented algorithm that can handle diverse machine learning problems. The normalized database structure and set operation framework provide a problem-agnostic foundation that reduces adaptation time while maintaining effectiveness across different applications.
Solution Approach 2:
The system changes the fundamental parameters of how data is represented and processed by using tokenization and database set operations instead of traditional neural network architectures. This parameter change enables faster adaptation to new problems while maintaining model effectiveness.
3Productivity
If traditional machine learning training processes are used, then predictive models can be generated, but resource consumption in terms of memory and CPU load becomes considerable
Solution Approach 1:
The patent replaces traditional CPU-intensive machine learning training mechanisms with database-oriented set operations. By leveraging the database engine's optimized data processing capabilities instead of custom neural network training code, the system achieves comparable or superior training efficiency with reduced resource consumption.
Solution Approach 2:
The system creates simplified copies of data in tokenized form and uses set operations on these copies to derive predictive models. This copying approach avoids the need to process the entire original dataset repeatedly during training, reducing memory and CPU requirements while maintaining model quality.
Data Source
Figure 1A
Figure 1B~2
Figure 3
AI summary
The invention relates to a computer-implemented method for generating a predictive model by machine learning. The method comprises: - providing (1302) a plurality of input and output data records; - tokenizing (1304) the input data records; - tokenizing (1306) the output data records; - storing (1308) all input tokens and output tokens, each associated with one or more data record pointers, in a non-redundant token list; - applying (1312) database-internal set operations to the pointers in the token list to identify input-token-output token tuples—hereinafter referred to as IO-tuples—that contain at least one correlated input token and at least one correlated output token; and - storing (1314) the identified IO-tuples as part of a predictive model.