In-Database Analysis Model Conversion for Real-Time Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data mining and predictive analysis methods require data to be extracted from databases for processing, which can be time-consuming and cumbersome, especially when dealing with vast volumes of data, and do not efficiently leverage the processing power of database servers for real-time predictive analytics.
Innovation Solution
Converting data models into in-database analysis models that can be executed within the database system, allowing predictive analysis to be performed directly on the database server without extracting data, using techniques such as converting data models into PMML format and then into SQLScript stored procedures for efficient execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is extracted from database for processing, then data mining and predictive analysis can be performed, but processing time and memory consumption increase
Solution Approach 1:
The patent merges the data storage function and data processing function into a single integrated system. By converting external data models into in-database analysis models (stored as SQL stored procedures), the system eliminates the separation between data repository and analysis engine, allowing predictive analysis to be performed directly on data while it resides in the database, thus eliminating data extraction time and improving processing speed.
Solution Approach 2:
The patent introduces a model conversion mechanism as an intermediary that transforms external data models (e.g., from R or Python) into database-compatible stored procedures. This intermediary layer enables seamless integration between external analytical tools and the database system, allowing models to be executed in-database without manual data extraction while maintaining compatibility with various programming languages and frameworks.
2Productivity
If data models are converted into in-database analysis models, then processing efficiency improves, but model conversion complexity increases
Solution Approach 1:
The patent implements a self-service model conversion system where the database management system automatically performs the conversion of data models into in-database analysis models. The conversion process is automated through programmatic interfaces that accept external models and generate corresponding SQL stored procedures without requiring manual intervention, thus reducing operational complexity despite the technical sophistication of the conversion process.
Solution Approach 2:
The patent creates a universal model conversion framework that can handle multiple types of external data models from different programming languages and frameworks (e.g., R, Python, Java) and convert them all into a common database-executable format. This multi-functional conversion system simplifies the overall process by providing a single entry point for model integration regardless of the source system.
Data Source
AI summary
Data sets are analyzed to discover trends and determine predictive data models. A data model is determined based on analyzing the data set with a specific algorithm from a set of different analytical algorithms. The data model is stored in a structural file format. The data model is converted into an instantiated object model with the use of a predefined object model. The instantiated object model is converted into an in-database analysis model to score new data within a database system. The scoring is based on the logic in the data model. The in-database analysis model is stored on a database server part of a database system to provide analytical functionality defined in the data model. The new data can be scored with the in-database analysis model. The new data is not extracted from the database system and the scoring is accomplished in-memory.