Unified Scoring Engine Workflow for Data Analytics Model Deployment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data analytics model deployment systems, such as PMML-based systems, fail to capture all data pre-processing functions used for building the model, requiring third-party platforms and manual documentation for data conversion, which is inefficient and incomplete.
Innovation Solution
A method and system that record and execute data pre-processing stages, including ETL functions and machine learning algorithms, to generate a scoring engine workflow that enables seamless deployment and pre-processing of production data in the target environment, capturing all necessary transformations and algorithms for model compatibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If PMML-based system is used for deploying data analytics model, then model porting and deployment is enabled, but data pre-processing functions are not captured completely and third party platform is required
Solution Approach 1:
The patent merges the data pre-processing functions and the predictive model into a single unified artifact (the scoring engine workflow). This combination eliminates the need for separate third-party platforms to execute pre-processing functions, as all necessary transformations and the model itself are embedded within the deployed scoring engine package.
Solution Approach 2:
The patent performs preliminary action by capturing and embedding all data pre-processing functions during the model building phase. The scoring engine workflow is generated in advance with complete pre-processing instructions, eliminating the need for separate pre-processing execution during deployment.
2Ease of operation
If manual documentation is used for data pre-processing functions, then data conversion can be performed, but the process is inefficient and incomplete
Solution Approach 1:
The patent creates an automated copy of the data pre-processing functions from the model building environment directly into the scoring engine workflow. This automated copying process captures all pre-processing functions with complete accuracy, eliminating manual documentation errors and inefficiencies.
Solution Approach 2:
The patent replaces the manual mechanical process of documenting and converting data pre-processing functions with an automated system. The workflow generation module automatically generates the scoring engine workflow with embedded pre-processing functions, substituting manual IT/Software engineer work with automated computational processes.
3Device complexity
If data pre-processing functions are not captured, then model deployment is simpler, but production data compatibility cannot be ensured
Solution Approach 1:
The patent performs preliminary action by capturing all data pre-processing functions during the model building phase. The scoring engine workflow is generated in advance with complete pre-processing instructions embedded, ensuring that production data compatibility is guaranteed before deployment occurs.
Data Source
AI summary
The present disclosure relates to system(s) and method(s) for deploying a data analytics model in a target environment. The system records a set of data pre-processing stages, associated with the data analytics model. The set of data pre-processing stages may comprise receiving raw data, executing a set of ETL functions on the raw data, and executing a set of algorithms on the raw data. Further, the system generates the data analytics model based on the set of algorithms. Furthermore, the system generates a scoring engine workflow, associated with the data analytics model, based on the set of data pre-processing stages. The scoring engine workflow comprises one or more ETL functions and one or more algorithms. Further, the system deploys the data analytics model and the scoring engine workflow in the target environment. The scoring engine workflow enables pre-processing of production data in the target environment.