Reuse Compute Items for ML Training Without Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Service providers face inefficiencies in utilizing computing resources as a significant percentage of computational power is diverted from real-time production environments to offline audit environments for ML model training, affecting latency and availability.
Innovation Solution
A framework that publishes compute items from real-time prediction pools to audit pools, utilizing a centralized graph to reuse calculated values and delta processing, reducing redundant computations and optimizing resource usage by separating model serving from training processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If computational resources are diverted from real-time production environments to offline audit environments for ML model training, then ML model training performance is improved, but real-time prediction latency and system availability deteriorate
Solution Approach 1:
The system segments computational work into two distinct parts: real-time prediction execution in the production environment and ML model training in the offline audit environment. This segmentation allows each environment to operate independently with appropriate resource allocation, preventing training computations from impacting real-time prediction latency while still enabling comprehensive model training using available compute items.
2Productivity
If computational resources are diverted from real-time production environments to offline audit environments for ML model training, then ML model training performance is improved, but computing resource efficiency deteriorates
Solution Approach 1:
The system performs preliminary computation of compute items in the real-time production environment before they are needed for training. By pre-computing these items during normal operations, the system eliminates the need for redundant computations during offline training, thereby improving resource efficiency while still enabling comprehensive ML model training.
Solution Approach 2:
The system creates copies of compute items from the production environment and transmits them to the audit environment for training purposes. This copying approach allows the training environment to work with identical data without requiring duplicate computation resources, reducing overall processing power consumption while maintaining training effectiveness.
3Adaptability or versatility
If multiple computing environments are used for adjudication and audit, then model training capability is improved, but device complexity increases
Solution Approach 1:
The system implements a centralized graph that serves multiple functions: it tracks compute items in real-time production, manages their transmission to offline audit environments, and coordinates resource sharing between the two environments. This universal graph structure reduces the complexity that would otherwise arise from managing separate systems for production and training.
Solution Approach 2:
The system introduces a centralized graph as an intermediary layer between the production and audit environments. This intermediary manages the complexity of coordinating multiple computing environments by providing a unified tracking and management mechanism, simplifying the overall system architecture while enabling comprehensive model training capabilities.
Data Source
AI summary
There are provided systems and methods for optimizing training data generation from real-time prediction systems for intelligent model training. A service provider, such as an electronic transaction processor for digital transactions, may utilize different computing environment and services that implement machine learning models and engines. The service provider may have a live adjudication environment where models use live data to adjudicate on requests by users, as well as an audit environment where models are trained and tested before deployment. Models may have directed graphs that designate the model dependencies on variables that are processed and values for those variables are used for an output. When variables are shared between models in the adjudication and audit environment, the values for the shared variables may be published to the audit computing environment for use without reloading and processing data, thereby reducing computational load from the audit environment.