Model Score Recall System for Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning systems require significant computing resources and costs to make real-time predictions, with cloud scoring APIs incurring high fees for processing and storage, making near real-time predictions economically burdensome.
Innovation Solution
A system that compresses and caches scored datasets, allowing for near real-time matching and retrieval of scores, reducing the need for continuous computation and minimizing costs by using cached scores for similar requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If machine learning systems use cloud scoring APIs for real-time predictions, then prediction speed is improved, but computing costs increase significantly
Solution Approach 1:
The system pre-computes scores for all possible feature value combinations during an offline phase and stores them in a cache. When a prediction request arrives, the system retrieves the pre-computed score instead of performing real-time computation. This preliminary action eliminates the need for expensive cloud API calls during real-time operations, resolving the contradiction between fast predictions and high computing costs.
Solution Approach 2:
The system creates a local copy of the scoring model and pre-computed scores by ingesting training data and generating a scored dataset. This local copy enables the system to make predictions without continuously calling expensive cloud APIs, significantly reducing computing costs while maintaining prediction speed.
2Measurement precision
If machine learning systems process all scoring requests through computation, then prediction accuracy is maintained, but processing time increases
Solution Approach 1:
The system performs preliminary scoring computations for all possible feature combinations and stores the results. During real-time operation, it retrieves pre-computed scores instead of performing new computations, dramatically reducing processing time while maintaining accuracy by using scores generated from the same training model.
Solution Approach 2:
The system dynamically adapts its processing approach by first checking the cache for pre-computed scores before attempting new computations. This dynamic switching between cache retrieval and computation optimizes processing time while ensuring accuracy by falling back to the scoring model when necessary.
3Speed
If machine learning systems store and compress scored datasets, then retrieval speed for similar requests is improved, but data processing complexity increases
Solution Approach 1:
The system extracts and removes features that are not present in the live scoring environment from the training dataset before compression. This extraction step simplifies the data structure by keeping only the relevant features needed for real-time matching, reducing processing complexity while maintaining retrieval speed.
Solution Approach 2:
The system changes the parameter representation by converting feature data into a compressed format suitable for storage and rapid matching. This parameter transformation enables efficient retrieval while the system manages the complexity through automated compression and decompression processes.
Data Source
AI summary
The present disclosure is for a system and a method for processing scoring request in a machine learning system in a computationally efficient and low-cost manner to enable near real-time scoring of incoming scoring requests. Specifically, the present invention is a for a model score recall system and method that enables a system to recall model scores for input rows of cross features values that have already been scored a machine learning model without having to perform a search on very large datasets. As such, the present disclosure provides a system and a method for obtaining scores provided by a machine learning model without having to run the model on each new incoming scoring request, which saves computational resources and saves costs associated with performing a compute transaction.