Memoizing ML Pre-processing Pipeline Lookup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Feature engineering and data pre-processing in machine learning can be performance bottlenecks, especially for real-time inferencing, as they require repeated computations that are not efficiently optimized.
Innovation Solution
A method is introduced to memoize data transformation patterns in machine learning pre-processing by creating a memo table of keys and values during the training phase, measuring hit rates and lookup times, and determining whether to use the memo table or bypass it based on thresholds, thereby optimizing the execution of machine learning pre-processing pipelines during inferencing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If memo table lookup is performed for all elements, then inferencing speed is improved, but system complexity increases
Solution Approach 1:
The patent pre-computes and stores pre-processing results in a memo table during the training phase. During inferencing, the system performs preliminary lookup in this pre-built table before executing the full pre-processing pipeline, thereby avoiding redundant computations and improving inferencing speed without adding significant complexity to the inferencing phase.
2Loss of time
If memo table is used, then processing time is reduced, but memory usage increases
Solution Approach 1:
The patent applies memoization selectively based on local characteristics of the data. The system analyzes the distribution of input values and identifies regions or patterns where memoization would be most beneficial. The memo table stores pre-computed results for specific input patterns or ranges, rather than attempting to cache all possible inputs, thereby reducing memory usage while still achieving significant processing time reductions for common cases.
3Reliability
If full pre-processing pipeline is executed for each element, then accuracy is maintained, but productivity decreases
Solution Approach 1:
The patent implements a two-stage processing approach where not all elements undergo the full pre-processing pipeline. Instead, the system performs a preliminary filter or lookup stage that handles common cases with simplified operations, and only executes the complete pre-processing pipeline for elements that require it. This partial action approach maintains accuracy for all elements while significantly improving processing throughput by avoiding redundant full pipeline executions for elements that can be handled by the simplified stage.
Data Source
AI summary
A method creates a table of keys and values. Each key is an element of an input array which is an input of a machine-learning pre-processing pipeline, and each value is an output of the pipeline. The method measures (1) a hit rate H to the memo table, (2) an average time Ttable to look up the table, (3) an average time Tpipeline to execute the pipeline, and (4) a threshold Telements on a number of elements of the input array. The method looks up the value in the table by using an element of the input array as a key when Tpipeline×H>Ttable and the number of elements in the input array is less than Telements. The method calls the pipeline in place of the lookup for all of the remaining elements in the input array when the value is not in the table.