Feature Store RPC Architecture for ML Prediction Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing server-client architectures for machine learning model training and prediction are resource-intensive and inefficient, especially when deploying hundreds or millions of models, due to duplicative processing tasks.
Innovation Solution
The implementation of a system that uses remote procedure calls (RPCs) and the WebSocket protocol to enable push-based sharing of data and engineered features between subsystems associated with training and supporting machine learning models, thereby reducing processing resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional server-client architectures are used for machine learning model training and prediction, then model prediction functionality can be provided, but processing resource utilization becomes excessive and system efficiency decreases
Solution Approach 1:
The patent segments the machine learning system into specialized components: edge devices perform local inference, cloud servers handle model training and updates, and a feature store manages data. This segmentation allows each component to operate independently and efficiently, reducing overall processing resource utilization while maintaining high productivity.
Solution Approach 2:
The patent introduces a new architectural dimension by implementing a feature store that decouples data management from computation. This dimensional change allows the system to efficiently manage features across multiple models without duplicative processing, thereby improving system efficiency while reducing processing resource demands.
2Adaptability or versatility
If multiple machine learning models are deployed simultaneously, then model prediction capabilities are enhanced, but duplicative processing tasks increase resource consumption
Solution Approach 1:
The patent merges the feature storage and management functionality into a centralized feature store that serves all machine learning models. This consolidation eliminates duplicative feature processing across multiple models, reducing energy loss while maintaining enhanced model prediction capabilities.
Solution Approach 2:
The feature store implements a universal interface that can serve multiple different machine learning models with a single data infrastructure. This multi-functionality allows the system to support diverse model prediction capabilities without requiring separate processing pipelines for each model, thereby eliminating duplicative processing.
3Speed
If feature data is accessed through traditional request-response protocols, then data can be retrieved, but latency increases and real-time prediction performance deteriorates
Solution Approach 1:
The patent implements preliminary action by pre-computing and storing feature data in the feature store before it is needed for prediction. Features are prepared and cached in advance, allowing edge devices to retrieve pre-processed data quickly during inference, thereby reducing prediction latency while maintaining high data access speed.
Data Source
AI summary
In some implementations, a device may receive a request for a machine learning prediction. The device may access a feature store that stores data associated with a plurality of possible machine learning model features, wherein accessing the feature store comprises performing a remote procedure call (RPC) to generate a new data object at the feature store, the new data object having the device subscribed as a client. The device may receive, from the feature store in connection with the device being subscribed to the new data object as the client, a set of values associated with a composite feature. The device may execute a machine learning model using the set of values associated with the composite feature. The device may output a machine learning prediction based on a result of executing the machine learning model using the set of values associated with the composite feature.