Machine-Learning Cacheable Query Prediction for Lower Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Heterogeneous computing systems face challenges in efficiently processing data queries due to computational expenses, latency, and network congestion, leading to reduced throughput and increased latency in data retrieval.
Innovation Solution
Implementing a machine learning model to predict cacheable queries based on current and previous queries, using identifiers and key-value pairs to generate predicted requests, cache associated data, and remove cache entries after a threshold time, thereby reducing latency and bandwidth consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is retrieved from remote computing systems, then data availability is improved, but latency and network congestion increase
Solution Approach 1:
The system performs preliminary actions by predicting future queries using machine learning models and pre-caching the corresponding data before the actual queries are executed. This allows data to be available locally in the cache store when needed, eliminating retrieval latency and network congestion while ensuring data availability.
2Reliability
If multiple synchronous requests are performed for data retrieval, then data completeness is improved, but system throughput decreases
Solution Approach 1:
The system predicts multiple future queries in advance and pre-caches their corresponding data simultaneously, ensuring data completeness is achieved through comprehensive pre-retrieval. This eliminates the need for multiple synchronous requests during actual query execution, thereby maintaining high system throughput.
3Speed
If cache size is increased to reduce latency, then data retrieval speed is improved, but memory consumption increases
Solution Approach 1:
The system dynamically changes the parameter of cache content by using machine learning models to predict which specific data will be queried and pre-caching only that data. This selective pre-caching approach optimizes cache utilization, improving data retrieval speed while minimizing memory consumption by storing only the most relevant predicted data.
4Loss of time
If predictive modeling is implemented to reduce cache misses, then query response time is improved, but computational overhead increases
Solution Approach 1:
The system applies partial predictive action by using machine learning models to predict only the most likely future queries and pre-caching data for those specific predictions. This selective approach reduces query response time for predicted queries while minimizing computational overhead by avoiding exhaustive prediction and caching of all possible queries.
Data Source
AI summary
Predicting cacheable queries is provided. For example, a system can one or more processors, coupled with memory, to generate a plurality of predicted requests based on one or more requests received from a client device. The system can compute a metric associated with retrieving data for a respective predicted request from a database. The system can select a subset of predicted requests from the plurality of predicted requests based on the metric corresponding to each predicted request and a threshold metric. The system can generate labels classifying the subset of predicted requests. The system can store, using the labels, data from the database for the subset of predicted requests in a cache store. The system can transmit, responsive to receiving a client request matching a predicted request of the subset of predicted requests, corresponding data from the cache store.