AI Query Routing Using Feature Segmentation and Encrypted Offloading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently processing query requests between local and remote artificial intelligence engines while preserving privacy and managing latency, accuracy, and computational resources.
Innovation Solution
A query scheduler determines whether to process queries locally or remotely using a smaller or larger machine learning model, segregating sensitive and non-sensitive data, and employing homomorphic encryption to protect privacy, with the option to offload complex tasks to a remote inference server.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If queries are processed locally on edge servers, then latency is reduced and response time is improved, but processing power and model complexity are limited
Solution Approach 1:
The system segments the AI processing workload by dividing features into sensitive and non-sensitive categories. Non-sensitive features are processed locally on edge servers using smaller models to maintain low latency, while sensitive features are processed remotely using larger, more complex models. This segmentation allows the system to achieve both low latency for simple queries and high accuracy for complex queries requiring sensitive data.
2Measurement precision
If sensitive data is transmitted to remote servers, then processing accuracy is improved, but privacy protection is compromised and transmission time increases
Solution Approach 1:
The system extracts only the necessary non-sensitive features from the input data and transmits them to remote servers for processing. Sensitive features are excluded from transmission and processed locally or through secure mechanisms. This extraction approach maintains processing accuracy by including relevant non-sensitive information while protecting privacy by removing sensitive data from the transmission channel.
Solution Approach 2:
The system introduces an intermediary processing layer that handles sensitive data through secure mechanisms such as homomorphic encryption or trusted execution environments. This intermediary allows remote processing of sensitive features without directly exposing the raw sensitive data to untrusted remote servers, thus maintaining both accuracy and privacy protection.
3Adaptability or versatility
If all queries are processed remotely on powerful servers, then processing capability and accuracy are improved, but latency increases and resource consumption rises
Solution Approach 1:
The system applies partial action by processing only the necessary subset of features remotely rather than all features. Non-sensitive features are processed locally with smaller models, while only sensitive features requiring powerful remote processing are transmitted. This partial approach reduces transmission time and latency while still utilizing remote processing capability when needed, avoiding the excessive action of transmitting all data remotely.
4Measurement precision
If larger and more complex AI models are used, then processing accuracy is improved, but computational resources and power consumption increase
Solution Approach 1:
The system applies local quality by deploying different model complexities to different locations based on the query requirements. Edge servers use smaller, less power-consuming models for processing non-sensitive features, while remote servers use larger, more accurate models for sensitive features. This local differentiation optimizes power consumption by avoiding the deployment of large models everywhere, while maintaining accuracy where needed.
Data Source
AI summary
Provided are a computer program product, system, and method for processing a subset of a feature set to determine where to process a query. One of the following is performed: 1) in response to the determining to process the query at a first machine learning model, forwarding non-sensitive input data of the query to the first machine learning model to produce a first query result to return to an initiator of the query; and 2) in response to determining to process the query at a second machine learning model, forwarding sensitive input data and the non-sensitive input data to an encryption engine to encrypt to send to the second machine learning model; and receiving an encrypted second query result from the second machine learning model to decrypt to produce a second query result to return to the initiator of the query.