AI Workload Classification for Secure Low-Latency Placement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The assignment of artificial intelligence (AI) workloads to computing devices or production environments can impact performance and user experience, necessitating optimal placement strategies that consider latency, completion time, and security considerations.
Innovation Solution
A workload placement service determines the placement of AI workloads, such as inferencing, training, and model adaptation, based on constraints like latency minimization, completion time minimization, and security, using a variant selection agent that selects between secured and public variants, and employs a workload placement model trained with telemetry data and causal variables to optimize resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If AI workloads are assigned to computing devices without security-based classification, then resource utilization and productivity are improved, but security and reliability deteriorate due to potential exposure of sensitive information
Solution Approach 1:
The patent segments AI workloads into different categories based on security sensitivity (public vs. secured variants). The workload placement service classifies workloads and routes them to appropriate computing devices - public workloads to general devices and secured workloads to devices with enhanced security measures. This segmentation allows optimal resource utilization for public workloads while maintaining security for sensitive operations.
Solution Approach 2:
The workload placement service acts as an intermediary between AI workload submissions and computing device execution. It evaluates workload security requirements, selects appropriate computing devices, and manages the assignment process. This intermediary layer ensures that security considerations are integrated into resource allocation without preventing overall productivity.
2Ease of operation
If AI workloads are assigned to remote computing devices, then access flexibility and ease of operation are improved, but latency and loss of time increase due to network communication delays
Solution Approach 1:
The workload placement service dynamically changes assignment parameters based on front-end environment characteristics. It evaluates device proximity, network conditions, and workload requirements to optimize the balance between access flexibility and latency. By adjusting placement decisions based on real-time parameters, the system maintains remote access benefits while minimizing communication delays.
3Productivity
If AI workloads are assigned to local computing devices, then speed and productivity are improved, but security and reliability worsen due to potential vulnerabilities in local environments
Solution Approach 1:
The patent applies local quality by providing different security configurations to different computing devices based on their assigned workload types. Local devices running public workloads receive standard configurations for maximum performance, while devices assigned secured workloads receive enhanced security configurations. This allows local processing speed benefits to be maintained for public workloads while security is enhanced where needed.
Data Source
AI summary
A system for managing of a security based inferencing workload placement includes a security a variant selection agent, production environments, and a front-end device that includes a security variant initial agent and a variant selection agent. The security variant initial agent is programmed to: obtain a request for an inferencing payload for an artificial intelligence (AI) model, wherein the request comprises a prompt for the AI model generated by the user, in response to the request: perform an initial classification analysis on the request to obtain a classification-eligible tag for the request, make a first determination, based on the classification-eligible tag, that the request is classification-eligible, based on the first determination, transmit the request to the variant selection agent of the front-end device for a variant selection operation.