AI Model Instance Selection in Heterogeneous Computing Platforms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The transition from x86 to ARM-based processors in Information Handling Systems (IHSs) presents challenges in management, customization, optimization, interaction, servicing, and configuration, particularly in deploying Artificial Intelligence (AI) models with varying complexity levels across heterogeneous computing platforms.
Innovation Solution
A heterogeneous computing platform is implemented, comprising multiple devices with firmware instructions that enable context or telemetry data-driven selection of AI model instances, where an orchestrator selects and instructs devices to execute appropriate AI models based on policies and performance metrics, without involving the host Operating System (OS).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple AI model instances with different complexity levels are deployed in a heterogeneous computing platform, then the system's adaptability and performance optimization are improved, but the device complexity and management difficulty increase
Solution Approach 1:
The patent segments the AI model deployment into multiple instances with different complexity levels (e.g., quantization levels, pruning ratios) that can be independently selected and deployed on heterogeneous devices. This allows the system to divide the AI model into variants suitable for different computational resources, resolving the contradiction between adaptability and complexity by organizing models into manageable segments.
Solution Approach 2:
The system dynamically selects and switches between different AI model instances based on real-time context or telemetry data from devices. The orchestrator can dynamically adjust which model instance is executed based on device performance, power constraints, and workload requirements, enabling adaptability without requiring all complex models to be simultaneously active.
2Productivity
If dynamic selection of AI model instances based on context or telemetry data is implemented, then the performance and resource utilization are optimized, but the management and configuration complexity increase
Solution Approach 1:
The patent implements feedback mechanisms where the orchestrator receives context or telemetry data from devices, evaluates performance metrics, and adjusts AI model instance selection accordingly. This closed-loop feedback system optimizes performance by continuously adapting to device conditions while managing complexity through automated decision-making based on predefined policies.
Solution Approach 2:
The system enables self-service deployment where the orchestrator automatically selects and deploys appropriate AI model instances based on device context and performance goals without requiring manual configuration. The self-service mechanism reduces management complexity by automating the selection process while maintaining high performance through data-driven decisions.
3Extent of automation
If firmware instructions are used to enable AI model execution without host OS involvement, then the system's automation and efficiency are improved, but the difficulty of detecting and measuring system state increases
Solution Approach 1:
The patent introduces firmware as an intermediary layer between the AI model execution and the host operating system. The firmware instructions enable AI models to be executed directly on hardware without full OS involvement, improving automation and efficiency. The intermediary firmware layer also provides structured interfaces for collecting context and telemetry data, making system state detection more manageable despite the reduced OS involvement.
Data Source
AI summary
Systems and methods for deploying instances of Artificial Intelligence (AI) models having different levels of complexity in a heterogenous computing platform are described. In an embodiment, an Information Handling System (IHS), may include a heterogeneous computing platform comprising a plurality of devices and a memory having a plurality of sets of firmware instructions that, upon execution by a respective device, enable the respective device to provide a corresponding service, and where at least one of the devices operates as an orchestrator configured to: select an instance of an AI model among a plurality of instances of the AI model based, at least in part, upon context or telemetry data received from at least a subset of the plurality of devices, where each of the plurality of instances of the AI model has a different level of complexity, and instruct a device to execute the selected instance of the AI model.