Template-Based Machine Learning Execution Across Heterogeneous Hardware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models are difficult and time-consuming to schedule and execute across heterogeneous computing hardware infrastructure due to the complexity of software frameworks, middleware ecosystems, and the need for highly skilled developers, leading to errors and inconsistent governance processes.
Innovation Solution
A centralized priority-based scheduling system and standardized process for executing machine learning models on various computing hardware infrastructure, using a management service to compile applications once and provide a single interface for scheduling and execution, with features like application templates and priority-based resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If machine learning models are executed across heterogeneous computing hardware infrastructure using multiple software frameworks and middleware, then the models can run on various platforms, but the execution time increases and errors occur due to complexity
Solution Approach 1:
The patent introduces a standardized interface layer that acts as an intermediary between machine learning models and heterogeneous computing hardware infrastructure. This interface layer translates model execution requests into platform-specific operations, enabling models to run on various hardware platforms without requiring multiple software frameworks. The intermediary layer abstracts the complexity of different hardware architectures, allowing single-model deployment across diverse infrastructure while maintaining efficient execution.
2Adaptability or versatility
If multiple software frameworks and middleware are used to support different machine learning models, then the models can be executed on various hardware, but the system complexity increases and requires highly skilled developers
Solution Approach 1:
The patent implements a universal standardized interface that can handle multiple machine learning models across different hardware platforms through a single software architecture. This universal interface replaces the need for multiple specialized software frameworks, allowing the same interface to accommodate various model types and hardware configurations. The multi-functional interface reduces software stack complexity while maintaining the ability to execute diverse machine learning models on heterogeneous infrastructure.
3Adaptability or versatility
If various software technologies and middleware are integrated to execute machine learning models, then the models can run on different hardware platforms, but governance processes become inconsistent and errors increase
Solution Approach 1:
The patent enforces homogeneous governance processes through a standardized interface that applies uniform validation, scheduling, and execution rules across all hardware platforms. This standardized approach ensures consistent governance policies are maintained regardless of the underlying hardware or model type. The homogeneous interface layer prevents governance inconsistencies by providing a single point of control that enforces the same rules and procedures for all machine learning model executions across diverse infrastructure.
Data Source
AI summary
Disclosed are various embodiments for accelerating the execution of machine learning model-based application on various computing hardware infrastructure. In non-limiting example, a system comprises a computing device that is configured to initiate a run-time execution of an application that includes a machine learning model. The computing device is further configured to determine a plurality of eligible application templates and select an application template among the plurality of eligible applications templates. The application can be executed in a run-time environment specified by the application template.


