Deep Learning Resource Management via Out-of-Band Control

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Deep learning infrastructure with integrated AI capabilities for training and inferencing resources faces challenges in being transparent, powerful, power-efficient, and flexible across various scenarios, often resulting in inefficient resource utilization and high costs due to fixed configurations.

Innovation Solution

A platform with out-of-band (OOB) management logic for training and inference modules, using separate communication links and switches to manage and configure resources dynamically, allowing for software-defined, transparent, and flexible deep learning infrastructure deployment across edge, IoT, cloud, and data center scenarios.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If fixed configuration deep learning infrastructure is used, then hardware simplicity is maintained, but resource utilization efficiency deteriorates and power consumption increases

Engineering Contradiction:

Improveresource utilization efficiencyVSAvoidinfrastructure configuration complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent implements dynamic resource allocation by introducing a resource manager that can dynamically assign and reconfigure training and inference resources based on real-time workload demands. The system transitions from fixed hardware configurations to dynamic software-defined resource management, allowing resources to be flexibly allocated across multiple tasks and applications, thereby improving resource utilization efficiency while adapting to changing computational needs.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent creates a universal deep learning infrastructure where a single pool of resources can serve multiple functions - both training and inference operations can share the same underlying hardware resources. The resource manager enables these resources to be dynamically allocated to different tasks based on demand, making the infrastructure multi-functional and adaptable to various deep learning workloads without requiring separate dedicated hardware for each function.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Reliability

If dedicated training and inference resources are allocated, then task performance is ensured, but power consumption and costs increase

Engineering Contradiction:

Improvetask performance guaranteeVSAvoidpower consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent merges previously separate training and inference resources into a unified shared resource pool. The resource manager intelligently allocates these combined resources to training or inference tasks based on real-time priorities and workload characteristics. This consolidation eliminates the need for dedicated hardware for each function, reducing overall power consumption while maintaining task performance through dynamic resource assignment and priority-based scheduling.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system dynamically changes resource allocation parameters based on workload demands and task priorities. The resource manager adjusts the amount and type of resources allocated to training versus inference operations in real-time, optimizing the balance between performance guarantees and energy consumption. This parameter-based dynamic allocation allows the system to adapt resource usage to actual needs rather than maintaining fixed dedicated allocations.

Inventive Principle:

Principle #35Parameter changes

3Adaptability or versatility

If software-defined flexible infrastructure is implemented, then adaptability across scenarios is improved, but system complexity and management overhead increase

Engineering Contradiction:

Improvedeployment scenario flexibilityVSAvoidmanagement system complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces a resource manager as an intermediary layer between the physical hardware resources and the virtualized resource pools. This intermediary handles the complexity of resource allocation, configuration, and management, presenting a simplified interface to users and applications. The resource manager abstracts the underlying system complexity while enabling flexible software-defined resource allocation across diverse deployment scenarios including edge, cloud, and data center environments.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11494633B2Techniques to manage training or trained models for deep learning applications

Publication Date: 2022.11.08 INTEL CORP

AI summary

Examples include techniques to manage training or trained models for deep learning applications. Examples include routing commands to configure a training model to be implemented by a training module or configure a trained model to be implemented by an inference module. The commands routed via out-of-band (OOB) link while training data for the training models or input data for the trained models are routed via inband links.