Decoupling Machine Learning Workloads from Infrastructure

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Running machine learning workloads is complex and cumbersome due to tight coupling with infrastructure resource management and embedded machine learning libraries, leading to time-consuming and error-prone source code updates and configuration challenges, especially for large-scale and distributed workloads.

Innovation Solution

Decomposing machine learning workloads into primitives and decoupling AI/ML algorithms from infrastructure building blocks, using role and channel building blocks to simplify workload definition and configuration, and introducing a connector mechanism to connect upstream and downstream workloads for forming combined workloads.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If machine learning workloads are tightly coupled with infrastructure resource management and embedded machine learning libraries, then the workloads can be executed on existing infrastructure, but the definition and configuration become complex and time-consuming

Engineering Contradiction:

Improveease of workload definitionVSAvoidworkload configuration complexity

Core Design Contradiction:

Ease of manufactureVSDevice complexity

Solution Approach 1:

The patent segments machine learning workloads into modular components: infrastructure-agnostic workload definitions separated from infrastructure-specific configuration. Workloads are divided into computational graphs, parameter sets, and infrastructure binding layers, allowing independent development and configuration without mutual interference.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent extracts the infrastructure coupling from the core workload definition. By removing infrastructure-specific dependencies from the workload specification layer and placing them in a separate binding/configuration layer, the system enables simple workload definition while maintaining infrastructure execution capability.

Inventive Principle:

Principle #2Taking out (Extraction)

2Adaptability or versatility

If source code updates are performed for different machine learning workloads, then the workloads can be customized, but the process becomes time-consuming and error-prone

Engineering Contradiction:

Improveworkload customizationVSAvoidsource code update time

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent uses template-based workload definitions that can be copied and reused. Pre-defined computational graphs, operator libraries, and configuration templates allow users to replicate proven workload patterns without rewriting code, reducing customization time while maintaining adaptability through parameter modification.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent enables workload customization through parameter modification rather than source code changes. Users can adapt workloads by changing configuration parameters, hyperparameters, and runtime settings while the underlying computational structure remains intact, making customization faster and less error-prone.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If large-scale machine learning workloads consisting of multiple models are configured, then comprehensive processing capabilities are achieved, but tracking behavior and updating becomes extremely complex

Engineering Contradiction:

Improveprocessing capabilityVSAvoidworkload management

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The patent implements a nested structure where individual machine learning models are encapsulated as modular units within larger workload compositions. Each model can be independently tracked, updated, and managed, while the overall workload provides coordinated execution. This nesting enables comprehensive processing capability while maintaining manageable complexity through hierarchical organization.

Inventive Principle:

Principle #7Nested doll (Nesting)

4Reliability

If machine learning workloads are embedded into machine learning libraries, then the workloads can leverage library functions, but configuration becomes error-prone and ad-hoc

Engineering Contradiction:

Improveworkload executionVSAvoidconfiguration accuracy

Core Design Contradiction:

ReliabilityVSManufacturing precision

Solution Approach 1:

The patent creates a universal workload definition framework that works across different machine learning libraries and infrastructure platforms. By defining workloads in a library-agnostic manner with standardized interfaces, the system ensures reliable execution while eliminating library-specific configuration errors through consistent, standardized configuration procedures.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11822976B2Extending machine learning workloads

Publication Date: 2023.11.21 CISCO TECHNOLOGY INC

AI summary

In one embodiment, a device presents information regarding an upstream machine learning workload and a downstream machine learning workload via a user interface. The device receives, via the user interface, a request to form a combined machine learning workload by connecting the upstream machine learning workload and the downstream machine learning workload. The device identifies, after receiving the request, a node associated with the upstream machine learning workload and a node associated with the downstream machine learning workload. The device forms the combined machine learning workload by configuring the node associated with the upstream machine learning workload to use one or more connector application programming interfaces to send data from the upstream machine learning workload to the node associated with the downstream machine learning workload for consumption.