Dynamic Data Pipeline Composition for HPC Accelerator Systems

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Implementing an efficient distributed computing environment for high-performance computing (HPC) applications like deep learning model training is challenging due to intensive computational workloads and massive data volumes, which hinder system performance and scalability.

Innovation Solution

Decoupling data pipeline tasks from the execution flow in a distributed computing system by dynamically composing a data flow pipeline with logical nodes to perform data storage, input/output, and pre-processing operations concurrently with the execution flow, optimizing resource usage and scalability.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data pipeline tasks are integrated with execution flow in distributed computing system, then system complexity is reduced, but system performance and scalability are hindered due to intensive computational workloads and massive data volumes

Engineering Contradiction:

Improvesystem performanceVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments the computing system into distinct components: data pipeline tasks are separated from execution flow tasks, with data pipeline tasks handled by dedicated data pipeline nodes and execution flow tasks handled by compute nodes. This segmentation allows independent optimization of each component, improving overall system performance while managing complexity through clear separation of concerns.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary data pipeline layer that mediates between data storage and compute nodes. This data pipeline layer includes components such as data loading nodes, data preprocessing nodes, and data caching nodes that handle data preparation independently, allowing compute nodes to focus on execution without being burdened by data management complexity.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If data pipeline tasks are decoupled from execution flow, then resource efficiency and scalability are improved, but system complexity increases due to separate data processing and task execution flows

Engineering Contradiction:

ImprovescalabilityVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system is segmented into independent data pipeline nodes and compute nodes, allowing each to be scaled independently. Data pipeline nodes can be increased to handle larger data volumes without affecting compute node configuration, and vice versa, enabling flexible scalability while maintaining manageable complexity through modular design.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The data pipeline nodes are designed with universal functionality to handle various data processing operations (loading, preprocessing, caching, transformation) across different compute nodes and workloads. This multi-functionality reduces the need for specialized components for each function, managing complexity while providing versatile scalability.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Power

If massive volume of data is streamed and coordinated between shared computing resources, then computational capability is utilized, but system performance is limited by data coordination overhead

Engineering Contradiction:

Improvecomputational capabilityVSAvoiddata coordination overhead

Core Design Contradiction:

PowerVSLoss of time

Solution Approach 1:

The data pipeline performs preliminary actions by pre-processing and pre-positioning data in caching nodes before compute nodes need it. Data loading nodes extract and prepare data in advance, and data caching nodes store pre-processed data closer to compute nodes, reducing the time and overhead of data coordination during actual computation.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Data caching nodes act as intermediaries between data storage and compute nodes, buffering and managing data flow to reduce coordination overhead. These intermediary nodes handle data staging and transfer optimizations, allowing compute nodes to retrieve data with minimal coordination complexity and time loss.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS10776164B2Dynamic composition of data pipeline in accelerator-as-a-service computing environment

Publication Date: 2020.09.15 EMC IP HLDG CO LLC

AI summary

Techniques are provided to decouple data pipeline tasks from an execution flow of a high-performance computing task (e.g., distributed deep model training) in a distributed computing system. For example, a method includes receiving a client request to provision resources for executing a computing job, provisioning accelerator resources of one or more accelerator server nodes in the distributed computing system to perform tasks associated with an execution flow of the computing job, and provisioning a logical nodes within the distributed computing system to compose a data flow pipeline which is configured to perform data flow operations associated with the computing job for providing data to the provisioned accelerator resources to perform the tasks associated with the execution flow of the computing job. The data flow operations include, e.g., data storage input/output operations, data pre-processing operations, and data staging operations, which are decoupled from the execution flow of the computing job.