Graph Context Operator Checks for AI Accelerator Clustering

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing deep learning frameworks face challenges in efficiently executing graph clustering and execution due to unsupported operators on target hardware, leading to runtime errors such as crashes and sub-optimal performance.

Innovation Solution

The proposed solution involves a graph context-based operator check mechanism that analyzes the deep learning graph in topological order to identify and mark nodes as supported or unsupported based on attributes like input node parameters, dimensions, and shapes, thereby preventing unsupported operators from being scheduled on target hardware.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If graph clustering is performed without operator checks, then clustering speed is improved, but runtime errors occur due to unsupported operators on target hardware

Engineering Contradiction:

Improveclustering speedVSAvoidruntime error prevention

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

The patent performs operator checks during the graph clustering phase before execution, marking nodes as supported or unsupported based on target hardware capabilities. This preliminary identification prevents runtime errors by ensuring unsupported operators are detected early in the clustering process rather than during execution.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If comprehensive operator checks are performed on all nodes, then reliability is improved, but processing time increases

Engineering Contradiction:

Improveoperator support verificationVSAvoidgraph processing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent segments the graph into clusters and performs operator checks at the cluster level rather than individually on all nodes. The graph is partitioned into multiple clusters, and operator support verification is performed on each cluster, reducing the overall processing time while maintaining comprehensive reliability checks.

Inventive Principle:

Principle #1Segmentation

3Productivity

If graph is partitioned into multiple clusters, then hardware utilization is improved, but cluster formation complexity increases

Engineering Contradiction:

Improvehardware utilizationVSAvoidclustering algorithm complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent applies local quality by optimizing each cluster individually based on its specific characteristics and the target hardware capabilities. Each cluster is formed and optimized locally with considerations for hardware utilization, allowing different clusters to have different optimization strategies rather than applying a uniform complex algorithm to the entire graph.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS12242973B2Graph context-based operator checks to improve graph clustering and execution in AI accelerator framework integration

Publication Date: 2025.03.04 INTEL CORP

AI summary

Systems, apparatuses and methods may provide for technology that parses, at runtime, a deep learning graph in topological order to identify a plurality of nodes, marks a first set of nodes in the plurality of nodes as unsupported by target hardware, and marks a second set of nodes in the plurality of nodes as supported by the target hardware, wherein the first set of nodes and the second set of nodes are marked based on one or more attributes defining operation functionality, and wherein the one or more attributes include one or more of an input node parameter, a dimension, or a shape.