AI Model Partition Scheduling via Reversed Computation Graph

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current information processing systems face challenges in efficiently deploying and executing large-scale AI models due to limitations in computing resources and communication bandwidth, particularly in implementing model parallelism across different computing devices with varying neural network architectures and parameter formats.

Innovation Solution

The proposed solution involves generating a reversed computation graph from an intermediate representation of an AI model, allowing for 'scheduling at last moment' (SALM) policy that partitions functions into sequential sets for parallel execution, optimizing resource usage and balancing computation loads across devices.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a single computing device is used to execute AI models, then device complexity is reduced, but computing capability and communication bandwidth are insufficient to satisfy large-scale AI model demands

Engineering Contradiction:

Improvecomputing capabilityVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent divides a large-scale AI model into multiple smaller partitions, each of which can be executed on different computing devices. The computation graph representing the AI model is segmented into sub-graphs that can be independently processed and scheduled across multiple devices, enabling distributed execution while maintaining model functionality.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a temporal dimension to the scheduling problem by using a reversed computation graph approach. Instead of scheduling partitions in the forward temporal direction, the system schedules from the end of execution backwards, allowing partitions to be assigned to devices based on both spatial (device) and temporal (execution timing) dimensions simultaneously.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Productivity

If AI model partitions are scheduled early, then resource allocation is planned in advance, but scheduling efficiency is reduced due to inability to optimize parallel computation at the last moment

Engineering Contradiction:

Improvescheduling efficiencyVSAvoidscheduling time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent inverts the traditional scheduling approach by reversing the computation graph and scheduling partitions from the end of execution backwards to the beginning. This inversion allows the system to identify which partitions can be executed in parallel at the last moment, maximizing scheduling efficiency while minimizing the time lost to scheduling decisions.

Inventive Principle:

Principle #13The other way round (Inversion)

3Productivity

If partitions are executed sequentially without parallelization, then execution order is simple to manage, but computation efficiency is reduced due to inability to utilize multiple computing devices simultaneously

Engineering Contradiction:

Improvecomputation efficiencyVSAvoidscheduling complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The computation graph is segmented into independent partitions that can be executed in parallel. By identifying independent sub-graphs within the overall computation graph, the system can assign different partitions to different computing devices simultaneously, achieving parallel execution while maintaining manageable scheduling complexity through the reversed graph approach.

Inventive Principle:

Principle #1Segmentation

4Productivity

If computing resources are allocated to AI model partitions, then model execution is enabled, but resource wastage occurs due to imbalanced scheduling and poor locality between caller and callee functions

Engineering Contradiction:

Improveresource utilizationVSAvoidresource wastage

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

By reversing the computation graph and scheduling from the end backwards, the system can identify which partitions will be executed last and prioritize their resource allocation. This ensures that resources are allocated to partitions with the best locality and execution timing, reducing resource wastage caused by imbalanced scheduling and poor caller-callee function locality.

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentUS11579924B2Scheduling artificial intelligence model partitions based on reversed computation graph

Publication Date: 2023.02.14 EMC IP HLDG CO LLC

AI summary

Techniques are disclosed for scheduling artificial intelligence model partitions for execution in an information processing system. For example, a method comprises the following steps. An intermediate representation of an artificial intelligence model is obtained. A reversed computation graph corresponding to a computation graph generated based on the intermediate representation is obtained. Nodes in the reversed computation graph represent functions related to the artificial intelligence model, and one or more directed edges in the reversed computation graph represent one or more dependencies between the functions. The reversed computation graph is partitioned into sequential partitions, such that the partitions are executed sequentially and functions corresponding to nodes in each partition are executed in parallel.