Fused Computation Graph for Multi-Framework Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing and machine learning frameworks maintain separate incompatible memory spaces, leading to extensive memcopy operations when transferring data between them, which limits data sharing and efficiency.
Innovation Solution
Combining multiple frameworks into a fused computation graph, removing memcopy operations between them, and remapping memory spaces to a target hardware's memory spaces for unified execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is transferred between different frameworks (e.g., PyTorch to NumPy), then compatibility and flexibility are improved, but extensive memcopy operations increase memory consumption and reduce execution speed
Solution Approach 1:
The patent merges multiple framework computation graphs into a single unified computation graph that can be executed on target hardware. By combining the computation logic from different frameworks (PyTorch, TensorFlow, NumPy, Pandas) into one unified graph, the system eliminates the need for repeated data copying between framework memory spaces, thereby reducing memory consumption while maintaining the ability to handle workloads from multiple frameworks.
Solution Approach 2:
The unified computation graph acts as an intermediary layer between the source frameworks and the target hardware. Instead of directly transferring data between framework memory spaces (which causes extensive copying), the computation graph serves as a mediator that processes data in a unified memory space, reducing the need for memcopy operations while preserving framework compatibility.
2Adaptability or versatility
If data is transferred between different frameworks, then framework interoperability is improved, but execution speed decreases due to extensive copying operations
Solution Approach 1:
By merging multiple framework computation graphs into a single unified computation graph, the system enables direct execution on target hardware without intermediate copying steps. This consolidation removes the sequential copying operations that slow down execution, thereby improving speed while preserving the ability to process workloads from multiple frameworks.
Solution Approach 2:
The system performs preliminary extraction and fusion of computation graphs from different frameworks before execution. By preparing the unified computation graph in advance and removing unnecessary memcopy operations, the system optimizes the execution path, eliminating runtime copying overhead and improving execution speed.
3Stability of the object's composition
If separate memory spaces are maintained for each framework, then framework independence is preserved, but data sharing efficiency decreases due to necessary copies
Solution Approach 1:
The patent merges computation graphs from multiple independent frameworks into a unified graph that executes on target hardware with unified memory management. This merging enables efficient data sharing across what were previously separate framework memory spaces, improving data sharing efficiency while the abstraction layer preserves the logical independence of each framework's computation logic.
Data Source
AI summary
A method for combining multiple different data processing, artificial intelligence and/or machine learning frameworks for execution by a target hardware includes extracting one or more computation graphs from each of the different frameworks. The computation graphs are combined into a fused computation graph. Memcopy operations are removed at edges between the computation graphs of the different frameworks. Memory spaces for computations in the fused computation graph are remapped to memory spaces of the target hardware.