Neural Network Data Flow Optimization via Global Variable Scope
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for data transfer between subgraphs in neural network computation are inefficient, particularly for models with cycle structures, leading to suboptimal data flow and memory usage.
Innovation Solution
A data flow method for neural network computation that initializes variable lifecycles, defines propagation rules for variables in use and redefined variables, analyzes valid variables and expressions, allocates memory cell blocks, and optimizes available expressions to improve computational efficiency and reduce memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If frequent swapping with memory through computational nodes is used for data transfer between subgraphs, then data flow can be maintained, but computational efficiency deteriorates and memory usage increases
Solution Approach 1:
The patent merges multiple subgraphs into a unified computational graph with global variable scope. This allows variables to be shared across subgraph boundaries without frequent memory swapping, as the global variable mechanism enables direct access to variable definitions throughout the entire computation graph, thereby improving computational efficiency and reducing memory transfer overhead.
Solution Approach 2:
The patent introduces a global variable mechanism as an intermediary between subgraphs. Instead of directly swapping data between subgraph memory spaces, the global variable acts as a mediator that coordinates data flow across subgraph boundaries, enabling efficient data sharing without the overhead of frequent memory swaps.
2Productivity
If a data flow algorithm is designed for each subgraph, then local data flow can be optimized, but global data flow efficiency for cyclic structures deteriorates
Solution Approach 1:
The patent implements a universal data flow algorithm that operates at the global computational graph level rather than requiring separate algorithms for each subgraph. This global algorithm handles all data flow scenarios uniformly, including cyclic structures, by maintaining a global variable scope and lifecycle management system that works consistently across the entire computation graph.
Solution Approach 2:
The patent segments the computational graph into subgraphs for modular organization while maintaining global variable scope. This segmentation allows the system to handle complex cyclic structures by breaking them into manageable subgraph components that can be processed independently while still benefiting from global coordination through the unified variable management mechanism.
3Adaptability or versatility
If variables are frequently redefined in computational graphs, then computation flexibility is improved, but variable lifecycle management complexity increases
Solution Approach 1:
The patent implements a feedback mechanism in the form of a global variable scope system that automatically tracks and manages variable lifecycles. When a variable is redefined, the system receives feedback about the new definition and automatically adjusts the variable's scope and lifetime, eliminating the need for manual lifecycle management and reducing complexity while maintaining computation flexibility.
Data Source
AI summary
Disclosed are a data flow method and apparatus for neural network computation. The data flow method for neural network computation includes initializing the lifecycle of a variable in a computational graph; and defining a propagation rule for a variable in use to flow through a node. A definition of the variable is produced at a precursor node of the node, such that an input set of valid variables flowing through the node contains the variable. The method may be used on neural network computation in a deep learning training system.