Compiler Fusion of Multi-Dimensional Array Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing technologies face inefficiencies in parallel processing, particularly in repetitive calculations, as intermediate values are often recalculated multiple times, leading to increased processing time and memory bandwidth usage.
Innovation Solution
A compiler is configured to recognize repetitive operations in multi-dimensional arrays of processing units, fuse them into single operations, and store intermediate results for reuse, reducing the number of computations and memory accesses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If intermediate values are recalculated multiple times in parallel processing, then processing can be performed using standard algorithms, but processing time increases and memory bandwidth usage increases
Solution Approach 1:
The patent applies preliminary action by computing intermediate values once and storing them in memory before they are needed for subsequent calculations. The compiler identifies repetitive intermediate values and pre-calculates them, allowing multiple processing units to reuse the same stored values without recalculating, thus reducing processing time while maintaining algorithmic correctness
Solution Approach 2:
The patent merges repetitive calculations into a single computation by identifying that multiple processing units will compute the same intermediate values. Instead of each unit performing independent calculations, the system combines these redundant operations into one computation whose result is then shared across all needed units, eliminating wasted processing time
2Productivity
If intermediate values are stored for reuse, then processing time is reduced, but memory bandwidth usage increases
Solution Approach 1:
The patent applies local quality by storing intermediate results in the local memory of processing units rather than requiring frequent accesses to centralized memory. Each processing unit maintains copies of intermediate values it needs, allowing computations to proceed with minimal memory bandwidth consumption while maintaining high processing speed through local data availability
Solution Approach 2:
The patent uses copying by creating multiple copies of intermediate results and distributing them to different processing units. Instead of having units repeatedly access centralized memory, the compiler generates code that copies intermediate values to local storage at each unit, enabling fast access without consuming memory bandwidth during the main computation phase
3Productivity
If operations are fused into single operations, then the number of computations is reduced, but compiler complexity increases
Solution Approach 1:
The patent applies feedback by implementing a cost model that analyzes computation graphs and provides feedback on which intermediate values are worth caching. The compiler evaluates the frequency of intermediate value reuse and the computational cost of recalculating versus storing, using this feedback to automatically decide which operations to fuse and which intermediate results to preserve in memory
Solution Approach 2:
The patent changes parameters by transforming the computation graph representation to identify fusion opportunities. The compiler modifies operation parameters such as storage location and computation timing, converting a sequence of separate operations into fused operations with optimized memory access patterns, thereby improving computational efficiency without manual intervention
Data Source
AI summary
Methods, systems, and apparatus, including instructions encoded on storage media, for performing reduction of gradient vectors and similarly structured data that are generated in parallel, for example, on nodes organized in a mesh or torus topology defined by connections in at least two dimension between the nodes. The methods provide parallel computation and communication between nodes in the topology.