View-Class Operator Subgraph Optimization for Tensor Data Contiguity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In intelligent computing systems, view-class operators in computing graphs lead to discontiguous tensor data in memory, resulting in low memory access efficiency and high time consumption due to the need for frequent data rearrangement to achieve contiguity.
Innovation Solution
A computing graph optimization method that constructs a view-class operator subgraph for subsequent memory data contiguity processing and performs operator fusion to reduce data moving and operator calling, thereby improving data access efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If view-class operators are used to change tensor data appearance, then programming flexibility is improved, but memory access efficiency deteriorates due to discontiguous data arrangement
Solution Approach 1:
The patent performs preliminary detection of view-class operators in the computing graph during the compilation phase. By identifying operators that will cause discontiguous data arrangements in advance, the system can pre-plan data contiguity processing strategies, such as inserting data copying operations or adjusting memory layout beforehand, thereby avoiding inefficient random access during actual computation and improving memory access efficiency while preserving programming flexibility
Solution Approach 2:
The patent segments the computing graph by identifying and extracting view-class operators from the overall computation flow. This segmentation allows the system to separately analyze and optimize data layout operations from computational operations, enabling targeted optimization of memory access patterns without affecting the flexibility provided by view-class operators in the computational logic
2Productivity
If data is moved and rearranged one piece by one piece to achieve contiguity, then memory contiguity is improved, but time consumption increases significantly
Solution Approach 1:
The patent merges multiple scattered data copying operations into a single batched memory copy operation. By collecting all necessary data elements that need to be copied for contiguity and performing them in one consolidated memory operation, the system reduces the overhead of multiple small operations and significantly decreases the total time required for data rearrangement while achieving the desired memory contiguity
Solution Approach 2:
The patent performs preliminary analysis of the computing graph to determine exactly which data elements need to be copied and their target positions before execution. This pre-computed copy plan allows the system to perform optimized batched memory operations rather than ad-hoc piece-by-piece copying, reducing time consumption by eliminating redundant operations and enabling efficient memory bandwidth utilization
3Adaptability or versatility
If many view-class operators are used to process tensor data, then data transformation capability is improved, but the number of operator calls increases leading to high time overhead
Solution Approach 1:
The patent merges multiple consecutive view-class operators into a single optimized operator when possible. By analyzing the composition of multiple view operations (such as multiple transpose or slice operations) and combining them into one equivalent operation, the system reduces the number of operator calls and their associated overhead while maintaining the same data transformation capability, thereby reducing time overhead
Data Source
AI summary
A computing apparatus performing a computing graph optimization method is included in a combined processing apparatus. The combined processing apparatus includes an interface apparatus and other processing apparatus. The computing apparatus interacts with other processing apparatus to jointly complete a computing operation specified by a user. The combined processing apparatus further includes a storage apparatus which is connected to the computing apparatus and other processing apparatus respectively and is configured to store data of the computing apparatus and other processing apparatus. The disclosed scheme may optimize data access by constructing a view-class operator subgraph. By optimizing the view-class operator subgraph, memory moving and operator calling on the device side may be reduced. By reversely deducing a view-class operator that causes tensor data to become a memory discontiguous state, a suitable computing library operator may be called to convert the tensor data into a memory contiguous state.