Neural Network Data Arrangement Conversion for AI Chip Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In neural network operations, the need for frequent data arrangement conversions between operators leads to high memory consumption and slow computing speeds, especially when dealing with multiple operators requiring different data arrangement types.
Innovation Solution
A method that dynamically adjusts input data to match the target data arrangement type of each operator only when necessary, eliminating the need for data arrangement conversions on output data and allowing for variable data arrangements during network training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data arrangement conversion operations are performed on both input and output of each operator to maintain unified data arrangement type, then data compatibility between operators is improved, but memory consumption increases and computing speed decreases
Solution Approach 1:
The patent introduces dynamic data arrangement conversion where the conversion is performed only when necessary (when input data arrangement type differs from operator's target type) rather than universally applying conversion. This dynamic approach maintains data compatibility when needed while avoiding unnecessary conversions that reduce computing speed.
Solution Approach 2:
The patent changes the parameter of data arrangement type dynamically based on operator requirements and input data characteristics. Instead of using a fixed unified data arrangement type throughout the entire computation graph, the system adapts the data arrangement type at each operator based on its specific needs, reducing unnecessary conversions.
2Stability of the object's composition
If data arrangement conversion operations are performed on output data of each operator, then data arrangement consistency is improved, but memory overhead increases due to intermediate results
Solution Approach 1:
The patent extracts the data arrangement conversion operation from the output side and relocates it to the input side of operators. By performing conversion only when input data needs to be transformed to match the operator's target arrangement type, the system eliminates redundant intermediate results and reduces memory overhead while maintaining data arrangement consistency.
Solution Approach 2:
The patent inverts the traditional approach by performing data arrangement conversion at the input of operators rather than at the output. This inversion allows the system to maintain consistency without generating unnecessary intermediate results, as conversion happens only when the input data arrangement differs from what the operator expects.
3Device complexity
If a unified data arrangement type is used for all operators, then system simplicity is improved, but flexibility to handle different operator requirements is reduced
Solution Approach 1:
The patent implements a unified data arrangement management system that can handle multiple data arrangement types through a single adaptive mechanism. The system maintains simplicity by using a consistent conversion framework while achieving flexibility by dynamically adapting to different operator requirements, effectively making the system multi-functional.
Data Source
AI summary
The present disclosure relates to the field of artificial intelligence chips, specifically involving a neural network computation method and related devices. The computing device of the present disclosure includes a processor, a communication interface, and other processing devices. The processor and the communication interface are connected to each other through a bus to jointly complete the computations specified by a user. The computing device may also include a storage device, which is connected to the processor and other processing devices for data storage of the computing device and other processing devices. Based on an input data arrangement type and a target data arrangement type, the present disclosure performs data arrangement type conversion only on input of an operator at most, which can reduce unnecessary data arrangement type conversions during the computing process, reduce memory overhead, and improve computing efficiency.