ANN Weight Array Tiling for Computational Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies, including CPUs, GPUs, and FPGAs, face challenges in efficiently processing artificial neural networks (ANNs) due to high computational loads, power consumption issues, and lack of integration with AI software environments, limiting their ability to handle massive data movements and parallel computations effectively.
Innovation Solution
A system and method that modify the structure of ANNs by reordering, splitting, or inserting arrays of weights to reduce computational operations, allowing weights to be processed in a single cycle while maintaining output equality, thereby reducing the overall computational complexity and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If GPUs are used to process large ANNs, then throughput capacity is improved, but power consumption increases excessively
Solution Approach 1:
The patent segments the weight arrays into multiple sub-arrays and distributes them across multiple processing elements. Each processing element handles a portion of the computation, allowing the system to achieve high throughput through parallel processing while each individual element consumes less power, thereby resolving the contradiction between throughput capacity and power consumption.
Solution Approach 2:
The patent introduces a new dimensional organization of weight arrays by creating tiled structures with multiple sub-arrays. This dimensional transformation allows weights to be organized in a way that enables parallel access and computation across multiple processing elements, achieving high throughput without requiring a single high-power processor.
2Productivity
If FPGAs are used to compute ANNs, then parallel computation capability is improved, but programming complexity increases significantly
Solution Approach 1:
The patent creates a universal weight array structure with standardized tiling and sub-array organization that can be applied to different ANN configurations. This universal approach allows the same programming methodology to work across various FPGA implementations, reducing programming complexity while maintaining parallel computation capabilities.
Solution Approach 2:
The patent changes the organizational parameters of weight arrays from traditional flat structures to tiled multi-dimensional structures. This parameter transformation enables systematic parallel processing on FPGAs while providing a regular, predictable pattern that simplifies programming compared to custom parallel architectures.
3Ease of operation
If weight arrays are processed in traditional sequential manner, then programming simplicity is maintained, but computational efficiency decreases
Solution Approach 1:
The patent performs preliminary organization of weight arrays into tiled structures with multiple sub-arrays before the actual computation begins. This pre-processing step creates an optimized data layout that enables efficient parallel access during computation, improving computational efficiency while the tiling process itself can be performed once during initialization rather than during each computation.
Solution Approach 2:
The patent transforms the traditional one-dimensional sequential weight array into a multi-dimensional tiled structure. This dimensional change enables parallel processing by organizing weights in a way that allows simultaneous access from multiple processing elements, thereby improving computational efficiency while maintaining programming simplicity through regular pattern repetition.
Data Source
AI summary
Systems and methods for modifying a structure of an artificial neural network (ANN) are provided. An example method comprises receiving, by one or more processing units, a plurality of arrays of weights associated with the ANN, modifying, by the processing units, the plurality of arrays of weights to generate a further plurality of further arrays of weights, where after the modification the following conditions are satisfied: an amount of operations required for computing neurons of the ANN using the further plurality of further arrays of weights is less than an amount of operations required for computing same neurons of the ANN using the plurality of arrays of weights; and outputs of the neurons of the ANN computed using the plurality of arrays of weights are substantially equal to further outputs of the neurons of the ANN using the further plurality of further arrays of weights.