Distributed Neural Network Acceleration With Universal Interconnect

The patent divides the neural network computation into multiple cycles, where each cycle processes a subset of input neurons. The computation is segmented across time (multiple cycles) rather than requiring a single high-power GPGPU, allowing lower-power hardware to achieve the same overall computational performance through sequential processing.

The patent employs dynamic resource allocation where the number of active computation units and memory access patterns are adjusted across different cycles. The system dynamically activates only the necessary computation resources for each cycle, optimizing the balance between computational performance and power consumption without requiring continuous high-power hardware operation.

The patent replaces expensive GPGPU hardware with cheaper, specialized computation units designed for specific neural network operations. These simpler, less expensive computation units are activated in cycles, providing the necessary computational performance at a lower hardware cost, similar to using disposable or specialized components rather than high-cost general-purpose hardware.

By segmenting the computational task across multiple cycles and using simpler hardware units for each cycle, the system achieves the same overall computational performance as GPGPU but with lower hardware cost. The segmentation allows the use of less expensive components that can be activated sequentially rather than requiring expensive hardware to run continuously.

The patent employs a universal communication interface that enables different types of computation units to work together in the distributed system. This standardized interface simplifies the system architecture by allowing heterogeneous hardware components to interact through a common protocol, reducing the complexity that would otherwise arise from integrating diverse hardware types.

The system performs preliminary actions by pre-loading computation units and preparing data structures before each cycle begins. This preparation reduces the complexity of coordinate computation and data transfer during actual execution, as the necessary resources are already in place and configured, simplifying the runtime operations.

The patent ensures continuity of useful action by overlapping data preparation and computation operations. While one computation unit is processing a cycle, other units are preparing data for the next cycle, and memory operations are performed in parallel with computation. This continuous utilization of resources minimizes idle time and maintains high computational throughput despite the multi-cycle approach.

The system performs preliminary actions by pre-computing and buffering data structures, weights, and computation graphs before each cycle. This preparation allows the actual computation to proceed more quickly without delays, reducing the time penalty associated with multiple cycles. The preliminary actions are performed in parallel with computation operations to minimize overall processing time.