Coefficient Mapping for Matrix Multiply Accelerator Utilization

The patent segments the neural network computation into distinct phases (training and inference) and implements specialized hardware architectures for each phase. The training system uses traditional high-power compute resources, while the inference system employs energy-efficient dedicated hardware accelerators that can be deployed at the edge, thus resolving the contradiction between compute power and energy consumption.

The patent introduces an intermediary component - a specialized inference engine or accelerator - that bridges the gap between the training phase on traditional systems and the deployment phase at the edge. This intermediary hardware is optimized for inference workloads, providing sufficient compute power while consuming significantly less energy than general-purpose computers.

The patent applies local quality by implementing heterogeneous computing architectures where different components have specialized functions. Instead of using uniform high-power processors throughout, the system employs dedicated inference accelerators with optimized circuitry specifically for neural network inference, reducing overall device size while maintaining necessary compute power at the edge.

The patent employs dynamic voltage and frequency scaling (DVFS) and adaptive resource allocation in the edge deployment. The inference engine can dynamically adjust its power consumption and computational capacity based on workload demands, allowing the device to maintain adequate compute power when needed while minimizing size and power consumption during idle or low-demand periods.

The patent implements preliminary action by pre-training neural network models on centralized systems and then deploying the trained model weights to edge devices in advance. This allows the edge devices to perform inference locally without needing to transmit data back and forth with remote systems, eliminating network latency while maintaining sufficient compute power through the pre-deployed models.

The patent extracts the inference capability from remote computing systems and places it directly at the edge devices. By taking out the neural network model and deploying it locally, the system eliminates the need for continuous network communication during inference, thus resolving the latency issue while maintaining adequate compute power through specialized local hardware.