Embedded ML Accelerator Weight Loading Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing embedded machine learning accelerators face challenges in reducing power consumption and latency, particularly due to high energy demands during data movement and inefficient weight parameter loading processes.
Innovation Solution
The implementation of a hardware function that optimizes weight loading and configuration by reorganizing configuration parameter data into block sizes that align with the hardware accelerator's architecture, reducing data movement and latency, and using auto-incrementing addresses for efficient data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If weight parameters are loaded using conventional methods, then the hardware accelerator can perform machine learning operations, but power consumption is high due to excessive data movement
Solution Approach 1:
The patent segments weight parameters into multiple blocks and loads them in parallel using multiple load ports. This divides the single sequential loading process into concurrent parallel operations, reducing total loading time while maintaining efficient power usage through optimized data movement patterns
Solution Approach 2:
The patent implements preliminary action by pre-processing and reorganizing weight parameters into optimized formats before loading them into the hardware accelerator. Configuration data is prepared in advance with proper addressing and blocking, reducing the computational overhead and power consumption during actual operation
2Speed
If data is moved frequently for processing, then computation speed is maintained, but power consumption increases
Solution Approach 1:
The patent ensures continuous useful action by maintaining weight parameters in optimized block formats within the hardware accelerator once loaded. The parallel load ports continuously feed processed data without interruption, and the blocking optimization ensures data remains in accessible formats throughout computation, minimizing re-loading and reducing power consumption
Solution Approach 2:
The patent applies dynamics by implementing configurable load ports that can dynamically adjust their operation based on the specific machine learning model requirements. The system can adapt the number of active load ports, block sizes, and data movement patterns to match the computational demands of different models, optimizing the balance between speed and power consumption
3Use of energy by moving object
If weight parameters are loaded in small increments, then power consumption is reduced, but latency increases
Solution Approach 1:
The patent segments weight parameters into blocks that are loaded in parallel through multiple load ports. This segmentation transforms the sequential loading process into concurrent operations, where multiple blocks are loaded simultaneously, reducing total latency while maintaining manageable data movement sizes that optimize power consumption
Solution Approach 2:
The patent merges multiple data loading operations into a single coordinated parallel process. By combining the efforts of multiple load ports working simultaneously on different weight parameter blocks, the system achieves the power efficiency of incremental loading with the speed of bulk loading, as the combined parallel operations complete faster than sequential processing
Data Source
AI summary
Systems and methods reduce power consumption in embedded machine learning hardware accelerators and enable cost-effective embedded at-the-edge machine-learning and related applications. In various embodiments this may be accomplished by using hardware accelerators that comprise a programmable pre-processing circuit that operates in the same clock domain as the accelerator. In some embodiments, tightly coupled data loading first-in-first-out registers (FIFOs) eliminate clock synchronization issues and reduce unnecessary address writes. In other embodiments, a data transformation may gather source data bits in a manner that allows loading full words of native bus width to reduce the number of writes and, thus, overall power consumption.