Neural Network Load Balancing for Low-Power Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks in low-power and low-performance devices face challenges in efficiently processing large amounts of input data in real-time, requiring effective load balancing to optimize operations.
Innovation Solution
A method and apparatus that acquire operation groups with input feature maps and kernels, determine idle operating units, and perform convolution operations, using a load balancing unit to match queue units with idle operating units, and post-processing units to create output feature maps through pooling or activation functions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural networks process large amounts of input data in real-time on low-power devices, then processing efficiency and real-time performance are improved, but computational complexity and resource consumption increase
Solution Approach 1:
The patent divides the neural network computation into separate operation groups that are distributed across multiple operating units. Each operation group contains specific input feature maps, kernels, and associated metadata, allowing the system to process computations in manageable segments rather than monolithically, thereby improving efficiency while controlling complexity
Solution Approach 2:
The system dynamically assigns operation groups to idle operating units based on real-time availability. The load balancing unit monitors the state of operating units and routes operation groups to available resources, enabling adaptive resource utilization that optimizes processing efficiency without requiring fixed complex hardware configurations
2Productivity
If multiple operation groups are processed simultaneously on different operating units, then processing throughput is improved, but coordination complexity and data management overhead increase
Solution Approach 1:
The load balancing unit serves as an intermediary between the queue units and operating units. It receives operation groups from queue units, determines which operating units are idle, and routes the operation groups accordingly. This mediator simplifies coordination by centralizing the routing decision logic and eliminating the need for complex peer-to-peer coordination between processing units
Solution Approach 2:
Operation groups are prepared with all necessary metadata and dependencies beforehand, including tag information that specifies post-processing requirements. This preliminary organization of data allows the load balancing unit to make rapid routing decisions without needing to analyze computation details in real-time, reducing coordination overhead
3Productivity
If operation groups are routed to idle operating units dynamically, then resource utilization is improved, but routing decision complexity and latency increase
Solution Approach 1:
Each operating unit autonomously reports its availability state (idle or busy) to the load balancing unit. The load balancing unit uses these self-provided status signals to make routing decisions, eliminating the need for complex centralized scheduling algorithms and reducing the computational overhead associated with dynamic resource allocation
Solution Approach 2:
The system implements a feedback mechanism where operating units continuously signal their state to the load balancing unit. This feedback loop enables the load balancing unit to make real-time routing decisions based on current resource availability, optimizing resource utilization while keeping latency low through direct state-based routing rather than complex prediction algorithms
Data Source
AI summary
A method of performing an operation in a neural network apparatus may include acquiring operation groups each comprising at least one input feature map and at least one kernel, and tag information corresponding to each of the operation groups, determining an operating unit in an idle state from among operating units, performing, at the operating unit in the idle state, a convolution operation between an input feature map and a kernel included in a operation group from among the operation groups to create an intermediate feature map, determining, based on tag information corresponding to the operation group, a post-processing unit from among post-processing units, and creating, at the post-processing unit, an output feature map using the intermediate feature map.