Neural Processing Tiling Optimization via Deep Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for determining the optimal tiling condition for neural processing devices are time-consuming and do not accurately account for random factors such as binary size and operation temperature, requiring frequent re-optimization due to changes in hardware and software characteristics and neural network models.
Innovation Solution
A system and method using deep reinforcement learning to optimize tiling conditions by generating agents that iteratively determine optimal tiling conditions based on reward values and Q-values, with a simulation environment and device environment working together to calculate and correct prediction and compensation Q-values, respectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If heuristic schemes are used to determine tiling conditions, then the process is simple to implement, but it takes much time and does not accurately reflect random factors such as binary size and operation temperature
Solution Approach 1:
The patent replaces traditional heuristic schemes (mechanical/manual experimentation) with a deep reinforcement learning-based automated system. The agent circuit generates multiple agents that iteratively determine optimal tiling conditions through learning, substituting manual heuristic adjustment with an intelligent automated system that considers binary size, operation temperature, and other dynamic factors.
Solution Approach 2:
The system enables self-service by allowing the neural processing device to automatically optimize its own tiling conditions through the deep reinforcement learning agent. The agent continuously learns from environmental feedback and adjusts tiling conditions without external intervention, making the system self-adapting to changing hardware and software characteristics.
2Ease of manufacture
If heuristic schemes are used to determine tiling conditions, then the method is simple, but it requires frequent re-optimization when hardware, software, or neural network model characteristics change
Solution Approach 1:
The patent implements dynamics by creating a tiling condition determination system that is inherently adaptive to changes. The deep reinforcement learning agent continuously monitors environmental feedback and dynamically adjusts tiling conditions in response to changes in hardware characteristics, software characteristics, or neural network models, making the system flexible and adaptable rather than static.
Solution Approach 2:
The system incorporates feedback mechanisms where the agent receives environmental feedback about the actual performance and consequences of selected tiling conditions. This feedback loop enables the agent to learn from outcomes and continuously improve its ability to adapt to changing conditions, ensuring optimal performance across different hardware, software, and model configurations.
3Productivity
If deep reinforcement learning is used to optimize tiling conditions, then the optimization efficiency is improved, but the device complexity increases due to agent circuits and simulation environments
Solution Approach 1:
The patent applies segmentation by dividing the complex optimization task into multiple independent agents, each responsible for specific layers of the neural network model. This modular approach allows parallel processing and distributes the computational complexity across multiple simpler units rather than requiring one monolithic complex system.
Solution Approach 2:
The system uses copying by creating multiple agents that replicate the same reinforcement learning structure. Each agent is a copy of the fundamental agent architecture, trained to handle specific layers. This replication strategy simplifies the overall design by using identical modular units rather than requiring unique complex logic for each optimization task.
Data Source
AI summary
A system of controlling neural processing based on deep reinforcement learning includes an agent circuit and an environment circuit. The agent circuit generates a plurality of agents based on layers included in a neural network model. Each agent repeatedly performs an iteration to determine a next action among a plurality of candidate actions based on a reward value and a plurality of Q values corresponding to a present action, where the candidate actions indicate a change of a tiling condition of an input feature map of each layer. Each agent determines an optimal tiling condition of each layer based on change of the reward value according to repeatedly-performed iterations. The environment circuit generates the reward value and the plurality of Q values with respect to each layer based on a tiling condition corresponding to the present action, where the Q values indicate prediction reward values of the candidate actions.