Multi-task recurrent neural networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks struggle to efficiently perform multiple tasks in rapid succession, particularly in predicting memory accesses in computing systems, due to sequential dependencies and high latency in processing time-series data.
Innovation Solution
Implementing a multi-task recurrent neural network with state registers to store internal states for different tasks, allowing the network to process inputs from multiple tasks in parallel, and using sequence identifying data to select the appropriate state for each task, thereby reducing latency and improving prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a recurrent neural network processes multiple tasks sequentially using a single state register, then the device complexity is reduced, but the processing speed and productivity deteriorate due to sequential dependencies and high latency
Solution Approach 1:
The patent divides the single state register into multiple separate state registers, with each register dedicated to storing internal state data for a specific task. This segmentation allows the recurrent neural network to maintain independent state information for multiple tasks simultaneously, enabling parallel processing without interference between tasks, thus resolving the contradiction between device complexity and processing speed
Solution Approach 2:
The recurrent neural network cell is designed to be universal and can process multiple different tasks by selecting which state register to read from and write to based on the current task. The same RNN cell infrastructure handles different tasks (e.g., memory access prediction, cache miss prediction, branch prediction) by switching between multiple state registers, achieving multi-functionality without increasing the fundamental processing architecture complexity
2Measurement precision
If a recurrent neural network uses a single state register for one task, then the measurement precision for that task is maintained, but the adaptability to perform multiple tasks deteriorates
Solution Approach 1:
By segmenting the state storage into multiple dedicated registers, each task receives its own isolated state memory, preserving the precision and integrity of task-specific predictions while simultaneously enabling the system to adapt to multiple different prediction tasks through the selective use of appropriate state registers
Solution Approach 2:
The system changes the parameter of state storage capacity from a single shared register to multiple task-specific registers. This parameter change allows the recurrent neural network to maintain high prediction accuracy for each individual task while gaining the adaptability to switch between different prediction tasks by selecting the corresponding state register
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, relating to multi-task recurrent neural networks. One of the methods includes maintaining data specifying, for a recurrent neural network, a separate internal state for each of a plurality of memory regions; receiving a current input; identifying a particular memory region of the memory access address defined by the current input; selecting, from the internal states specified in the maintained data, the internal state for the particular memory region; processing, in accordance with the selected internal state for the particular memory region, the current input in the sequence of inputs using the recurrent neural network to: generate an output, the output defining a probability distribution of a predicted memory access address, and update the selected internal state of the particular memory region; and associating the updated selected internal state with the particular memory region in the maintained data.