Skip Predictor for RNN State Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Pre-trained recurrent neural networks (RNNs) require retraining or fine-tuning when used with a binary predictor, which is inefficient and limits the use of pre-trained models in applications where computational savings are needed, such as in edge devices with energy and power constraints.
Innovation Solution
A skip predictor is trained independently from the pre-trained RNN to determine whether to skip certain time steps, allowing for efficient processing without retraining the RNN, using distance metrics like cosine or L2 norm to decide on processing input data values based on hidden state vector changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a binary predictor is added to determine whether to skip time steps, then computational efficiency is improved, but the pre-trained RNN must be retrained or fine-tuned
Solution Approach 1:
The system is segmented into two independent components: a pre-trained RNN model and a separately trained skip predictor. The skip predictor is trained independently on training data to determine which time steps should be skipped, without requiring retraining or fine-tuning of the pre-trained RNN. This segmentation allows the RNN to remain frozen while the skip predictor learns to identify redundant time steps, thereby improving computational efficiency without the complexity of retraining the entire model.
2Reliability
If all time steps are processed by the RNN, then accuracy is maintained, but computational resources and energy consumption increase
Solution Approach 1:
The skip predictor identifies and skips redundant time steps in the sequence processed by the RNN. By using the skip predictor to determine which time steps can be safely skipped without processing, the system rushes through unnecessary computations while maintaining prediction accuracy. This selective skipping significantly reduces energy consumption and computational resource usage, particularly for edge devices with limited power supplies.
3Speed
If a binary predictor is used to skip time steps, then processing speed is improved, but the system complexity increases due to coordinated training
Solution Approach 1:
The training complexity is extracted and isolated to the skip predictor component, which is trained separately on training data. The pre-trained RNN remains frozen and unchanged. This extraction of the training process to a separate, simpler component (the skip predictor) maintains processing speed improvements while reducing overall system architecture complexity, as the two components can be developed and trained independently.
Data Source
AI summary
The present disclosure advantageously provides a system and a method for skipping recurrent neural network (RNN) state updates using a skip predictor. Sequential input data are received and divided into sequences of input data values, each input data value being associated with a different time step for a pre-trained RNN model. At each time step, the hidden state vector for a prior time step is received from the pre-trained RNN model, and a determination, based on the input data value and the hidden state vector for at least one prior time step, is made whether to provide or not provide the input data value associated with the time step to the pre-trained RNN model for processing. When the input data value is not provided, the pre-trained RNN model does not update its hidden state vector. Importantly, the skip predictor is trained without retraining the pre-trained RNN model.