Reverse Reinforcement Learning for Neural Network Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
State-of-the-Art neural networks require adaptation when transitioning from one dataset to another, as their performance is task-dependent and may not generalize well across different datasets, necessitating a method to modify training datasets to maintain optimal performance.
Innovation Solution
The process involves benchmarking a training dataset using a State Of The Art neural network, dividing it into slices, applying a sequence of atomic operations generated by a selection strategy generator, and performing reverse reinforcement learning to modify these slices, incorporating techniques like mask, out of order, data deletion, and hidden layer transition operations, to create a revised dataset that aligns better with the SOTA neural network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a State Of The Art neural network is used for a particular task, then performance for that task is optimized, but the network cannot generalize to different datasets without retraining
Solution Approach 1:
Instead of training the neural network to adapt to different datasets through conventional reinforcement learning, the patent inverts the approach by using the SOTA network as a benchmark to guide the modification of training data. The reverse reinforcement learning process uses the SOTA network's performance expectations to generate modified training slices that should enable the network to maintain SOTA performance across different tasks without retraining.
Solution Approach 2:
The patent modifies training data parameters by applying atomic operations (mask, out of order, data deletion, hidden layer transition) to training slices. These parameter changes transform the training data into revised slices that align better with SOTA network expectations, enabling generalization without retraining the network itself.
2Reliability
If training data is modified to improve compatibility with SOTA neural network, then performance maintenance is achieved, but the complexity of the data modification process increases
Solution Approach 1:
The training dataset is divided into multiple slices, and the modification process is broken down into atomic operations that can be applied independently to each slice. This segmentation allows complex data modification to be managed through simple, composable operations (mask, out of order, data deletion, hidden layer transition), reducing the overall complexity of the modification process.
Solution Approach 2:
The reverse reinforcement learning process uses the SOTA neural network as a feedback mechanism to evaluate and guide data modifications. The SOTA network provides performance expectations that feedback into the reverse RL process, which then generates modified training slices. This feedback loop simplifies the modification process by automatically adjusting data based on proven performance criteria.
Data Source
AI summary
A computer-implemented process for modifying a training dataset includes the following operations. The training dataset is benchmarked using a State Of The Art (SOTA) neural network to determine a benchmark for the training dataset. The training set is divided into a plurality of slices. A sequence of a plurality of atomic operations are selected using a selection strategy generator operating on one of the plurality of slices. The sequence of the plurality of atomic operations is applied to modify the one of the plurality of slices to generate a revised one of the plurality of slices. Reverse reinforcement learning is performed on the revised one of the plurality of slices using the benchmark and the SOTA neural network. The training dataset is modified by replacing the one of the plurality of slices with the revised one of the plurality of slices to generate a modified training dataset.