Transformer Reinforcement Learning Text Conversion for Sequential Decisions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional reinforcement learning algorithms using Transformer networks suffer from low-accuracy prediction results and lack of rationality and interpretability due to unstable training and irrational models.
Innovation Solution
Preprocess historical trajectory data to include state, action, and reward information, and transform reinforcement learning into a language conversion task using a Transformer network model with an input/output encoding module, position encoding, network mask, and encoder/decoder modules to predict action information and maximize target rewards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional reinforcement learning algorithms use Transformer networks for sequential decision-making, then the model can process reinforcement learning data, but the training becomes unstable and prediction accuracy decreases
Solution Approach 1:
The patent introduces an intermediary text conversion mechanism that transforms reinforcement learning trajectories into natural language sequences. This intermediary representation allows the Transformer network to process RL data effectively while maintaining stable training and improving prediction accuracy, as the language conversion provides a structured intermediate layer between the raw RL data and the network processing
Solution Approach 2:
The patent changes the parameter representation by converting discrete reinforcement learning states and actions into natural language tokens. This parameter transformation enables the Transformer network to better capture sequential dependencies and improve prediction accuracy while maintaining adaptability to different RL environments
2Productivity
If traditional reinforcement learning algorithms use Transformer networks, then data can be processed into training rules, but the trained model becomes irrational and uninterpretable
Solution Approach 1:
The text conversion mechanism serves as an intermediary that preserves interpretability by representing reinforcement learning decisions in natural language. This allows the model to maintain rationality and interpretability while efficiently processing data, as the language representation retains the semantic meaning of states, actions, and rewards in a human-understandable format
Data Source
AI summary
Provided are a reinforcement learning method and system based on sequential decision-making, a device, and a medium. The method includes: preprocessing historical trajectory data of reinforcement learning to generate preprocessed historical trajectory data to train a Transformer network model, and transforming the reinforcement learning into a language conversion model task by using a text conversion mechanism in the Transformer network model, to generate a trained Transformer network model, where the Transformer network model is used to predict action information at a next time point in a historical environment, and determine a maximum target reward value in a historical environmental state to obtain a complete trajectory in the historical environmental state; and predicting action information at the next time point in a real environmental state by using the trained Transformer network model, to obtain a complete trajectory in the real environmental state.