Game AI Policy Training Using Virtual Physiological Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video game AI agents often exhibit behavior that is not human-like, reducing realism and player immersion, and current reinforcement learning methods are computationally inefficient.
Innovation Solution
A data processing apparatus and method that combines imitation learning with reinforcement learning, using physiological data from human players to train AI agents, constraining actions based on human-like behavior patterns and reducing computational burden.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If reinforcement learning is used to train AI agents, then the agents can complete tasks effectively and autonomously, but the computational requirements become excessively high and the behavior becomes less human-like
Solution Approach 1:
The patent introduces an intermediary mechanism that captures human player behavior patterns through physiological data (heart rate, pupil dilation, skin conductance) and uses this as a mediator to guide the reinforcement learning process. This intermediary human behavior model constrains the RL training to generate more human-like actions while maintaining autonomous task completion capability.
Solution Approach 2:
The patent changes the training parameters by incorporating physiological data dimensions (heart rate, pupil dilation, skin conductance) alongside traditional game state observations. This parameter expansion allows the RL agent to learn human-like behavioral patterns without requiring proportional increases in computational resources, as the physiological constraints naturally guide the action selection.
2Extent of automation
If reinforcement learning is used to train AI agents, then the agents can autonomously perform tasks, but the behavior becomes less human-like and realistic
Solution Approach 1:
The patent uses physiological data as an intermediary to bridge the gap between autonomous RL training and human-like behavior. The physiological signals serve as a mediator that translates human emotional and cognitive states into actionable constraints for the RL agent, ensuring the learned behavior reflects human patterns while maintaining automation.
Solution Approach 2:
The patent implements feedback mechanisms where physiological data from human players is continuously fed back into the RL training process. This feedback loop allows the agent to adjust its action selection based on observed human responses, gradually converging on more human-like behavior patterns while preserving autonomous task completion.
3Reliability
If manual policy creation is used, then the agent behavior can be precisely controlled to match human-like patterns, but the development time and labor requirements become excessive
Solution Approach 1:
The patent applies copying by capturing human player behavior patterns through physiological data and replicating these patterns in the RL training process. Instead of manually crafting policies, the system copies human response patterns to game stimuli and uses them to train autonomous agents, significantly reducing development time while maintaining human-like behavior.
Solution Approach 2:
The patent performs preliminary action by collecting and analyzing physiological data from human players before the actual RL training begins. This preliminary phase captures human behavior patterns and prepares training datasets that encode these patterns, allowing the RL agent to learn human-like behavior directly without requiring manual policy creation during development.
Data Source
AI summary
A data processing apparatus comprising circuitry configured to: execute a machine learning, ML, model configured to receive, as an input, a game state of a video game and first virtual physiological data indicative of a first virtual physiological state of an agent of the video game, and generate, as an output, a probability of each of a plurality of actions of the agent and second virtual physiological data indicative of a second, subsequent, virtual physiological state of the agent; and perform reinforcement learning to generate a policy for completion of a task by the agent, the reinforcement learning comprising, for each of a plurality of attempts at the task by the agent, executing one or more successive iterations of the ML model and, for each attempt, controlling the agent to perform a different respective set of actions based on the output probability of each of the plurality of actions at each of the one or more successive iterations.