NPC Behavior Control Using Reinforcement Learning for RPG Skill Casting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for controlling non-player character (NPC) behavior in multiplayer role-playing games (RPGs) are limited by fixed and singular skill casting actions, making it difficult to match the skill level of human players and requiring extensive time to construct behavior trees for multiple NPCs.
Innovation Solution
A game character behavior control method that uses a reinforcement learning network model to extract feature information from current game state data, construct action probability distributions, and determine target behavior information for NPCs, allowing for intelligent skill casting that adapts to game conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If decision tree method is used to determine NPC behaviors, then behavior control can be implemented, but skill casting actions become fixed and singular, failing to match human player skill level
Solution Approach 1:
The patent replaces the mechanical decision tree system with a deep reinforcement learning system. The neural network model learns optimal skill casting strategies through self-play training, substituting the rigid if-then rules of decision trees with adaptive neural network policies that can generalize to new situations while maintaining reliable behavior control.
Solution Approach 2:
The patent changes the parameter representation from discrete decision tree nodes to continuous neural network output probabilities. The skill casting probabilities are derived from neural network outputs combined with character attribute parameters, allowing flexible adaptation while maintaining control through probability thresholds and action space constraints.
2Adaptability or versatility
If supervised learning method is used to determine NPC behaviors, then behavior control can be implemented, but skill casting actions become fixed and singular, failing to match human player skill level
Solution Approach 1:
The patent replaces supervised learning with deep reinforcement learning through self-play training. Instead of learning from labeled human player data, the system trains agents to play against each other, automatically discovering effective skill casting strategies without requiring extensive real game data annotation.
Solution Approach 2:
The patent implements self-service through self-play training where NPC agents train themselves by playing against each other. The system generates its own training data through automated gameplay, eliminating the need for external human player data collection and annotation while achieving adaptive skill casting behavior.
3Ease of operation
If behavior trees are constructed for each game character, then individual character control is achieved, but construction time becomes extremely long when there are many characters
Solution Approach 1:
The patent creates a universal deep reinforcement learning training framework that can control any number of game characters simultaneously. Instead of constructing individual behavior trees for each character, a single multi-agent training system learns policies for all characters, reducing construction time from linear to constant complexity while maintaining individual character control capabilities.
Solution Approach 2:
The patent merges the control of multiple game characters into a single multi-agent reinforcement learning system. Multiple NPC agents are trained together in a unified training environment, combining what would have been separate behavior tree constructions into one efficient training process that scales to large numbers of characters.
4Reliability
If extensive real game data is collected for training, then NPC intelligence can be improved, but data collection time and storage requirements increase significantly
Solution Approach 1:
The patent eliminates external data collection by implementing self-play training where NPC agents generate their own training data through automated gameplay. The system trains itself by having agents play against each other, producing training data on-demand without requiring human players to generate extensive game logs or datasets.
Solution Approach 2:
The patent performs preliminary action by pre-training NPC agents through extensive self-play training before deployment. The agents learn optimal strategies in advance through automated training, so when deployed they immediately exhibit high intelligence without requiring additional real-game data collection during actual gameplay.
Data Source
AI summary
The present disclosure provides a game character behavior control method. The method includes obtaining current game state information of a game, and performing feature extraction on the current game state information to obtain first feature information that reflects parameters of game characters of opposing parties in the game; constructing second feature information associated with a target game character, the second feature information reflecting the first feature information and features of target state information associated with target game character in the current game state information, the target game character being game characters controlled by game logic; performing classification on the second feature information to obtain action probability distribution associated with the target game characters; determining target behavior information based on an action probability distribution and target state information associated with the target game character; and controlling the target game character to perform actions corresponding to the target behavior information.