NPC Battle Decision-Making Using LSTM and Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The determination of battle decisions for combat NPCs in games using behavior trees is labor-intensive and inefficient, requiring the establishment of numerous trees for various combat situations, leading to high labor costs.
Innovation Solution
A reinforcement-learning-based method that processes current and historical battle state information through a long short-term memory network to determine NPC battle policies, utilizing feature extraction, concatenation, and reinforcement learning to make decisions based on reward values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If behavior trees are used to determine NPC battle decisions, then battle policy control is achieved, but labor costs increase significantly
Solution Approach 1:
The patent replaces the mechanical behavior tree system with a reinforcement learning model. Instead of manually constructing and maintaining complex behavior trees, the system uses an automatic learning model that processes battle state information through neural networks (including LSTM for historical data) to generate battle decisions. This substitution eliminates the need for manual behavior tree configuration while maintaining effective battle policy control.
2Adaptability or versatility
If multiple behavior trees are established for various combat situations, then battle decision coverage is improved, but system complexity increases
Solution Approach 1:
The patent implements a universal reinforcement learning model that handles all battle situations through a single integrated system. The model processes diverse battle states (including different enemy types, player levels, and combat scenarios) through unified neural network architectures with LSTM components for historical context. This single multi-functional model replaces the need for multiple specialized behavior trees, reducing system complexity while maintaining comprehensive battle decision coverage.
3Stability of the object's composition
If behavior trees are used for battle decision determination, then battle policy structure is maintained, but decision-making efficiency decreases
Solution Approach 1:
The patent transitions from static behavior tree structures to dynamic reinforcement learning models. The system continuously processes current battle state information and historical data through neural networks, generating adaptive battle decisions in real-time. The dynamic nature of the learning model allows it to respond flexibly to changing combat conditions without requiring pre-defined structural paths, significantly improving decision-making efficiency while maintaining coherent battle policy through learned patterns.
Data Source
AI summary
This application discloses a reinforcement-learning-based non-player character (NPC) battle decision-making method and a related product. The method includes: obtaining current state information, current relative pose information, and historical battle state information of a target NPC and its opponent; processing the current state information and the current relative pose information, through a long short-term memory network, to obtain a first processing result; and making a decision about a battle policy that is to be taken by the target NPC at a next moment based on the first processing result, context information of skill release of the target NPC, a skill set of the target NPC, and a reward item of a reinforcement learning model. As a result, an NPC battle decision is obtained through the reinforcement learning model, without establishing a complex behavior tree, thereby saving labor costs.