Depth-Dependent Sampling for Imperfect Information Games
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for solving Imperfect Information Games (IIG) face challenges in reducing computational complexity and variance, while also improving convergence speed, especially in large games with limited memory.
Innovation Solution
The proposed solution involves implementing sampling schemes such as robust sampling and depth-dependent sampling for the Counterfactual Regret Minimization (CFR) algorithm, as well as using neural networks to perform CFR, which reduces memory requirements and enhances convergence speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional Counterfactual Regret Minimization (CFR) algorithm is used to solve Imperfect Information Games, then convergence to Nash equilibrium is achieved, but computational complexity and memory requirements become prohibitive for large games
Solution Approach 1:
The patent segments the game tree into different depth levels and applies different sampling strategies to each level. Depth-dependent sampling divides the computation into manageable segments where different sampling rates are used at different depths, reducing overall computational complexity while maintaining convergence properties.
Solution Approach 2:
The patent employs partial action by using robust sampling that selectively samples only certain actions at certain information sets rather than evaluating all actions everywhere. This partial evaluation reduces computational load while still providing sufficient information for convergence to Nash equilibrium.
2Quantity of substance
If outcome sampling is used in MCCFR algorithm, then memory efficiency is improved, but variance of the algorithm increases
Solution Approach 1:
The patent changes the sampling parameters by using depth-dependent sampling probabilities that vary by game state depth. This parameter adjustment allows the algorithm to maintain lower variance in critical regions while preserving memory efficiency through selective sampling, effectively balancing both concerns.
3Measurement precision
If external sampling is used in MCCFR algorithm, then variance is reduced, but memory requirements increase
Solution Approach 1:
The patent applies local quality by using different sampling strategies at different locations (depths) in the game tree. At shallower depths where memory is less constrained, external sampling reduces variance. At deeper depths where memory is more constrained, robust sampling maintains memory efficiency. This localized adaptation resolves the contradiction between variance reduction and memory efficiency.
4Ease of operation
If uniform sampling is applied at all states, then implementation simplicity is maintained, but convergence speed decreases in large games
Solution Approach 1:
The patent introduces dynamics by making the sampling probability adaptive rather than static. The depth-dependent sampling probability changes dynamically based on the depth of the game state, allowing the algorithm to focus computational resources where they are most needed for convergence while maintaining relatively simple implementation through a systematic probability formula.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed herein are methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing counterfactual regret minimization (CRF) for strategy searching in strategic interaction between parties. One of the methods includes: identifying N1 possible actions of a first party in a first state of the first party; sampling a possible action out of the N1 possible actions in the first state of the first party with a first sampling probability; identifying N2 possible actions of the first party in a second state of the first party, wherein the first state of the first party is closer to a beginning state of the IIG than the second state of the first party; sampling a possible action out of the N2 possible actions in the second state of the first party with a second sampling probability, wherein the first sampling probability is less than the second sampling probability.