Action Selection Policy Generation for Imperfect Information Games
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for solving large or very-large-scale Imperfect Information Games (IIGs) are computationally expensive and inefficient, requiring substantial resources and being impractical for real-time applications due to high computational complexity and memory requirements, especially when handling complex real-world scenarios like autonomous vehicle control and resource allocation.
Innovation Solution
The method involves dividing the game into multiple subtasks, using strategy neural networks (SNNs) and value neural networks (VNNs) to predict action selection policies, where SNNs are trained based on VNNs to reduce computational complexity and memory usage, allowing for efficient computation of strategies in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional methods are used to solve large or very-large-scale Imperfect Information Games, then solution accuracy can be maintained, but computational complexity and memory requirements become prohibitively high
Solution Approach 1:
The patent divides the large or very-large-scale Imperfect Information Game into multiple smaller subgames. Each subgame is solved independently using traditional methods to obtain subgame perfect equilibria, which are then combined to form the overall solution. This segmentation reduces the computational complexity and memory requirements while maintaining solution accuracy for the original large game.
2Measurement precision
If traditional methods are used to solve large or very-large-scale Imperfect Information Games, then solution accuracy can be maintained, but memory requirements become prohibitively high
Solution Approach 1:
The patent segments the large game into smaller subgames, each requiring significantly less memory to solve. By solving subgames independently and combining their solutions, the overall memory requirements are reduced from needing to store the entire game state to storing only subgame states, making the solution feasible for very-large-scale games.
3Reliability
If traditional methods are used to solve large or very-large-scale Imperfect Information Games, then complete strategy coverage can be achieved, but computation time becomes prohibitively long
Solution Approach 1:
The patent divides the game into smaller subgames that can be solved independently and in parallel. This segmentation reduces the computation time from needing to process the entire game tree sequentially to processing smaller subgame trees, while still achieving complete strategy coverage through the combination of subgame perfect equilibria.
Solution Approach 2:
The patent performs preliminary decomposition of the large game into subgames before solving them. This preliminary action of identifying and separating subgames allows for more efficient computation by avoiding redundant calculations across the entire game space, thereby reducing overall computation time while maintaining strategy completeness.
Data Source
AI summary
Computer-implemented methods, systems, and apparatus, including computer-readable medium, for generating an action selection policy for causing an execution device to complete a task are described. Data representing a task that is divided into a sequence of subtasks are obtained. Data specifying a strategy neural network (SNN) for a subtask in the sequence of subtasks are obtained. The SNN receives inputs include a sequence of actions that reach an initial state of the subtask, and predicts an action selection policy of the execution device for the subtask. The SNN is trained based on a value neural network (VNN) for a next subtask that follows the subtask in the sequence of subtasks. An input to the SNN is determined. The input includes a sequence of actions that reach a subtask initial state of the subtask. An action selection policy for completing the subtask is determined based on an output of the SNN.