Control Hierarchy Search Using Neural Sub-Policies
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Synthesizing policies for large, complex dynamic systems becomes computationally expensive due to the increasing complexity of state and control input spaces, making it challenging to optimize control hierarchies efficiently.
Innovation Solution
A method involving policy decomposition and neural network-based control sub-policies is used to generate control hierarchies, employing genetic and Monte Carlo algorithms to search and identify optimal control policies for subsystems, which are then assembled to control the entire system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If policy optimization methods are used for large, complex dynamic systems, then control performance can be optimized, but computational cost increases exponentially with system complexity
Solution Approach 1:
The patent decomposes the policy optimization problem into a hierarchy of sub-problems, where the overall policy is segmented into multiple sub-policies operating at different levels. Each sub-policy optimizes a specific aspect of control, breaking down the exponentially complex full-system optimization into manageable components that can be solved independently and then composed together.
Solution Approach 2:
The patent introduces a hierarchical dimension to the control architecture, transforming the flat, monolithic policy optimization into a multi-level structure. This dimensional change allows the system to trade off some direct control authority for reduced computational burden by organizing control decisions across multiple hierarchical levels rather than optimizing all parameters simultaneously.
2Adaptability or versatility
If the state and control input spaces of a dynamic system increase in complexity, then the system can handle more sophisticated tasks, but policy optimization becomes computationally intractable
Solution Approach 1:
The patent segments the high-dimensional state and control input spaces into smaller, manageable subspaces that correspond to different subsystems or control objectives. Each sub-policy operates on a reduced-dimensional subspace, making optimization tractable while collectively covering the full system capability through composition of sub-policies.
Solution Approach 2:
The patent applies partial action by having sub-policies address specific portions of the control task rather than attempting to optimize all control dimensions simultaneously. Each sub-policy focuses on particular state variables or control objectives, and their compositions achieve comprehensive system control through coordinated partial actions.
Data Source
AI summary
Provided is a system, method, and device for searching control hierarchies. The system includes a computing device configured to generate at least one control hierarchy comprising a plurality of subsystems of a system to be controlled based on a plurality of neural network-based control sub-policies, each subsystem corresponding to at least one neural network-based control sub-policy arranged in the control hierarchy, search a plurality of control hierarchies based on a fitness value to identify a control hierarchy, generate a control policy for the system to be controlled comprising of neural network-based sub-policies based on the control hierarchy, and control the system based on the control policy.