Control Model State Reduction for Constrained Action Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Control systems for complex application systems, such as wastewater treatment plants, face challenges in processing large numbers of system states and control actions, leading to computationally intensive constrained mathematical models that exceed processing power and sensor capabilities, making it difficult to optimize operations efficiently while adhering to constraints.
Innovation Solution
The method involves using constrained reinforcement learning (CRL) and constrained Markov decision process (CMDP) techniques to automatically reduce the state space dimensionality from thousands of variables to fewer than 20, simplifying the models and allowing for efficient optimization by formulating them as linear programming problems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If constrained reinforcement learning models are used to optimize control systems for complex application systems, then operational optimization is improved, but computational complexity increases beyond processing capabilities
Solution Approach 1:
The patent extracts and removes unnecessary state variables from the mathematical model, retaining only the essential subset that drives control decisions. This extraction process reduces the state space from thousands of variables to a manageable subset, making the constrained reinforcement learning model computationally tractable while preserving optimization capability.
Solution Approach 2:
The patent creates a simplified copy or representation of the full system model by using surrogate models that approximate the complex constrained reinforcement learning model. These surrogate models capture the essential input-output relationships without requiring full computational complexity, enabling efficient deployment on limited hardware.
2Productivity
If the state space dimensionality is reduced from thousands of variables to fewer than 20, then processing efficiency is improved, but model accuracy may be compromised
Solution Approach 1:
The patent applies local quality by making different parts of the model have different levels of detail. The essential state variables that drive control decisions are represented with high fidelity, while less critical variables are aggregated or omitted. This selective representation maintains accuracy for decision-critical aspects while reducing overall complexity.
Solution Approach 2:
The patent performs preliminary analysis to identify and select the most influential state variables before building the final model. By pre-screening variables based on their impact on control decisions, the system ensures that the reduced state space retains the essential information needed for accurate optimization while eliminating redundant variables.
Data Source
AI summary
A method for automatically reducing the dimensionality of a mathematical representation of a controlled application system is provided. The method includes receiving, at a control system, data corresponding to control action and system state variables relating to the controlled application system, fitting a constrained reinforcement learning (CRL) model to the controlled application system based on the data, and automatically identifying a subset of the system state variables by selecting control action variables of interest and identifying system state variables that drive the CRL model to recommend each control action variable of interest. The method also includes automatically performing state space dimensionality reduction of the CRL model using the subset of system state variables, estimating a transition probability matrix for a constrained Markov decision process (CMDP) model of the controlled application system, and formulating the CMDP model as a linear programming (LP) problem using the transition probability matrix and several costs.