Water Distribution Network Control With Hybrid RL Warm-Start
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing water distribution network optimization systems are expensive, require extensive computing power, and fail to guarantee convergence to an optimal solution in real-time, leading to hesitation in user adoption due to significant differences from traditional control methods.
Innovation Solution
An AI-powered controller using reinforcement learning (RL) with a warm-start strategy, leveraging the GYM framework to formulate the optimization problem as a Markov Decision Process, ensuring real-time interference and incremental introduction of optimal control setpoints, thus addressing complexity and uncertainty.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If prior art optimization systems (clustering/GA/evolution algorithms) are used to find optimal control solutions, then the solution quality may be improved, but the processing time becomes excessively long (over 10 hours) and computing power requirements increase significantly
Solution Approach 1:
The system performs preliminary actions by pre-processing historical data to create training datasets and pre-training the neural network model offline. This allows the optimization to be performed in advance, so that when real-time control decisions are needed, the system can quickly query pre-computed optimal solutions or make rapid predictions based on the trained model, rather than performing lengthy optimization iterations at the moment of decision-making.
Solution Approach 2:
The patent replaces traditional mechanical optimization algorithms (genetic algorithms, evolution algorithms, clustering methods) with an intelligent system based on neural networks and reinforcement learning. This substitution transforms the optimization process from iterative computational searches to pattern recognition and prediction based on learned representations, dramatically reducing computation time while maintaining solution quality.
2Productivity
If prior art systems generate optimization recommendations significantly different from traditional control methods, then optimization benefits may be achieved, but user adoption is delayed or rejected due to hesitancy
Solution Approach 1:
The system applies local quality by making targeted, localized adjustments to control parameters rather than completely overhauling the control strategy. The neural network identifies specific aspects of control that can be improved while maintaining familiar patterns in other areas, and the incremental introduction of changes allows users to adapt gradually. This localized optimization approach maintains solution benefits while reducing the perceived disruption to users.
Solution Approach 2:
The patent implements partial action by introducing optimization recommendations incrementally rather than all at once. The system can start with conservative adjustments and gradually increase the degree of optimization as users become comfortable with the results. This staged approach allows users to see benefits while maintaining confidence in the system, reducing resistance to adoption.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
A method and system control a water distribution network. A database is maintained of prior states based on a residential water demand, a tank level, and an energy tariff. A current state of the water distribution network is determined. Rewards are determined and include a tank level constraint, an energy cost, and a toggle count. A query based model is used to determine a set of control points used to control a first prior state. An RL agent is trained based on the prior states and rewards. The RL agent determines a control setpoint (that changes the pump speed) that maintains the tank level, minimizes the energy cost, and complies with the toggle count. The RL agent determines time slots and selects one of the time slots. Hybrid setpoints are generated to control the water distribution network within the selected time slot.