Modified UCB Algorithm for Well Placement Planning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Well placement planning in the energy industry is a time-consuming and computationally intensive process, making it inefficient to explore the complete solution space effectively, especially due to the manual nature of the process and the complexity of simulations which can take days, weeks, or even years.
Innovation Solution
Implementing a modified Upper Confidence Bound (UCB) algorithm in an agent-simulator environment that balances exploration and exploitation of well placement sequences based on hydrocarbon recovery and cost, allowing for efficient selection of optimal well placement sequences by iteratively updating reward distributions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual well placement planning is used, then ease of operation is maintained, but productivity is low and time consumption is high
Solution Approach 1:
The patent replaces manual mechanical well placement planning with an automated agent-simulator system that uses reinforcement learning algorithms. The agent autonomously explores the action space, evaluates well placement sequences through simulations, and selects optimal placements without manual intervention, thereby dramatically improving productivity while reducing time consumption from days/weeks to minutes.
2Measurement precision
If complete solution space exploration is performed, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The patent applies partial action by having the agent explore only the necessary portion of the action space through intelligent sampling rather than exhaustive enumeration. The reinforcement learning algorithm efficiently navigates the search space by focusing on promising regions, achieving high measurement precision (optimal well placement identification) without requiring complete solution space exploration, thus reducing simulation time from days/weeks to minutes.
Solution Approach 2:
The agent receives feedback from the simulator in the form of reward signals based on hydrocarbon recovery and cost metrics. This feedback mechanism allows the agent to iteratively improve its well placement decisions by learning from simulation outcomes, achieving high measurement precision through adaptive optimization rather than brute-force exploration of all possible placements.
3Reliability
If multiple well placement sequences are simulated, then reliability is improved, but use of energy increases
Solution Approach 1:
Instead of simulating all possible well placement sequences exhaustively, the reinforcement learning agent performs partial exploration of the action space, selectively evaluating only the most promising placement sequences. This approach maintains reliability by identifying high-confidence optimal placements through intelligent search rather than complete enumeration, significantly reducing computational energy consumption while preserving the reliability needed for confident well placement decisions.
Data Source
AI summary
Various computer-implemented methods for utilizing a modified upper confidence bound (UCB) in an agent-simulator environment in well placement planning for oil fields are disclosed herein. A set of well placement sequences for placing well in a geographical region may be received, where each well placement sequent defines a sequence of multiple oil wells to be placed within the geographical region. A computer-implemented simulation may be executed on each of the well placement sequences to determine, for each of the well placement sequences, a reward based upon a calculated hydrocarbon recovery and a cost of the calculated hydrocarbon recovery. The well placement sequences may be iteratively selected for the computer-implemented simulations using the modified UCB algorithm and based upon the rewards determined for each of the plurality of well placement sequences.