Well track control method

Abstract

The invention discloses a well track control method, and relates to the technical field of oil and gas resource development. Comprising the following steps: selecting a characteristic value from measurement-while-drilling LWD data; predicting and identifying the characteristic values through a BP neural network to obtain upper boundary line data and lower boundary line data, and drawing a stratum profile map; constructing a drilling environment according to the stratigraphic profile map; determining an action award function according to the distance between the current position of the drill bit and the next target spot and the deviation angle; performing interactive training by adopting a deep Q network algorithm DQN to obtain a trained neural network model; in the process that the drill bit reaches the well bottom from the drilling point, the total value of the action reward function obtained by the trained neural network model is made to be maximum by continuously selecting control actions, the optimal drilling path of the drill bit is obtained, and the optimal drilling path of the drill bit serves as the optimal well track. Dynamic interaction of the drill bit and the drilling environment can be achieved, and the well track is controlled in real time to extend along the target reservoir to drill all target spots.

Description

Technical Field

[0001] The invention relates to the technical field of oil and gas resource development, and in particular to a wellbore trajectory control method. Background Art

[0002] The development direction of oil and gas resources has gradually shifted from conventional to unconventional. The large-scale and sustainable development of unconventional oil and gas is of great strategic significance for alleviating the contradiction between oil and gas supply and demand and ensuring energy security. Domestic unconventional oil and gas resources have shown great development potential, and the production of shale oil and gas, tight oil and gas, etc. has achieved rapid growth. Compared with conventional oil and gas resources, unconventional oil and gas resources have poor reservoir conditions and great development difficulties. The development of ultra-long horizontal sections (length greater than 3000m) is one of the key technical means to improve the single well production and development benefits of unconventional oil and gas. However, the lithofacies of unconventional oil and gas are highly heterogeneous and have great differences. The wellbore trajectory of ultra-long horizontal sections is frequently adjusted, the drilling risk is high, the wellbore quality is poor, and the precise control of the wellbore trajectory is difficult, which greatly affects the production capacity and economic benefits of unconventional oil and gas. Therefore, optimizing the wellbore trajectory design and accurately controlling the wellbore trajectory are of great significance to ensure the successful drilling of ultra-long horizontal sections, maximize the reservoir drilling rate, and reduce costs and increase efficiency.

[0003] Traditional spatial geometric mathematical models and modern optimization theory are the main methods for wellbore trajectory optimization design. The optimization objectives have evolved from single-objective optimization such as trajectory length, friction torque, etc. to multi-objective optimization such as minimum trajectory length, minimum target error, and minimum drilling cost. The optimization methods have evolved from conventional algorithms such as chord-step method and fox-step method to intelligent optimization algorithms such as genetic algorithm and particle swarm algorithm. However, multi-objective optimization still cannot meet the wellbore trajectory design requirements under all complex geological conditions. In addition, the performance of high-dimensional multi-objective optimization algorithms is reduced and the diversity of solutions is not ideal, which limits their engineering applications.

[0004] Geosteering technology is the most advanced wellbore trajectory control method. Geosteering technology is a semi-closed loop underground. Drilling decisions require human experts at the ground control center to analyze and process the transmitted real-time downhole data, and then pass the control instructions to the downhole guidance device and directional tools to continuously adjust the wellbore trajectory to the best position through the reservoir during drilling. This interactive underground semi-closed loop method is highly dependent on the transmission speed and efficiency of signals. Due to the harsh deep and complex unconventional oil and gas geological environment and insufficient understanding, accurate layer determination is difficult, and signal transmission quality is poor, resulting in the difficulty in ensuring the effectiveness of the drilling dual communication mechanism, and the lag in human expert decision-making. Summary of the invention

[0005] Based on this, it is necessary to provide a wellbore trajectory control method to address the above technical issues.

[0006] An embodiment of the present invention provides a wellbore trajectory control method, comprising: Select characteristic values ​​representing formation characteristics from LWD data; The eigenvalues ​​are predicted and identified through BP neural network to obtain the upper boundary line data and lower boundary line data of the target layer, and the stratigraphic profile of the target layer is drawn; According to the stratigraphic profile of the target layer, a drilling environment including a two-dimensional stratigraphic profile simulation diagram and a target reservoir boundary collision object is constructed in the Pygame window; the drill bit action is controlled in the two-dimensional stratigraphic profile simulation diagram, and the drill bit is prevented from encountering the target reservoir boundary collision object during the control process, and the action reward function is determined according to the current position of the drill bit; According to the drilling environment and action reward function, the deep Q network algorithm DQN is used for interactive training to obtain a trained neural network model. The trained neural network model is used to predict the action reward function obtained by the drill under different control actions. In the process of the drill bit reaching the bottom of the well from the drilling point, the total value of the action reward function obtained by the trained neural network model is maximized by continuously selecting control actions, and the optimal drilling path of the drill bit is obtained, and the optimal drilling path of the drill bit is used as the optimal wellbore trajectory.

[0007] Optionally, the LWD data measured while drilling include: natural gamma data, acoustic wave time difference data and ACS logging curve; Natural gamma ray data, acoustic time difference data and ACS logging curves are used as characteristic values ​​for formation lithology identification. The stratigraphic sequence is identified through BP neural network. The stratigraphic interface line is then smoothed to obtain the upper and lower boundary line data of the target layer and draw the stratigraphic profile.

[0008] Optionally, controlling the drill bit action in the two-dimensional stratigraphic profile simulation diagram specifically includes: Set the drill action flag to move; When move is True, it means the drill moves; if α γ = 0, the control amount γ of the drill direction decreases; if α γ =1, the control amount γ of the drill direction increases; where α γ Control sub-actions for the drill; When move is False, it means that the drill does not move and the control amount γ of the drill direction remains unchanged; The value range of the control amount γ of the drill direction is [0, γ max ], γ max =45°.

[0009] Optionally, the action reward function is determined according to the current position of the drill head, which specifically includes: The upper boundary line data and the lower boundary line data of the target layer are sampled at intervals, and the average values ​​of the horizontal and vertical coordinates of the upper and lower sampling points are taken as the target point coordinates; Through the collection S x Indicates the state, S x = {S0,S1,S2,…,S n} Where S0 is the drill bit entry position, S n The bottom of the well indicates the termination state, and the drill bit reaches S n The action ends after the state; the reward function is designed based on the value of the state quantity. The action reward function of the drill is determined by the distance between the current position and the next target point and the offset angle, and its expression is: ; in, R ( d,θ ) is the immediate reward for the drill action, R 1 is the reward value for drilling a single target point, R all The reward value for drilling all the targets. R m is the reward value for drilling to the bottom of the well, R d is the reward value for drilling out the formation, d is the offset distance, θ is the offset angle.

[0010] Optionally, when the drill bit moves in the drilling environment, if a single target point is encountered, the reward is set to the reward value of the single target point. R 1. If all targets are encountered, the reward is set to the reward value of all targets encountered. R all ; If the target point is not encountered and the offset d If the value is greater than the preset value, it means it is far away from the target and a negative reward is set; if the value is less than the preset value, it means it is close to the target and a positive reward is set; Set a minimum threshold, and the action ends when the reward obtained by the drill is less than the minimum threshold.

[0011] Optionally, the offset d is the distance from the drill bit to the line connecting the current target point and the next target point. θ is the angle between the parallel line c connecting the current target point and the next target point and the horizontal x-axis; Being close to the target will give you a positive reward, being far away from the target will give you a negative reward, and the instant reward of the drill action R ( d,θ ) limits the movement of the drill head toward the target point.

[0012] Compared with the prior art, the wellbore trajectory control method provided by the embodiment of the present invention has the following beneficial effects: The present invention first constructs a drilling environment. In the process of the drill bit reaching the bottom of the well from the drilling point, by continuously selecting control actions, the drill bit and the drilling environment can be dynamically interacted with, and the wellbore trajectory can be controlled in real time to extend along the target reservoir to drill all target points, thereby solving the problems in the prior art of the difficulty in ensuring the effectiveness of the dual communication mechanism in a complex geological environment and the lag in human expert decision-making.

[0013] In addition, in the constructed drilling environment, the present invention controls the action of the drill bit to maximize the total value of the action reward function obtained by the trained neural network model, which means maximizing the drilling rate of high-quality reservoirs. The obtained optimal drilling path of the drill bit is the designed optimal wellbore trajectory, thereby realizing the integration of drill bit trajectory control and wellbore trajectory design.

[0014] In addition, the present invention can accurately predict the horizontal well target area in real time and accurately characterize the drill bit's target penetration depth, well inclination and azimuth. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic flow chart of a wellbore trajectory control method provided in an embodiment; Figure 2 A schematic diagram of the interaction between an intelligent agent (drill bit) and a drilling environment in a wellbore trajectory control method provided in an embodiment; Figure 3 A Markov decision process state transition diagram of a wellbore trajectory control method provided in one embodiment; Figure 4 A schematic diagram of the action direction of a deflection tool in a wellbore trajectory control method provided in one embodiment; Figure 5 A schematic diagram of a drilling position of a drill bit in a wellbore trajectory control method provided in an embodiment; Figure 6 A wellbore trajectory diagram drawn at the initial stage of training for a wellbore trajectory control method provided in an embodiment; Figure 7 A wellbore trajectory diagram drawn in the later stage of training of a wellbore trajectory control method provided in an embodiment. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0017] In one embodiment, a wellbore trajectory control method is provided, the method comprising: The characteristic values ​​representing the formation characteristics are selected from the Logging While Drilling (LWD) data.

[0018] The eigenvalues ​​are predicted and identified through the BP neural network to obtain the upper boundary line data and the lower boundary line data of the target layer, and the stratigraphic profile of the target layer is drawn.

[0019] According to the stratigraphic profile of the target layer, a drilling environment including a two-dimensional stratigraphic profile simulation diagram and a target reservoir boundary collision object is constructed in the Pygame window. The drill bit action is controlled in the two-dimensional stratigraphic profile simulation diagram to avoid the encounter between the drill bit and the target reservoir boundary collision object during the control process, and the action reward function is determined according to the current position of the drill bit.

[0020] According to the drilling environment and action reward function, the deep Q network algorithm DQN is used for interactive training to obtain a trained neural network model. The trained neural network model is used to predict the action reward function obtained by the drill under different control actions.

[0021] In the process of the drill bit reaching the bottom of the well from the drilling point, the total value of the action reward function obtained by the trained neural network model is maximized by continuously selecting control actions, and the optimal drilling path of the drill bit is obtained, and the optimal drilling path of the drill bit is used as the optimal wellbore trajectory.

[0022] The specific implementation process includes: Step 1: Predict stratigraphic profile data: According to the knowledge related to logging interpretation, data that can characterize the formation characteristics are selected as characteristic values ​​in the LWD data. LWD data refers to the measurement of formation rock physical parameters during the drilling process, and the measured formation rock physical parameters are sent to the ground in real time for processing using a data telemetry system.

[0023] The BP neural network is used to predict and identify the upper and lower boundary line data of the target layer. The natural gamma data, acoustic time difference data and ACS logging curve (Acoustic Logging curve) are used as characteristic values ​​for stratum lithology identification. The stratigraphic sequence is identified through the BP neural network, and the stratigraphic interface line is smoothed to obtain the upper and lower boundary line data of the target layer, and the stratigraphic profile is drawn.

[0024] Step 2: Build the drilling environment: The purpose of building a drilling environment is to interact with the intelligent agent (drill bit) to train and test the network model. The predicted stratigraphic profile data, the upper boundary line data of the target oil layer, and the lower boundary line data of the target oil layer are required to build the drilling environment.

[0025] According to the stratigraphic profile data predicted in step 1, a two-dimensional stratigraphic profile simulation diagram is generated in the Pygame window, and a target reservoir boundary collision object is generated. The target point coordinates are obtained by sampling the upper and lower boundary data of the reservoir at intervals, and the average values ​​of the horizontal and vertical coordinates of the upper and lower sampling points are taken as the target point coordinates.

[0026] Step 3: Define the state of the method and design the reward function: The state set S x Indicates that S x = {S0,S1,S2,…,S n} Where S0 is the drill bit entry position, S n The position of the bottom of the well indicates the termination state, that is, the drill bit reaches S n The action ends after the state; the reward function is designed according to the value of the state quantity, and the action reward function of the drill is determined by the distance between the current position and the next target and the offset angle.

[0027] Step 4: Based on the environment, state and reward function in steps 2 and 3, the deep Q network algorithm (DQN) is used for interactive training.

[0028] Step 5. Save the network model trained in step 4, load the model, give the initial parameters as input, select actions in the prediction network, get the state at the next moment, and finally reach the bottom of the well after continuously selecting actions. The wellbore trajectory results are plotted in the window.

[0029] In step 3, the “drill action” refers to the drill control action. γ is a preset value as the control amount of the drill direction. Due to the limitation of the deflection tool, the value range of γ is [0, γ max ], γ max =45°. When move is True, it means the drill moves; if α γ = 0, the control amount γ of the drill direction decreases; if α γ = 1, the control amount γ of the drill direction increases. Among them, α γ It is the drill control sub-action. When move is False, it means that the drill does not move and the control amount γ of the drill direction remains unchanged.

[0030] Among them, the "reward" mentioned in step three is divided into single-step reward and cumulative reward. The cumulative reward is also called return, and the reward is given by a designed reward function.

[0031] The upper boundary line data and the lower boundary line data of the target layer are sampled at intervals, and the average values ​​of the horizontal and vertical coordinates of the upper and lower sampling points are taken as the target point coordinates; Through the collection S x Indicates the state, S x = {S0,S1,S2,…,Sn} Where S0 is the drill bit entry position, S n The bottom of the well indicates the termination state, and the drill bit reaches S n The action ends after the state; the reward function is designed based on the value of the state quantity. The action reward function of the drill is determined by the distance between the current position and the next target point and the offset angle, and its expression is: ; in, R ( d,θ ) is the immediate reward for the drill action, R 1 is the reward value for drilling a single target point, R all The reward value for drilling all the targets. R m is the reward value for drilling to the bottom of the well, R d is the reward value for drilling out the formation, d is the offset distance, θ is the offset angle.

[0032] When the drill moves in the drilling environment, if it encounters a single target, the reward is set to the reward value of the single target. R 1. If all targets are encountered, the reward is set to the reward value of all targets encountered R all .

[0033] If the target point is not encountered and the offset d If it is greater than the preset value, it means it is far away from the target and a negative reward is set; if it is less than the preset value, it means it is close to the target and a positive reward is set.

[0034] Set a minimum threshold, and the action ends when the reward obtained by the drill is less than the minimum threshold.

[0035] Among them, the "Deep Q Network Algorithm (DQN)" described in step 4 is a classic deep reinforcement learning algorithm based on value function, which is a well-known technology.

[0036] Figure 2 Schematic diagram of the interaction between the drill bit and the drilling environment in the method of the present invention. The drilling environment is constructed by using the target stratum sequence data obtained by predicting the LWD data while drilling that can characterize the stratum through the BP neural network. The process of the interaction between the drill bit and the drilling environment belongs to the Markov decision process. The specific process of the interaction between the intelligent agent and the environment is as follows: At the current moment, the drill bit obtains the current state from the drilling environment, performs actions according to the greedy strategy, and returns the reward and the state of the next moment. The drill bit learns by constantly interacting with the drilling environment and is constantly updated during the learning process. After a lot of training and learning, the optimal solution to the problem is the optimal wellbore trajectory.

[0037] Figure 3 It is the state transition diagram of the Markov decision process during drilling. The state transition diagram contains all the states of the drill bit action. Among them, the drill bit entering the layer is the initial state, and the action is completed as the terminal state. The state is represented by the set S x Indicates that S x ={S0,S1,S2,…,S n} is used to describe the location of the drill bit. In a random environment, after the drill bit performs an action, there are multiple possibilities for reaching the next state. Six target points are set, and each target point has two situations: drilled and not drilled. The action ends when the drill bit drills the bottom of the well or drills out of the target layer and the current reward is less than the minimum reward threshold. The reward is the largest when the drill bit drills all the target points and successfully drills the bottom of the well, which is the optimal solution.

[0038] Figure 4 It is a schematic diagram of the direction of action of the deflection tool. The control amount of the drill bit direction is γ. Due to the limitation of the deflection tool, the value range of γ is [0, γ max ], γ =45°. The method of the present invention only considers the two-dimensional situation. From time t0 to time t1, the direction of action of the deflection tool changes from γ0 to γ1, and the drill bit drills in the two-dimensional formation space, which conforms to the characteristic of continuous change of the direction of the deflection tool. The action of controlling the action of the drill bit is directly set to adjust the control amount γ of the drill bit direction. The adjustment amount of γ each time is a preset value. The size of γ can be adjusted according to the actual situation, thereby adjusting the smoothness of the curve.

[0039] Figure 5 This is a schematic diagram of the drill bit drilling position. The figure only shows two target points, while the actual situation includes more target points. d The distance from the drill bit to the line connecting target 1 and target 2 is called the offset distance. θ The angle between the horizontal x-axis and the parallel line c is called the offset angle. c is the parallel line connecting target 1 and target 2. If you are close to the target, you will get a positive reward value, and if you are far away from the target, you will get a negative reward value. The instant reward of the drill action R ( d,θ ) limits the movement of the drill head toward the target point.

[0040] Figure 6 is the wellbore trajectory map drawn at the beginning of training. Figure 7 It is the wellbore trajectory map drawn in the later stage of training.

[0041] For example, a simulation case is provided: The simulation uses the deep learning framework Pytorch, and builds the drilling environment based on the logging data while drilling. Set the initial entry position and the initial entry well inclination to 80°, set 6 targets, and the last point represents the bottom of the well. The DQN algorithm is used to interactively train the agent and the drilling environment. The number of episodes is preset to 10,000. When the final value of the greedy strategy drops to 0.1, the training ends. At this time, the maximum number of episodes for training is 8,798. The initial training results are as follows: Figure 6 As shown in the figure, there are problems such as the drill bit going out of the layer and not encountering the target. This is because the agent is trained too few times and cannot learn from the drilling environment. The results of the later stage of training are as follows Figure 7 As shown, the wellbore trajectory is basically maintained in the best position of the reservoir and can drill most of the target points.

[0042] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A wellbore trajectory control method, characterized in that: include: Select characteristic values ​​representing formation characteristics from LWD data; The eigenvalues ​​are predicted and identified through BP neural network to obtain the upper boundary line data and lower boundary line data of the target layer, and the stratigraphic profile of the target layer is drawn; According to the stratigraphic section of the target layer, a drilling environment including a two-dimensional stratigraphic section simulation diagram and a target reservoir boundary collision object is constructed in the Pygame window; Control the drill bit in the two-dimensional stratigraphic profile simulation diagram to avoid the drill bit from colliding with the target reservoir boundary object during the control process, and determine the action reward function based on the current position of the drill bit; According to the drilling environment and the action reward function, a deep Q network algorithm DQN is used to perform interactive training to obtain a trained neural network model, and the trained neural network model is used to predict the action reward function obtained by the drill under different control actions; In the process of the drill bit reaching the bottom of the well from the drilling point, the total value of the action reward function obtained by the trained neural network model is maximized by continuously selecting control actions, and the optimal drilling path of the drill bit is obtained, and the optimal drilling path of the drill bit is used as the optimal wellbore trajectory.

2. A wellbore trajectory control method according to claim 1, characterized in that: The LWD data includes natural gamma data, acoustic time difference data and ACS logging curves; Natural gamma ray data, acoustic time difference data and ACS logging curves are used as characteristic values ​​for formation lithology identification. The stratigraphic sequence is identified through BP neural network. The stratigraphic interface line is then smoothed to obtain the upper and lower boundary line data of the target layer and draw the stratigraphic profile.

3. A wellbore trajectory control method according to claim 1, characterized in that: The controlling of the drill bit action in the two-dimensional stratigraphic profile simulation diagram specifically includes: Set the drill action flag to move; When move is True, it means the drill moves; if α γ = 0, the control amount γ of the drill direction decreases; if α γ = 1, the control amount γ of the drill direction increases; where α γ Control sub-actions for the drill; When move is False, it means that the drill does not move and the control amount γ of the drill direction remains unchanged; The value range of the control amount γ of the drill direction is [0, γ max ], γ max =45°.

4. A wellbore trajectory control method according to claim 1, characterized in that: The action reward function is determined according to the current position of the drill bit, which specifically includes: The upper boundary line data and the lower boundary line data of the target layer are sampled at intervals, and the average values ​​of the horizontal and vertical coordinates of the upper and lower sampling points are taken as the target point coordinates; Through the collection S x Indicates the state, S x = {S0,S1,S2,…,S n } Where S0 is the drill bit entry position, S n The bottom of the well indicates the termination state, and the drill bit reaches S n The action ends after the state; the reward function is designed based on the value of the state quantity. The action reward function of the drill is determined by the distance between the current position and the next target point and the offset angle, and its expression is: ； in, R ( d,θ ) is the immediate reward for the drill action, R 1 is the reward value for drilling a single target point, R all The reward value for drilling all the targets. R m is the reward value for drilling to the bottom of the well, R d is the reward value for drilling out the formation, d is the offset distance, θ is the offset angle.

5. A wellbore trajectory control method according to claim 4, characterized in that: When the drill moves in the drilling environment, if it encounters a single target, the reward is set to the reward value of the single target. R 1. If all targets are encountered, the reward is set to the reward value of all targets encountered. R all ; If the target point is not encountered and the offset distance d If it is greater than the preset value, a negative reward is set for being far away from the target; If it is less than the preset value, it means it is close to the target and a positive reward is set; Set a minimum threshold, and the action ends when the reward obtained by the drill is less than the minimum threshold.

6. A wellbore trajectory control method according to claim 4, characterized in that: The offset d is the distance from the drill bit to the line connecting the current target point and the next target point. θ is the angle between the parallel line c connecting the current target point and the next target point and the horizontal x-axis; Being close to the target will give you a positive reward, being far away from the target will give you a negative reward, and the instant reward of the drill action R ( d,θ ) limits the movement of the drill head toward the target point.

Images