Geosteering closed loop control system

Abstract

The present application relates to the technical field of geosteering, in particular to a geosteering closed-loop control system; the system comprises data acquisition, attitude prediction, vibration inference, stress evolution, risk assessment and closed-loop control modules; the system acquires multi-dimensional data, the attitude prediction module outputs clean prediction data set based on geological model and drilling engineering parameters; the vibration inference module infers high-frequency torsional vibration amplitude through recurrent neural network using clean prediction data set, measured data set and historical control action; the stress evolution module quantifies the observable vibration into cumulative stress entropy, and determines real-time asset risk index by the risk assessment module; the closed-loop control module generates updated control constraints based on risk hedging reward function, and feeds back to the attitude prediction module; the system analyzes the deviation between clean prediction data and disturbed measured data, infers implicit high-frequency torsional vibration, and realizes the transition from relying on perfect data to actively identifying high-risk state.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of geosteering, in particular to a geosteering closed-loop control system. BACKGROUND

[0002] In the geosteering drilling environment, the control system needs to realize trajectory tracking and efficient drilling according to multi-dimensional data; however, the inherent delay and data packet loss of well-ground communication cause serious distortion of measured data; at the same time, the downhole drilling tool bears high-frequency torsional vibration which cannot be directly observed on the ground, and this implicit vibration is the main factor causing tool fatigue damage and even catastrophic failure; the existing control scheme cannot accurately infer the amplitude of implicit vibration in distorted data, and also lacks an intelligent decision-making mechanism to actively balance the conflict between drilling efficiency and long-term safety of drilling tool assets under uncertain geological conditions; therefore, how to realize the dynamic balance between drilling targeting and asset risk avoidance under the dual constraints of geological uncertainty and unreliable communication has become a technical problem to be solved. SUMMARY

[0003] To solve the above technical problems, the present application provides a geosteering closed-loop control system, and specifically, the technical scheme of the present application comprises:

[0004] A data acquisition module for acquiring multi-dimensional data in the geosteering drilling process;

[0005] Among them, the multi-dimensional data includes: preset geological model parameters, preset target trajectory parameters, drilling engineering parameters, communication channel parameters, measured data sets and historical control action sequences;

[0006] The drilling engineering parameters include observable drilling tool vibration measurement values;

[0007] An attitude prediction module for outputting a clean prediction data set based on the geological model parameters and the drilling engineering parameters through a state space model of high-dimensional nonlinear model prediction control;

[0008] Among them, the clean prediction data set represents a predicted drilling tool attitude sequence;

[0009] A vibration inference module for inferring the amplitude of high-frequency torsional vibration through a pre-trained recurrent neural network inference model based on the clean prediction data set, the measured data set and the historical control action sequence;

[0010] A stress evolution module for calculating the current cumulative stress entropy through a cumulative stress entropy evolution model based on the amplitude of high-frequency torsional vibration and the observable drilling tool vibration measurement values;

[0011] A risk assessment module for determining a real-time asset risk index according to the current cumulative stress entropy and a preset material mechanics failure threshold;

[0012] wherein the real-time asset risk index represents a safety distance of the drilling tool asset;

[0013] a closed-loop control module configured to calculate a risk hedging reward function through a deep reinforcement learning strategy in response to the real-time asset risk index, and generate an updated control constraint based on the risk hedging reward function;

[0014] wherein the updated control constraint is fed back to the posture prediction module for adjusting boundary conditions of the high-dimensional nonlinear model predictive control.

[0015] Preferably, the posture prediction module is configured to output a clean prediction dataset through a state space model of the high-dimensional nonlinear model predictive control based on the geological model parameters and the drilling engineering parameters, including:

[0016] fitting a geological interaction function based on rock mechanics parameters in the geological model parameters;

[0017] calibrating a state transition matrix according to the preset physical structure and material mechanics properties of the drilling tool;

[0018] calibrating a control input matrix according to the preset physical structure and material mechanics properties of the drilling tool;

[0019] in each control step, calling the state transition matrix, the control input matrix and the geological interaction function to solve the state space model to output the clean prediction dataset.

[0020] Preferably, the vibration inference module is configured to solve the high-frequency torsional vibration amplitude through a pre-trained recurrent neural network inference model based on the clean prediction dataset, the measured dataset and a historical control action sequence, including:

[0021] aligning the measured dataset and the clean prediction dataset;

[0022] learning an asymmetric deviation between the measured dataset and the clean prediction dataset;

[0023] processing the asymmetric deviation, data delay and data packet loss rate represented by the communication channel parameters through the pre-trained recurrent neural network inference model to infer the high-frequency torsional vibration amplitude.

[0024] Preferably, the stress evolution module is configured to calculate a current cumulative stress entropy through a cumulative stress entropy evolution model based on the high-frequency torsional vibration amplitude and the observable drilling tool vibration measurement value, including:

[0025] obtaining a preset stress natural release coefficient, a high-frequency torsional vibration damage contribution weight, an observable vibration damage contribution weight, a fatigue damage index and a vibration reference benchmark value;

[0026] divide the high-frequency torsional vibration amplitude by the corresponding vibration reference benchmark value to obtain a first dimensionless vibration term;

[0027] divide the observable drilling tool vibration measurement value by the corresponding vibration reference benchmark value to obtain a second dimensionless vibration term;

[0028] update and output the current cumulative stress entropy based on the stress natural release coefficient, the high-frequency torsional vibration damage contribution weight, the observable vibration damage contribution weight, the fatigue damage index, the first dimensionless vibration term, and the second dimensionless vibration term.

[0029] Preferably, the risk assessment module is configured to determine a real-time asset risk index according to the current cumulative stress entropy and a preset material mechanics failure threshold, including:

[0030] call the current cumulative stress entropy and the preset material mechanics failure threshold;

[0031] obtain a risk indicator by calculating the ratio of the current cumulative stress entropy to the material mechanics failure threshold;

[0032] define the risk indicator as the real-time asset risk index.

[0033] Preferably, the risk assessment module is further configured to:

[0034] set a preset high-risk steady-state operation baseline and a preset collapse warning threshold;

[0035] compare the real-time asset risk index with the high-risk steady-state operation baseline and the collapse warning threshold;

[0036] when the real-time asset risk index is lower than or equal to the high-risk steady-state operation baseline, determine the current risk state as safe;

[0037] when the real-time asset risk index is greater than the high-risk steady-state operation baseline and lower than or equal to the collapse warning threshold, determine the current risk state as high-risk steady-state;

[0038] when the real-time asset risk index is greater than the collapse warning threshold, determine the current risk state as collapse warning.

[0039] Preferably, the closed-loop control module is configured to calculate a risk hedging reward function through a deep reinforcement learning strategy and generate an updated control constraint based on the risk hedging reward function in response to the real-time asset risk index, including:

[0040] obtain the mechanical drilling speed and the trajectory fit degree deviation from the system state;

[0041] determine an efficiency reward function based on the mechanical drilling speed;

[0042] determine a trajectory reward function based on trajectory adherence deviation;

[0043] determine a non-linear penalty function based on real-time asset risk index;

[0044] call preset balancing weight;

[0045] combine the efficiency reward function, the trajectory reward function and the non-linear penalty function through the balancing weight to construct a risk hedging reward function;

[0046] determine and generate an updated control constraint by maximizing the risk hedging reward function.

[0047] Preferably, the non-linear penalty function is preset as a function whose derivative increases with the increase of the real-time asset risk index, for:

[0048] when the real-time asset risk index is lower than or equal to the collapse warning threshold, the deep reinforcement learning strategy prioritizes the optimization of the efficiency reward function and the trajectory reward function;

[0049] when the real-time asset risk index is greater than the collapse warning threshold, the non-linear penalty function dominates in the risk hedging reward function, forcing the deep reinforcement learning strategy to sacrifice short-term performance indicators to reduce the real-time asset risk index.

[0050] Preferably, the geological model parameters are obtained through logging while drilling and are preset to have uncertainty;

[0051] The drilling engineering parameters are collected from surface and downhole sensors.

[0052] Compared with the prior art, the present application has the following beneficial effects:

[0053] 1. The present system does not rely on perfect measured data, but through analyzing the asymmetric deviation between non-interfered clean predicted data and interfered dirty measured data, infers the high-frequency torsional vibration which is not directly observable on the surface but extremely harmful; then, quantifies this implicit vibration together with observable vibration into cumulative stress entropy and standardized real-time asset risk index, providing a reliable physical measure for risk decision-making;

[0054] 2. The present system combines short-term passive risk hedging high-dimensional non-linear model predictive control (NMPC) with long-term active risk hedging deep reinforcement learning (DRL) strategy; through constructing a multi-objective risk hedging reward function containing efficiency, trajectory and risk penalty, the system can dynamically and intelligently balance between pursuing high drilling speed and trajectory adherence and avoiding catastrophic failure of drilling tools according to the risk index;

[0055] 3. The system adopts a nonlinear penalty function whose derivative increases with risk; when the risk is low, the deep reinforcement learning strategy prioritizes optimizing drilling performance indicators; but when the risk approaches the warning threshold, the penalty term increases sharply and dominates, forcing the system to automatically and actively sacrifice short-term performance to quickly reduce the risk index, causing the system to switch from a performance optimizer to a survivalist, thereby avoiding catastrophic failures such as tool loss in the well, and possessing anti-fragility;

[0056] 4. The system explicitly considers the uncertainty of the geological model parameters and the data delay and packet loss rate in well-site communication; the attitude prediction module predicts a clean attitude sequence based on a high-fidelity physical model as a reference, and the vibration inference module processes the communication channel parameters through a recurrent neural network to distinguish between real hidden vibrations and artifacts caused by poor communication quality, ensuring that the inferred risk index is based on real mechanical stress downhole, so that the control decision remains effective and robust under the dual constraints of uncertain geology and unreliable communication. BRIEF DESCRIPTION OF DRAWINGS

[0057] The application will be further explained below in conjunction with the accompanying drawings and examples:

[0058] Figure 1 is a structural diagram of the system of the application. DETAILED DESCRIPTION

[0059] To make the purpose, technical scheme and advantages of the application clearer and more apparent, the application will be further described in detail below in conjunction with specific examples.

[0060] Example 1:

[0061] Please refer to Figure 1 A geological steering closed-loop control system, comprising:

[0062] A data acquisition module for acquiring multi-dimensional data in the process of geological steering drilling;

[0063] The multi-dimensional data includes preset geological model parameters, preset target trajectory parameters, drilling engineering parameters, communication channel parameters, measured data sets, and historical control action sequences.

[0064] The drilling engineering parameters include observable drilling tool vibration measurement values.

[0065] An attitude prediction module for outputting a clean predicted data set based on the geological model parameters and the drilling engineering parameters through a high-dimensional nonlinear model predictive control state space model.

[0066] The clean predicted data set represents a predicted drilling tool attitude sequence.

[0067] a vibration inference module configured to infer a high-frequency torsional vibration amplitude based on a clean prediction dataset, a measured dataset, and a historical control action sequence via a pre-trained recurrent neural network inference model;

[0068] a stress evolution module configured to calculate a current cumulative stress entropy based on the high-frequency torsional vibration amplitude and the observable drilling tool vibration measurement value via a cumulative stress entropy evolution model;

[0069] a risk assessment module configured to determine a real-time asset risk index based on the current cumulative stress entropy and a preset material mechanics failure threshold;

[0070] wherein the real-time asset risk index represents a drilling tool asset safety distance;

[0071] a closed-loop control module configured to calculate a risk hedging reward function via a deep reinforcement learning strategy in response to the real-time asset risk index, and generate an updated control constraint based on the risk hedging reward function;

[0072] wherein the updated control constraint is fed back to the attitude prediction module to adjust boundary conditions of the high-dimensional nonlinear model predictive control.

[0073] The embodiment provides a geological steering closed-loop control system, which is an intelligent control system integrating short-term passive risk avoidance and long-term active risk hedging; the system comprises a data acquisition module, an attitude prediction module, a vibration inference module, a stress evolution module, a risk assessment module, and a closed-loop control module;

[0074] The data acquisition module is configured to provide real-time and historical multidimensional data input for the entire closed-loop control system; in the embodiment, the module is responsible for collecting information from various sensors and data sources at a drilling site, and dividing the multidimensional data into geological model parameters , drilling engineering parameters, and communication channel parameters; the geological model parameters refer to wellbore geological information obtained by logging while drilling (LWD), such as resistivity and gamma value; the model parameters are pre-set to have uncertainty in the system to reflect the limitations of geological understanding; the drilling engineering parameters refer to engineering data collected from surface and downhole sensors, such as mechanical drilling speed, drilling pressure, torque, and explicitly include observable drilling tool vibration measurement values ; the communication channel parameters refer to characteristic parameters of well-ground communication such as mud pulse, which are defined as delay time and data packet loss rate in the embodiment; a measured dataset refers to a dirty dataset actually transmitted to the ground via well-ground communication, which contains observable vibrations , LWD geological parameters, and the like, and the data quality thereof is affected by the above communication channel parameters and Severe impact of the wellbore trajectory deviation from the pre-planned trajectory Reference of the baseline trajectory History of control action sequence ;

[0075] Attitude prediction module, which aims to provide a clean baseline prediction of the future BHA attitude without the disturbance of the communication delay and the hidden vibration; in this embodiment, it is based on the geology model parameters and the drilling engineering parameters such as the control input , and outputs a clean prediction dataset through a high-dimensional nonlinear model predictive control (NMPC) state space model ; the clean prediction dataset refers to a predicted BHA attitude sequence, for example, the BHA attitude generated by the prediction model inside the NMPC controller without the disturbance of the communication delay or the hidden vibration, which contains a short-term prediction sequence of the position, inclination, and azimuth; the state space model is the prediction core of the NMPC controller, aiming to provide a deterministic and high-frequency short-term prediction of the BHA attitude, which can be mathematically expressed as follows: ; wherein, : the physical meaning is the predicted BHA attitude vector at t+1, which contains the position, inclination, and azimuth, and is derived from the calculation of this model; : the physical meaning is the BHA attitude vector at t, which is derived from the calculation result of the last control step or the current state; : the physical meaning is the BHA state transition matrix, which describes the inertia of the drilling tool, and is derived from the pre-calibration according to the specific physical structure of the BHA, such as the length of the drill collar, the position of the stabilizer, and the material mechanics properties; : the physical meaning is the control input matrix, which is also pre-calibrated according to the physical structure and material mechanics properties of the BHA; : the physical meaning is the control instruction vector applied at t, for example, the weight on bit and the rotary speed, which is derived from the solution of the NMPC controller in the current control step; : the physical meaning is the geology interaction function, which is derived from the rock mechanics parameters in the geology model parameters , such as the drillability fitting; : the physical meaning is the geology model parameters at t, which are derived from the real-time acquisition by the data acquisition module; in operation, the NMPC controller solves this model in each control step to output a safe control instruction sequence , achieving passive risk avoidance; the predicted attitude sequence output by the model is defined as the clean prediction dataset , and is transmitted to the vibration inference module;

[0076] a vibration inference module, which aims to infer the high-frequency torsional oscillation (HFTO) that cannot be directly observed on the surface but is extremely harmful from the disturbed measured data; in this embodiment, it solves the HFTO amplitude from the clean predicted data set , the measured data set , and the historical control action sequence by a pre-trained recurrent neural network (RNN) inference model; the mathematical expression of the inference model is as follows: ; wherein, : the physical meaning is the inferred HFTO amplitude at time t, and the source is calculated by the model; : the physical meaning is the pre-trained RNN inference model, and the source is obtained by generating a large-scale simulation data set containing various geological disturbances, communication delays , and packet loss rates on the BHA dynamics simulation platform, and performing offline supervised learning training; : the physical meaning is the "dirty" measured data set with delay / distortion, and the source is obtained by the data acquisition module; : the physical meaning is the "clean" data set predicted by the NMPC, and the source is calculated by the attitude prediction module; : the physical meaning is the historical control action sequence, and the source is obtained by the data acquisition module; the core logic of this module is, the model learns the asymmetric deviation between actual occurrence and expected occurrence, that is, the HFTO has little effect on the low-frequency trajectory , but has a unique modulation characteristic on the high-frequency vibration mode, thereby reversing the implicit in the dirty data;

[0077] a stress evolution module, which aims to quantify the cumulative fatigue damage of downhole tools caused by the implicit vibration HFTO and the observable vibration; in this embodiment, it calculates the current cumulative stress entropy from the high-frequency torsional oscillation amplitude solved by the vibration inference module and the observable drilling tool vibration measurement value extracted from the by a cumulative stress entropy evolution model; the evolution model is based on material fatigue damage such as the modified Miner's rule and ensures dimensional consistency, and the mathematical expression is as follows:

[0078] ;

[0079] wherein, ,​ : physical meaning is the cumulative stress entropy at t+1 and t, its source is : is the calculation result of the last time, : is the output of the model; : physical meaning is the stress natural release coefficient, representing the material damping, dimensionless, its source is the inherent physical property of the material, which is pre-set after experimental determination; : physical meaning is the HFTO damage contribution weight, the dimension is the same as ; : physical meaning is the observable vibration damage contribution weight, the dimension is the same as ; is the calibration of and , its source is to obtain a set of historical vibration data, the historical data here is defined as and , and the corresponding actual tool failure time, defined as , is calibrated by using the evolutionary model of the embodiment, and is calibrated by regression analysis method such as least square method, and usually to reflect the dominant position of HFTO; : physical meaning is the inferred HFTO amplitude, its source is calculated by the vibration inference module; : physical meaning is the reference benchmark value of HFTO amplitude, the dimension is the same as ; : physical meaning is the reference benchmark value of observable vibration, the dimension is the same as ; to further clarify the source of and , according to the mechanical endurance limit of the drilling tool or the statistical distribution of the historical vibration data, for example, taking 95% quantile, the calibration parameter is set; : physical meaning is fatigue damage index, dimensionless, its source is determined according to the slope of the S-N fatigue curve of the material, usually ; : physical meaning is the measured value of observable vibration, its source is extracted from the measured data set of the data acquisition module; this model introduces reference benchmarks and , and the vibration terms and are both dimensionless; at this time, the dimensions of the contribution weights and are the same as , which ensures the dimensional consistency of the entire equation; since , when HFTO breaks out, It will surge exponentially; the output of this module That is, the next moment It is transmitted to the risk assessment module;

[0080] The risk assessment module aims to transform the accumulated stress entropy value into a standardized risk index that can be used for decision-making; in this embodiment, it is based on the current accumulated stress entropy calculated by the stress evolution module. and preset material mechanical failure threshold Determine the real-time asset risk index Real-time asset risk index This refers to the safe distance representing the drilling asset from its catastrophic failure boundary, which is a normalized value. Risk indicators for the range; material mechanical failure threshold This refers to the failure boundary determined by the cumulative stress entropy based on the SN fatigue curve and historical failure data of the BHA drilling tool. Its source is based on historical failure data, such as... and the corresponding vibration data , ,use The cumulative stress entropy statistical value representing the catastrophic failure of a lost downhole tool (LIH) obtained by back-calculation from the evolution model is, for example, taken as its statistical average or median; the formula for calculating this risk index is: ;in, The physical meaning is the asset risk index 0-1 at time t, which is derived from the calculation of this model; The physical meaning is the cumulative stress entropy at time t, which is calculated by the stress evolution module. The physical meaning is the preset material mechanical failure threshold, which is derived from the above and pre-determined based on historical failure data and the material's SN curve; the output of this module is... It is transmitted to the closed-loop control module;

[0081] The closed-loop control module serves as the upper-level strategy optimization module for the NMPC controller, enabling a strategic shift from efficiency-first to survival-first priorities. In this embodiment, it responds to the real-time asset risk index determined by the risk assessment module. A risk-hedging reward function is calculated using a deep reinforcement learning (DRL) strategy. And generate updated control constraints based on maximizing the risk hedging reward function; this risk hedging reward function It is a utility function for multi-objective optimization, and its mathematical expression is as follows: ;in, Physically, t represents the total reward at time t, which is the optimization objective of the DRL strategy. The physical meaning is the balance weight between efficiency, trajectory and risk. Its source is the key hyperparameter in the DRL training process, which is obtained by convergence in a large number of simulation trainings, such as through grid search or Bayesian optimization. The physical meaning is based on the mechanical drilling rate. The efficiency reward function, its source It is obtained from the system status; The physical meaning is based on the deviation of trajectory fit. The trajectory reward function, its source It is obtained from the system status; The physical meaning is related to the risk index. The nonlinear penalty function, its source It is calculated by the risk assessment module; the goal of the DRL agent is to maximize ;when At a very high level, The penalty factor becomes dominant, forcing DRL to sacrifice short-term performance, i.e., reduce... and Actions to exchange for The fallback; the updated control constraints refer to the DRL agent maximizing The new control targets or constraints output, such as reducing the maximum allowable drilling rate, are... Or adjust the incentive for trajectory correction; these updated control constraints are fed back to the attitude prediction module as the NMPC controller solves in the next control step. The boundary conditions are updated in real time, thus forming a complete closed-loop control.

[0082] This invention provides a complete geological steering closed-loop control system. By combining NMPC (Non-Minimum Dynamic Management), which provides short-term passive risk mitigation, with DRL (Distant Dynamic Risk Management), which provides long-term active risk hedging, this system resolves the core conflict between drilling efficiency and asset security. This system does not rely on perfect measured data, but rather on innovative analysis of clean predictive data. With dirty measured data The deviation between them is used to infer the amplitude of the unobservable latent vibration HFTO. ; by quantifying this cost as cumulative stress entropy and standardized risk index The system has acquired antifragility: it can intelligently and proactively sacrifice short-term drilling efficiency in exchange for the long-term survival of downhole assets when risks approach, that is, avoid LIH disasters, and achieve a dynamic balance between drilling target finding and risk avoidance in uncertain geological and communication environments.

[0083] Example 2:

[0084] The attitude prediction module is used to predict the state-space model of the control system based on geological model parameters and drilling engineering parameters using a high-dimensional nonlinear model, and outputs a clean prediction dataset, including:

[0085] Based on the rock mechanics parameters in the geological model parameters, a geological interaction function is fitted;

[0086] Based on the preset physical structure of the drill string and the mechanical properties of the materials, the state transition matrix is ​​calibrated;

[0087] Based on the preset physical structure of the drill string and the mechanical properties of the materials, the control input matrix is ​​calibrated;

[0088] Within each control step, the state transition matrix, control input matrix, and geological interaction function are invoked to solve the state-space model and output a clean prediction dataset.

[0089] This embodiment, based on Example 1, specifies the specific implementation method of the attitude prediction module; the attitude prediction module predicts the state space model of the NMPC through a high-dimensional nonlinear model. Output clean prediction dataset The process specifically includes: based on geological model parameters Rock mechanics parameters in the data, fitting geological interaction function To illustrate this further, the fitting process involves acquiring a set of calibration datasets containing different rock mechanics parameters, defined as follows: The interaction force with the corresponding actual drill string is defined as The experimental measurements are based on and The relationship between them can be determined using methods such as polynomial fitting or neural network regression. The specific form of the function; at runtime, the module is based on... Calculate the geological interaction function using the extracted current rock mechanical parameters. Based on the preset physical structure of the drill string and the mechanical properties of the materials, the state transition matrix is ​​calibrated. Specifically, this calibration involves establishing a finite element dynamics model of the BHA, inputting the physical structure of the drill string, such as the drill collar length and stabilizer position, and the material mechanical properties, running the simulation, and then identifying or deriving the state transition matrix from the simulation results. Based on the preset physical structure and material mechanical properties of the drill bit, calibrate the control input matrix. Its calibration method is the same as The same principle applies: both are derived through simulation and identification using the aforementioned finite element dynamics model; within each control step, the calibrated and fitted state transition matrix is ​​invoked. Control input matrix Interaction functions with geology Solve the state-space model to output a clean prediction dataset; that is, the NMPC controller at each time step. Utilizing these established , , and the current and Solve the equations to obtain safe control commands. and the predicted posture at the next moment This predicted pose sequence This constitutes a clean prediction dataset. ;

[0090] By analyzing the state-space model , , By clearly defining the sources of the three core components—physical calibration and geological fitting—this implementation method ensures clean predictions output by the attitude prediction module. It is not an abstract simulation out of thin air, but a high-fidelity prediction based on specific BHA physical properties and specific formation mechanical parameters; this high fidelity greatly improves... The reliability of the benchmark enables the asymmetric deviations analyzed by the subsequent vibration inference module to more accurately reflect the latent vibrations, rather than the errors of the model itself, thereby improving the inference accuracy of the entire risk assessment closed loop.

[0091] Example 3:

[0092] The vibration inference module is used to calculate the amplitude of high-frequency torsional vibration based on a clean prediction dataset, a measured dataset, and historical control action sequences, using a pre-trained recurrent neural network inference model. This includes:

[0093] Align the measured dataset with the clean prediction dataset;

[0094] Learn the asymmetric bias between the measured dataset and the clean prediction dataset;

[0095] The high-frequency torsional vibration amplitude is inferred by using a pre-trained recurrent neural network inference model, which processes asymmetric bias, data delay and packet loss rate characterized by communication channel parameters.

[0096] This embodiment, based on Example 1, specifies the specific implementation method of the vibration inference module; the vibration inference module uses a pre-trained RNN inference model. Calculate the amplitude of high-frequency torsional vibration The process specifically includes: aligning the measured dataset. and clean prediction dataset This step is data preprocessing; the module receives data from the data acquisition module. This data is subject to Impact, and receive from the attitude prediction module ;because There is a communication delay, so it must be connected to a clean one. Perform time-series alignment to ensure that subsequent comparisons are between expected and actual values ​​at the same point in time; learn from the actual test dataset. With clean prediction dataset The asymmetric deviation between them; this is The core function of the model; as mentioned before, this asymmetric bias refers to the small impact of HFTO on low-frequency trajectories, i.e. and Their trajectories are similar, but they exhibit unique modulation of high-frequency vibration modes, namely... Vibration characteristics and The smoothing predictions are significantly different; The model is pre-trained to identify this specific bias pattern; to solve Interference caused by potential model errors in the prediction itself. The simulation dataset used for model pre-training must include not only HFTO conditions, but also a variety of geological parameters. mutation or Model mismatch conditions; this forces Learn to distinguish between true high-frequency deviation characteristics caused by HFTO and those caused by... Low-frequency predicted trajectory drift caused by model inaccuracies, thereby improving... Robustness of inference; inference model using a pre-trained recurrent neural network. This involves processing the asymmetric bias identified in the previous step and the data delay characterized by communication channel parameters. and packet loss rate To infer the amplitude of high-frequency torsional vibration RNN model Not only with and The bias is taken as input, and is also learned in an intrinsic way, i.e., through offline training, or in an extrinsic way, i.e. As an additional input, it was considered and The impact; this means Learned to distinguish between true bias caused by HFTO and true bias caused by HFTO. The resulting communication artifacts; by processing this complex, noisy bias signal, the RNN's final output modulates the implicit HFTO amplitude. The estimate;

[0097] This implementation method explicitly introduces and processes communication channel parameters. , greatly enhancing the robustness of the vibration inference; it enables the system to distinguish between signals, i.e. real HFTO, and noise, i.e. communication delay and packet loss; this avoids the system from generating false collapse warnings due to poor data quality, ensuring that the inferred is a reliable indicator of the real mechanical stress downhole, thus enabling subsequent risk decisions, such as DRL performance sacrificing decisions, to be based on facts rather than artefacts.

[0098] Embodiment 4:

[0099] a stress evolution module configured to calculate, based on the high-frequency torsional vibration amplitude and the observable drilling tool vibration measurement, a current cumulative stress entropy by a cumulative stress entropy evolution model, including:

[0100] obtaining a preset stress natural release coefficient, a high-frequency torsional vibration damage contribution weight, an observable vibration damage contribution weight, a fatigue damage index, and a vibration reference benchmark value;

[0101] dividing the high-frequency torsional vibration amplitude by the corresponding vibration reference benchmark value to obtain a first dimensionless vibration term;

[0102] dividing the observable drilling tool vibration measurement by the corresponding vibration reference benchmark value to obtain a second dimensionless vibration term;

[0103] updating and outputting the current cumulative stress entropy based on the stress natural release coefficient, the high-frequency torsional vibration damage contribution weight, the observable vibration damage contribution weight, the fatigue damage index, the first dimensionless vibration term, and the second dimensionless vibration term.

[0104] The embodiment limits the specific implementation of the stress evolution module on the basis of Embodiment 1; the process of calculating the current cumulative stress entropy by the cumulative stress entropy evolution model of the stress evolution module specifically includes: obtaining a series of preset physical and calibration parameters, including: a stress natural release coefficient , a high-frequency torsional vibration damage contribution weight , an observable vibration damage contribution weight , a fatigue damage index , and a vibration reference benchmark value and ; the sources and calibration methods of these parameters have been detailed in the implementation of Embodiment 1; dividing the high-frequency torsional vibration amplitude by the corresponding vibration reference benchmark value to obtain a first dimensionless vibration term; dividing the observable drilling tool vibration measurement by the corresponding vibration reference benchmark value to obtain a second dimensionless vibration term; updating and outputting the current cumulative stress entropy based on the stress natural release coefficient High frequency torsional vibration damage contribution weight Observable vibration damage contribution weight Fatigue damage index The first dimensionless vibration term and the second dimensionless vibration term update and output the current cumulative stress entropy The update process is to execute the evolution formula: The module calculates After that, it is output to the risk assessment module as the current cumulative stress entropy.

[0105] The embodiment defines the creation step of the dimensionless vibration term, ensures the physical meaning rigor and dimensional consistency of the cumulative stress entropy model, and converts the vibration measurement value with dimension into a dimensionless proportional term, so that the contribution weight , and and have the same clear physical dimension, i.e., entropy, which ensures that is a real physical measure that can be accumulated and compared, rather than an arbitrary score, which provides a basis for the reliability of the subsequent risk index .

[0106] Embodiment 5:

[0107] The risk assessment module is configured to determine a real-time asset risk index according to the current cumulative stress entropy and a preset material mechanics failure threshold, and includes:

[0108] The current cumulative stress entropy and the preset material mechanics failure threshold are called.

[0109] The risk index is obtained by calculating the ratio of the current cumulative stress entropy to the material mechanics failure threshold.

[0110] The risk index is defined as the real-time asset risk index.

[0111] The embodiment is based on embodiment 1, and the specific implementation of the risk assessment module is limited; the process of determining the real-time asset risk index by the risk assessment module specifically includes: calling the current cumulative stress entropy and the preset material mechanics failure threshold ; the module obtains the real-time calculation value of from the stress evolution module, and reads the calibrated from the system configuration, and the calibration method has been described in the embodiment of embodiment 1; the current cumulative stress entropy and the material mechanics failure threshold ​the ratio of the two values, obtaining a risk indicator; define this risk indicator as real-time asset risk index ; that is ; this value is output to the closed-loop control module;

[0112] The present embodiment realizes the normalization of risk through a simple ratio operation; it converts a physical quantity that can vary in a very large range , namely the cumulative stress entropy, into a normalized, usually running in the interval 0,1, and having an intuitive meaning , namely the risk percentage; this normalization makes the risk easy to understand and operate; for example, a value of 1000 units for drill A and a value of 5000 units for drill B, both mean the same imminent collapse state; this allows the downstream closed-loop control module DRL to define a set of general control strategies that are independent of the absolute threshold values of specific tools.

[0113] Example 6:

[0114] The risk assessment module is also used for:

[0115] setting a preset high-risk steady-state operation baseline and a preset collapse warning threshold;

[0116] comparing the real-time asset risk index with the high-risk steady-state operation baseline and the collapse warning threshold;

[0117] when the real-time asset risk index is lower than or equal to the high-risk steady-state operation baseline, determining the current risk state as safe;

[0118] when the real-time asset risk index is greater than the high-risk steady-state operation baseline and lower than or equal to the collapse warning threshold, determining the current risk state as high-risk steady-state;

[0119] when the real-time asset risk index is greater than the collapse warning threshold, determining the current risk state as collapse warning.

[0120] The present embodiment extends the function of the risk assessment module based on example 5, adding the function of risk state determination; the module is also used for: setting a preset high-risk steady-state operation baseline and a preset collapse warning threshold ; to ensure implementability, these thresholds are not set arbitrarily, but are pre-calibrated and stored according to historical drilling data and risk control strategies, for example, the value corresponding to an acceptable 9-minute failure risk is determined based on statistical analysis; for example, in the present embodiment, the high-risk steady-state operation baseline can be set to 0.5; This corresponds to , defines the upper limit of the acceptable mechanical vibration margin in daily drilling; a collapse warning threshold ; this defines the hard red line of approaching 9.5 points of risk, system is about to fail, must take immediate evasive action; compares the real-time asset risk index calculated in embodiment 5 with the high-risk steady-state operation baseline and the collapse warning threshold ; based on the comparison result, the current risk state is classified and determined: when the real-time asset risk index is lower than or equal to the high-risk steady-state operation baseline, i.e. , the current risk state is determined to be safe; when the real-time asset risk index is greater than the high-risk steady-state operation baseline and lower than or equal to the collapse warning threshold, i.e. , the current risk state is determined to be high-risk steady state; when the real-time asset risk index is greater than the collapse warning threshold, i.e. , the current risk state is determined to be collapse warning;

[0121] This embodiment realizes the discrete state division of the continuous risk index by introducing two additional thresholds; this division, i.e. safe, high-risk steady state, and collapse warning, has a dual role: first, it provides more intuitive and easy-to-understand risk alerts for ground operators; second, it defines clear, non-overlapping operation intervals for the DRL strategy of the closed-loop control module; the DRL strategy can execute different, e.g. aggressive or conservative, control logic according to which state interval it is currently in, which is the basis for implementing segmented strategies.

[0122] Embodiment 7:

[0123] The closed-loop control module is configured to calculate a risk hedging reward function through a deep reinforcement learning strategy in response to the real-time asset risk index, and generate updated control constraints based on the risk hedging reward function, including:

[0124] obtaining the mechanical drilling rate and the trajectory adherence deviation from the system state;

[0125] determining an efficiency reward function based on the mechanical drilling rate;

[0126] determining a trajectory reward function based on the trajectory adherence deviation;

[0127] determining a nonlinear penalty function based on the real-time asset risk index;

[0128] calling a preset balance weight;

[0129] combining the efficiency reward function, the trajectory reward function and the nonlinear penalty function through the balance weight to construct the risk hedging reward function;

[0130] By maximizing the risk hedging reward function, updated control constraints are determined and generated.

[0131] This implementation method, based on Example 1, specifies the concrete implementation of the closed-loop control module and details the risk hedging reward function. The construction process; this process specifically includes: obtaining the mechanical drilling rate from the system state. Deviation in alignment with trajectory These are the two core performance indicators of the system; based on mechanical drilling speed. Determine the efficiency reward function For example, it can be set to It itself or its normalized value, i.e. This is a positive correlation function. The higher, The higher the reward value, the greater the reward for drilling faster; based on trajectory fit deviation. Determine the trajectory reward function For example, it can be set as a function of the normalized bias, i.e. or ,in It is a preset trajectory deviation reference value to ensure It is a dimensionless utility value; it is a negative correlation function. The smaller the size, the closer it fits the trajectory. The higher the reward value, the greater the reward for accurate drilling; this is based on the real-time asset risk index input from the risk assessment module. Determine the nonlinear penalty function This is a positive correlation function. The higher, The higher the penalty value, the greater the punishment for the risk; the preset balance weights are invoked. These weights are hyperparameters obtained by the DRL model through extensive simulation training and convergence, and their acquisition method has been explained in the implementation of Example 1; by balancing the weights, the efficiency reward function, the trajectory reward function, and the nonlinear penalty function are combined to construct the risk hedging reward function. The function expression is: By maximizing the risk hedging reward function The DRL agent identifies and generates updated control constraints; it selects an action, such as lowering the allowed limits. The upper limit, the goal of this action is to maximize its total reward in the future. Expected value maximization; the selected action is reflected in the updated control constraints and is fed back to the NMPC module;

[0132] This implementation method is clearly defined. The construction process of this invention elaborates on the core of multi-objective optimization; it mathematically unifies the three conflicting drilling objectives of efficiency, trajectory, and risk into a single reward function. In the middle; DRL agents maximize This simple and unified driving objective can automatically and intelligently weigh the three objectives; this utility function-based construction method provides a clear mathematical basis and an executable path for DRL's counterintuitive decision-making.

[0133] Example 8:

[0134] The nonlinear penalty function is pre-defined as a function whose derivative increases with the real-time asset risk index, and is used for:

[0135] When the real-time asset risk index is lower than or equal to the collapse warning threshold, the deep reinforcement learning strategy prioritizes optimizing the efficiency reward function and the trajectory reward function.

[0136] When the real-time asset risk index exceeds the collapse warning threshold, the nonlinear penalty function dominates the risk hedging reward function, forcing the deep reinforcement learning strategy to sacrifice short-term performance indicators in order to reduce the real-time asset risk index.

[0137] This implementation method, based on Example 7, focuses on the key nonlinear penalty function. The design was subject to detailed constraints; in the design, the nonlinear penalty function was preset as a special function, the key feature of which is that its derivative varies with the real-time asset risk index. Increases with increasing; this refers to a convex function, such as an exponential function. or inverse proportional function ,in This is a preset coefficient; its physical meaning is that the marginal cost of risk increases: when risk... When the penalty increases from 0.8 to 0.9, The increase is much greater than The increment from 0.1 to 0.2; this non-linear design allows it to have two distinct operating modes: when the real-time asset risk index is below or equal to the collapse warning threshold, i.e. , The value of and its derivative are relatively small; at this point, in the total reward In the middle, penalty items The proportion is very small; in order to maximize Deep reinforcement learning strategies prioritize optimizing the efficiency reward function. Trajectory reward function i.e. to aggressively target and pursue high ROP strategy; when the real-time asset risk index is greater than the collapse warning threshold, i.e. , the value of the stress entropy spikes sharply due to its exponential nature; at this point, the penalty term becomes extremely large, dominating the calculation of ; in order to maximize i.e. to avoid this huge negative penalty, the only option for the DRL policy is to reduce at all costs; this forces the deep reinforcement learning policy to make counter-intuitive decisions: sacrificing short-term performance metrics, such as actively reducing at the expense of , or postponing trajectory correction at the expense of , because this performance loss, i.e. a smaller negative reward, is far smaller than the catastrophic penalty brought by ; this sacrifice allows the stress entropy to be naturally released by term, which in turn falls back, and the system escapes danger;

[0138] This embodiment realizes the strategic switching of the DRL control policy by precisely limiting the nonlinear mathematical form of ; this nonlinear design is the mathematical core of the anti-vulnerability of the present application; it allows the system to greedily pursue performance when the risk is low, i.e. ; but when the risk is high, i.e. , it provides an infinitely hard penalty barrier, forcing the system to automatically switch from a performance optimizer to a survivalist, thus achieving active and intelligent avoidance of catastrophic failure.

[0139] Example 9:

[0140] The geological model parameters are obtained by logging while drilling and are pre-set to have uncertainty;

[0141] The drilling engineering parameters are collected from surface and downhole sensors.

[0142] This embodiment further limits the source and nature of the two key input data collected by the data collection module on the basis of Example 1; the geological model parameters are obtained by logging while drilling LWD; this clearly defines are real-time geologic measurements from downhole tools, such as resistivity, gamma values, rather than a purely pre-drill model; and the parameters are pre-set with uncertainty; this acknowledges the inherent limitations of LWD measurements and geologic interpretation, is a necessary prerequisite for the design of the closed-loop feedback system of the present application, rather than an open-loop planning system; drilling engineering parameters are collected from both surface and downhole sensors; this clarifies that the data sources of engineering parameters, such as drilling speed, drilling pressure, torque, observable vibrations , are diverse, including both surface equipment and downhole MWD / LWD tools;

[0143] The present embodiment anchors the application scenario of the present application in a real, non-ideal drilling environment by limiting the input data sources; the explicit uncertainty of the geologic model reinforces the necessity of closed-loop feedback of NMPC and DRL, because the system cannot rely on a perfect prior geologic model; the explicit data sources including downhole sensors reinforce the necessity of the vibration inference module to handle dirty data, because data from downhole will inevitably be interfered by the communication channel ; therefore, the limitations of the present embodiment emphasize the robust control problem under the dual constraints of uncertain geology and unreliable communication that the present application is to solve.

[0144] It should be noted that the above examples are only used to illustrate the technical solutions of the present application rather than limit the present application, and although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A geological guidance closed-loop control system, characterized in that, include: The data acquisition module is used to collect multi-dimensional data during the geological directional drilling process; The multi-dimensional data includes: preset geological model parameters, preset target trajectory parameters, drilling engineering parameters, communication channel parameters, measured datasets, and historical control action sequences. Drilling engineering parameters include observable drill string vibration measurements; The attitude prediction module is used to predict the state-space model of the control system based on geological model parameters and drilling engineering parameters through a high-dimensional nonlinear model, and outputs a clean prediction dataset. Among them, the clean prediction dataset represents the predicted drill string attitude sequence; The vibration inference module is used to calculate the amplitude of high-frequency torsional vibration based on a clean prediction dataset, a measured dataset, and a historical control action sequence, using a pre-trained recurrent neural network inference model. The stress evolution module is used to calculate the current cumulative stress entropy based on the amplitude of high-frequency torsional vibration and observable drill string vibration measurements, using a cumulative stress entropy evolution model. The risk assessment module is used to determine the real-time asset risk index based on the current cumulative stress entropy and the preset material mechanical failure threshold. Among them, the real-time asset risk index represents the safe distance of drilling assets; The closed-loop control module is used to respond to the real-time asset risk index, calculate the risk hedging reward function through a deep reinforcement learning strategy, and generate updated control constraints based on the risk hedging reward function. The updated control constraints are fed back to the attitude prediction module to adjust the boundary conditions of the high-dimensional nonlinear model predictive control.

2. The geological guidance closed-loop control system according to claim 1, characterized in that, The attitude prediction module is used to predict the state-space model of the control system based on geological model parameters and drilling engineering parameters using a high-dimensional nonlinear model, and output a clean prediction dataset, including: Based on the rock mechanics parameters in the geological model parameters, a geological interaction function is fitted; Based on the preset physical structure of the drill string and the mechanical properties of the materials, the state transition matrix is ​​calibrated; Based on the preset physical structure of the drill string and the mechanical properties of the materials, the control input matrix is ​​calibrated; Within each control step, the state transition matrix, control input matrix, and geological interaction function are invoked to solve the state-space model and output a clean prediction dataset.

3. The geological guidance closed-loop control system according to claim 1, characterized in that, The vibration inference module is used to calculate the amplitude of high-frequency torsional vibration based on a clean prediction dataset, a measured dataset, and historical control action sequences, using a pre-trained recurrent neural network inference model. This includes: Align the measured dataset with the clean prediction dataset; Learn the asymmetric bias between the measured dataset and the clean prediction dataset; The high-frequency torsional vibration amplitude is inferred by using a pre-trained recurrent neural network inference model, which processes asymmetric bias, data delay and packet loss rate characterized by communication channel parameters.

4. The geological guidance closed-loop control system according to claim 1, characterized in that, The stress evolution module is used to calculate the current cumulative stress entropy based on the high-frequency torsional vibration amplitude and observable drill string vibration measurements, using a cumulative stress entropy evolution model, including: Obtain the preset stress natural release coefficient, high-frequency torsional vibration damage contribution weight, observable vibration damage contribution weight, fatigue damage index, and vibration reference value; Divide the high-frequency torsional vibration amplitude by the corresponding vibration reference value to obtain the first dimensionless vibration term. Divide the observable drill string vibration measurement value by the corresponding vibration reference value to obtain the second dimensionless vibration term; Based on the stress natural release coefficient, the contribution weight of high-frequency torsional vibration damage, the contribution weight of observable vibration damage, the fatigue damage index, the first dimensionless vibration term, and the second dimensionless vibration term, the current cumulative stress entropy is updated and output.

5. A geological guidance closed-loop control system according to claim 1, characterized in that, The risk assessment module is used to determine a real-time asset risk index based on the current cumulative stress entropy and a preset material mechanical failure threshold, including: Call the current accumulated stress entropy and the preset material mechanical failure threshold; The risk index is obtained by calculating the ratio of the current cumulative stress entropy to the material mechanical failure threshold. The risk indicator is defined as a real-time asset risk index.

6. A geological guidance closed-loop control system according to claim 5, characterized in that, The risk assessment module is also used for: Set a preset high-risk steady-state operating baseline and a preset collapse warning threshold; Compare the real-time asset risk index with the high-risk steady-state operating baseline and the collapse warning threshold; When the real-time asset risk index is lower than or equal to the high-risk steady-state operating baseline, the current risk status is determined to be safe. When the real-time asset risk index is greater than the high-risk steady-state operating baseline and lower than or equal to the collapse warning threshold, the current risk state is judged as a high-risk steady state. When the real-time asset risk index exceeds the collapse warning threshold, the current risk status is determined to be a collapse warning.

7. A geological guidance closed-loop control system according to claim 1, characterized in that, The closed-loop control module is used to respond to the real-time asset risk index, calculate the risk hedging reward function through a deep reinforcement learning strategy, and generate updated control constraints based on the risk hedging reward function, including: Obtain the mechanical drilling rate and trajectory fit deviation from the system status; Determine the efficiency reward function based on the mechanical drilling rate; The trajectory reward function is determined based on the trajectory fit deviation; A nonlinear penalty function is determined based on the real-time asset risk index. Invoke the preset balance weights; By balancing weights, an efficiency reward function, a trajectory reward function, and a nonlinear penalty function are combined to construct a risk hedging reward function. By maximizing the risk hedging reward function, updated control constraints are determined and generated.

8. A geological guidance closed-loop control system according to claim 7, characterized in that, The nonlinear penalty function is preset to be a function whose derivative increases with the increase of the real-time asset risk index, and is used for: When the real-time asset risk index is lower than or equal to the collapse warning threshold, the deep reinforcement learning strategy prioritizes optimizing the efficiency reward function and the trajectory reward function. When the real-time asset risk index exceeds the collapse warning threshold, the nonlinear penalty function dominates the risk hedging reward function, forcing the deep reinforcement learning strategy to sacrifice short-term performance indicators in order to reduce the real-time asset risk index.

9. A geological guidance closed-loop control system according to claim 1, characterized in that, The geological model parameters are obtained through logging while drilling and are preset to have uncertainty; Drilling parameters are collected from surface and downhole sensors.

Images