A method for adaptive optimization of electric drive drilling parameters based on reinforcement learning

Abstract

The application discloses a kind of electric drive well drilling parameter adaptive optimization methods based on reinforcement learning, it is related to intelligent drilling control technical field, including the following steps: acquisition electric drive top drive frequency converter operating data, wellhead sensing data and well logging data, generate well drilling state sample frame;Through the Smith predictor compensation electric drive frequency converter based on linear torsion equation, generate downhole alignment state;Through KAN state encoder generates well drilling smooth state feature;According to drill string torsional fluctuation equation and rock breakage specific energy equation generates electric drive well drilling physical residual trend item;Through mechanism embedding KAN-Critic value network generates smooth value evaluation result;Through Actor strategy network generates the strategy distribution of drilling pressure and rotating speed;According to strategy distribution generates target drilling pressure and target rotating speed and issues execution, updates network according to feedback.The application improves the stability, adaptability and safety of electric drive well drilling parameter optimization.

Description

Technical Field

[0001] This invention relates to the field of intelligent drilling control technology, and in particular to an adaptive optimization method for electric drilling parameters based on reinforcement learning. Background Technology

[0002] With the increasing application of electric drive top drive, frequency converter control, and drilling-while-drilling sensor data in deep and horizontal well drilling operations, automatic optimization technology for drilling parameters such as drilling pressure and rotational speed has gradually attracted attention. Among publicly available cases, CN106640035A discloses a VFD control system and method for automatic drilling parameter optimization, based on real-time drilling data and mechanical energy calculation; CN115628009A discloses the application of reinforcement learning for intelligent wellbore trajectory control; CN117764119A discloses the introduction of physical information into the Critic network loss function of reinforcement learning; and CN120630306A discloses the use of KAN networks and learnable B-spline basis functions for oil and gas logging shear wave velocity prediction. Therefore, it is evident that related disclosures exist regarding electric drive-side drilling parameter optimization, reinforcement learning control, physical constraint loss, and KAN modeling.

[0003] However, in the scenario of adaptive optimization of electric drilling parameters, existing technologies still have the following problems: the wellhead sensor data and the actual downhole drill bit state are affected by the execution time delay of the electric drive inverter and the propagation time delay of the drill string elastic wave. Simply adjusting based on wellhead drilling pressure, rotation speed, torque and mechanical energy can easily cause downhole state alignment lag; traditional mechanical energy optimization or ordinary reinforcement learning control mainly focuses on reward results, making it difficult to establish a constraint relationship between the drill string torsional fluctuation residual, rock fracture energy residual and Q-value surface change trend, resulting in insufficient stability of the strategy when the formation changes suddenly and the torque fluctuates; simply using physical loss or replacing conventional neural networks with KAN is also difficult to dynamically shrink the exploration variance of drilling pressure and rotation speed based on the safety margin of the second derivative of the spline, which can easily lead to overexploration, parameter fluctuation and execution shock in high residual well sections, affecting the continuity and safety of electric drilling control.

[0004] Therefore, how to provide an adaptive optimization method for electric drilling parameters based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an adaptive optimization method for electric drilling parameters based on reinforcement learning. This invention fully utilizes Smith predictors, KAN state encoding, mechanism-embedded KAN-Critic value networks, and Actor policy networks to perform time-delay compensation, state smoothing representation, physical residual constraints, and adaptive strategy optimization for drilling pressure and rotational speed during electric drilling. It has the advantages of stable parameter adjustment, strong adaptability to operating conditions, and high drilling safety.

[0006] An adaptive optimization method for electric drilling parameters based on reinforcement learning according to an embodiment of the present invention includes the following steps: Step 1: Collect operating data of the electric drive top drive frequency converter, wellhead sensor data and logging data, perform data preprocessing, and generate drilling status sample frames; Step 2: Input the drilling state sample frame into the Smith predictor based on the linear torsion equation to compensate for the execution delay of the electric drive frequency converter and the propagation delay of the drill string elastic wave, and generate the downhole alignment state. Step 3: Input the downhole alignment state into the KAN state encoder, and generate continuous and differentiable drilling smooth state features through the spline edge activation function; Step 4: Generate the physical residual trend term of electric drilling based on the drill string torsional wave equation and the rock fracture energy equation; Step 5: Embed the drilling smoothing state characteristics, drilling pressure action, and rotation speed action input mechanism into the KAN-Critic value network, and generate smoothing value evaluation results based on the constraints of the time series difference error and the trend term of the electric drilling physical residual on the second derivative of the Q-value surface. Step 6: Input the smoothed value assessment results into the Actor strategy network to generate the strategy distribution of drilling pressure and rotation speed, and dynamically adjust the exploration variance of the strategy distribution according to the safety margin of the second derivative of the spline. Step 7: Generate target drilling pressure and target rotation speed based on the strategy distribution, and then send them out for execution after first-order low-pass filtering and rate limiting processing. The KAN-Critic value network and Actor strategy network are embedded based on the feedback update mechanism of mechanical drilling speed, torque and standpipe pressure.

[0007] Optionally, the operating data of the electric drive top drive frequency converter includes the top drive motor speed, top drive output torque, frequency converter load rate and operating status mark; the wellhead sensor data includes drilling pressure, standpipe pressure, well depth and mechanical drilling speed; and the logging data includes lithology type, formation position, well depth mark and drilling condition mark. Using the sampling timescale of the electric drive top drive frequency converter operation data as the main timescale, the wellhead sensor data is incorporated into the main timescale sequence according to the sampling timescale, and the logging data is incorporated into the main timescale sequence of the corresponding well depth interval according to the well depth identifier to form the original drilling status sequence. The original drilling status sequence is preprocessed to convert continuous sampling data into continuous variables with uniform dimensions, and to convert lithology, formation position, operating status markers and drilling condition markers into discrete coded variables. Confidence markers are written for sampling points with missing data, equipment malfunctions or logging delays. The original drilling state sequence after data preprocessing is slid segmented according to the preset time window. Continuous variables, discrete coded variables, well depth intervals and confidence markers within the same time window are written into the same frame of data in chronological order to generate drilling state sample frames.

[0008] Optionally, step two specifically includes: Read the rotation speed sequence, torque sequence, drilling pressure sequence, standpipe pressure sequence, mechanical drilling rate sequence, well depth range and confidence marker sequence from the drilling status sample frame, and read the drill string assembly parameters for the current well section; The execution time delay of the electric drive frequency converter is determined based on the control response process of the electric drive frequency converter, and the propagation time delay of the drill string elastic wave is determined based on the drill string assembly parameters. The execution time delay of the electric drive frequency converter and the propagation time delay of the drill string elastic wave are added together to obtain the total time delay, and the total time delay is written into the prediction time parameter of the Smith predictor. A linear torsional state equation is established using the top drive end speed, downhole drill bit end speed, torsional angle difference between the top drive end and the downhole drill bit end, downhole torque, downhole drilling pressure and standpipe pressure as state variables, and the speed setting change and drilling pressure setting change as control variables. The linear torsional state equation is discretized according to the sampling period to form the state prediction equation; Based on the state prediction equation, a time-delay-free prediction branch and a time-delay prediction branch are established. The time-delay-free prediction branch calculates the downhole prediction state after the total time delay forward from the current sampling time, while the time-delay prediction branch calculates the time-delay prediction state at the same time scale as the wellhead measured data. The difference between the measured state at the wellhead and the predicted state with time delay is used to obtain the prediction error, and the prediction error is weighted and corrected according to the confidence level. The downhole predicted state obtained from the time-delay prediction branch is superimposed with the weighted and corrected prediction error to generate the downhole aligned state.

[0009] Optionally, determining the execution time delay of the electric drive inverter based on the control response process of the electric drive inverter, and determining the propagation time delay of the drill string elastic wave based on the drill string assembly parameters, specifically includes: The drill string assembly parameters include drill string length, drill string shear modulus, drill string material density, drill string equivalent moment of inertia, drill string equivalent torsional stiffness, and drill string equivalent damping coefficient. The execution time delay of the electric drive inverter is determined by the difference between the time scale of the control command issued by the electric drive inverter and the time scale of the top drive motor speed starting to change. The propagation velocity of the torsional elastic wave in the drill string is calculated based on the drill string length, drill string shear modulus, and drill string material density. Then, the propagation time delay of the elastic wave in the drill string is determined based on the drill string length and the propagation velocity of the torsional elastic wave in the drill string. A linear torsional state equation is established based on the drill string's equivalent moment of inertia, equivalent torsional stiffness, and equivalent damping coefficient. In the Smith predictor, the no-delay prediction branch generates the downhole prediction state based on the total time delay, while the time-delay prediction branch generates the time-delay prediction state with the same time scale as the wellhead measured data. The prediction error is obtained by subtracting the wellhead measured state from the time-delay prediction state. When the confidence level is marked as low confidence, the correction weight for prediction error is reduced; when the confidence level is marked as normal confidence, the correction weight for prediction error is maintained.

[0010] Optionally, step three specifically includes: Read the estimated downhole rotation speed, estimated downhole torque, estimated downhole drilling pressure, estimated standpipe pressure, estimated mechanical drilling rate, well depth range, and confidence marker in the downhole alignment status, and read the lithology code corresponding to the same well depth range; Based on the differences and time intervals between the estimated downhole rotation speed, estimated downhole torque, estimated downhole drilling pressure, estimated standpipe pressure, and estimated mechanical drilling speed under two adjacent aligned time scales, the rates of change of rotation speed, torque, drilling pressure, standpipe pressure, and mechanical drilling speed are generated respectively. Based on the starting well depth and preset well depth step size of the current well section, the well depth interval is converted into a well depth interval code, and the confidence mark is converted into a confidence weight; Write the downhole rotation speed estimate, downhole torque estimate, downhole drilling pressure estimate, standpipe pressure estimate, mechanical drilling rate estimate, well depth range code, lithology code, rotation speed change rate, torque change rate, drilling pressure change rate, standpipe pressure change rate, mechanical drilling rate change rate, and confidence weight into the KAN input vector. The KAN input vector is split into an electric drive response state channel, a rock breaking state channel, and a sample weight channel using a field index table. The electric drive response state channel includes downhole rotation speed estimates, downhole torque estimates, downhole drill pressure estimates, standpipe pressure estimates, and corresponding rates of change. The rock breaking state channel includes downhole drill pressure estimates, mechanical drilling rate estimates, well depth interval codes, lithology codes, drill pressure change rates, and mechanical drilling rate change rates. The sample weight channel includes confidence weights. Edge mapping is performed on each channel field using spline edge activation functions composed of B-spline basis functions, spline nodes, spline control points, and spline coefficients to generate node response values; The spline edge activation function is analytically differentiated to generate the first-order and second-order state derivative responses, and the node response values ​​are used as the state smoothing responses. The smoothed state response, first-order state response, and second-order state response corresponding to the electric drive response state channel are fused into the smoothed electric drive response feature. The smoothed state response, first-order state response, and second-order state response corresponding to the rock breaking state channel are fused into the smoothed rock breaking process feature. Drilling smoothed state features are generated based on the confidence weight.

[0011] Optionally, step four specifically includes: Read the smooth characteristics of electric drive response, smooth characteristics of rock breaking process, trend of state change, change of state curvature, well depth interval code and lithology code in the smooth state characteristics of drilling, and read the drill bit diameter, equivalent rotational inertia of drill string, equivalent torsional stiffness of drill string and equivalent damping coefficient of drill string corresponding to the current well depth interval. Based on the rotational speed response, torque response, and state curvature change in the smooth characteristics of the electric drive response, the torsional fluctuation response of the drill string is formed, and the theoretical torsional response is calculated based on the equivalent rotational inertia of the drill string, the equivalent torsional stiffness of the drill string, and the equivalent damping coefficient of the drill string. The drill string torsional fluctuation residual is generated by subtracting the drill string torsional fluctuation response from the theoretical torsional response. The actual rock-breaking specific energy is calculated based on the downhole torque and downhole speed in the smooth characteristics of the electric drive response, and the downhole drilling pressure, mechanical drilling speed, drill bit diameter and lithology code in the smooth characteristics of the rock-breaking process. Read the benchmark rock breaking energy corresponding to the current lithology code, and calculate the difference between the actual rock breaking energy and the benchmark rock breaking energy to generate the rock breaking energy residual. Based on the residuals of torsional fluctuation of the drill string and the residuals of rock breaking energy in a continuous arrangement of well depth intervals, calculate the direction of residual change, the magnitude of residual change, and the rate of residual change in adjacent well depth intervals; When the lithology code changes between adjacent well depths, a formation transition marker is written, and the formation transition marker is associated with the direction of residual change, the magnitude of residual change, and the rate of residual change. The dimensions of the drill string torsional fluctuation residual, rock breaking energy residual, residual change rate, and formation transition marker are unified and the amplitude is normalized to generate an electric-drive drilling physical residual trend term. The electric-drive drilling physical residual trend term includes torsional fluctuation residual components, rock breaking energy residual components, residual change rate components, and formation transition components.

[0012] Optionally, step five specifically includes: Read drilling smoothing state characteristics, electric drilling physical residual trend terms, drilling pressure actions and rotation speed actions, wherein the drilling pressure actions and rotation speed actions include historical executed actions in the experience cache or candidate actions of the current training round; Convert the drilling pressure action into a normalized value according to the allowable adjustment range of drilling pressure, and convert the rotation speed action into a normalized value according to the allowable adjustment range of rotation speed. The drilling smoothing state characteristics, normalized values ​​of drilling pressure and rotation speed are concatenated according to the preset field order to generate a joint vector of state and action. The joint state-action vector is input into the KAN value mapping layer. The KAN value mapping layer performs nonlinear mapping on the joint state-action vector through the spline edge activation function composed of B spline basis functions, and reads the spline control points and spline coefficients according to the node intervals where each field in the joint state-action vector is located, and calculates the value edge mapping value. Multiple value edge mapping values ​​are weighted and summed to generate intermediate value features. The intermediate value features are then weighted and summed through the Q-value output layer to generate the current Q-value. By performing second-order analytical differentiation on the spline edge activation function in the KAN value mapping layer through the curvature extraction layer, the second derivative of the current Q value with respect to the drilling smooth state feature is extracted, and a set of second derivatives of the Q value surface is generated. The target Q value is generated based on the drilling smoothing state characteristics, drilling pressure action, rotation speed action, and reward signal at the next sampling time, and the difference between the target Q value and the current Q value is used as the time series difference error. The value network training loss is generated based on the temporal difference error, the trend term of the physical residual of electric drilling, and the set of second derivatives of the Q-value surface. The KAN-Critic value network is then embedded according to the value network training loss update mechanism to generate a smooth value assessment result.

[0013] Optionally, the update mechanism embedded in the KAN-Critic value network specifically includes: The second derivative set of Q-value surface is matched with the physical residual trend term of electric drive drilling through the field matching table. Specifically, the second derivative of the current Q-value relative to the smoothing feature of the electric drive response is matched with the torsional fluctuation residual component, the second derivative of the current Q-value relative to the smoothing feature of the rock breaking process is matched with the rock breaking specific energy residual component, and the change of the second derivative of Q-value surface within adjacent well depth intervals is matched with the residual change rate component and the formation transition component. The second derivative of the matched Q-value surface is subtracted from the corresponding trend term of the physical residual of electric drilling to generate the physical residual constraint deviation. The physical residual constraint deviation is then squared to generate the physical residual second derivative penalty term. The weighted terms of the temporal difference error squared term and the physical residual second derivative penalty term are added together to generate the value network training loss; The spline control points, spline coefficients, and Q-value output layer parameters in the KAN value mapping layer are updated via backpropagation. The updated current Q value, value gradient, set of second derivatives of Q value surface, physical residual constraint bias, and value assessment credibility are written into the same result record according to the current sampling time scale and well depth interval to generate a smooth value assessment result.

[0014] Optionally, step six specifically includes: Read the value gradient, Q-value surface second derivative set, physical residual constraint bias and value assessment credibility from the smoothing value assessment results, and input the drilling smoothing state characteristics into the KAN strategy mapping layer; The KAN policy mapping layer maps drilling smoothing state features to a policy using spline edge activation functions composed of B-spline basis functions, and updates spline control points and spline coefficients in the KAN policy mapping layer according to the value gradient to generate intermediate policy features. The action mean output layer generates the mean drill pressure and mean rotation speed based on the intermediate features of the strategy, and the action variance output layer generates the drill pressure exploration variance and rotation speed exploration variance based on the intermediate features of the strategy and through non-negative mapping. The safety margin adjustment layer extracts the spline second derivatives of the mean drilling pressure and mean rotation speed with respect to the characteristics of the drilling smooth state by performing second-order analytical differentiation on the spline edge activation function in the KAN strategy mapping layer. The absolute value of the second derivative of the spline, the set of second derivatives of the Q-value surface, and the deviation of the physical residual constraint are normalized and then weighted and summed to generate the safety margin of the second derivative of the spline. The safety margin of the second derivative of the spline is compared with the first safety margin threshold and the second safety margin threshold. When the safety margin of the second derivative of the spline is less than or equal to the first safety margin threshold, the drilling pressure exploration variance and the rotation speed exploration variance are maintained. When the safety margin of the second derivative of the spline is greater than the first safety margin threshold and less than or equal to the second safety margin threshold, a variance shrinkage coefficient is generated based on the difference between the safety margin of the second derivative of the spline and the second safety margin threshold, and the drilling pressure exploration variance and rotation speed exploration variance are reduced according to the variance shrinkage coefficient. When the safety margin of the second derivative of the spline is greater than the second safety margin threshold, the exploration variance of drilling pressure and the exploration variance of rotation speed will be adjusted to the preset minimum exploration variance. Based on the torsional fluctuation residual component, rock breaking energy residual component, residual change rate component and formation transition component in the physical residual trend term of electric drilling, the variance shrinkage coefficient is corrected. When the credibility of the value assessment is lower than the preset credibility threshold, the drilling pressure exploration variance and rotation speed exploration variance are reduced, and the mean drilling pressure and mean rotation speed are smoothly rolled back to the effective action value at the previous sampling time. Based on the allowable adjustment range of drilling pressure and rotation speed, the mean values ​​of drilling pressure and rotation speed are checked for boundaries. The drilling pressure action distribution, rotation speed action distribution, mean value of drilling pressure, mean value of rotation speed, exploration variance of drilling pressure, exploration variance of rotation speed, safety margin of spline second derivative, variance shrinkage coefficient, and action boundary marker after the variance dynamic adjustment are written into the same strategy record to generate the strategy distribution of drilling pressure and rotation speed.

[0015] Optionally, step seven specifically includes: Read the drilling pressure action distribution, rotation speed action distribution, mean drilling pressure, mean rotation speed, spline second derivative safety margin, variance shrinkage coefficient, and action boundary markers from the strategy distribution, and read the effective drilling pressure and effective rotation speed issued in the previous control cycle; When the safety margin of the second derivative of the spline is less than or equal to the first safety margin threshold, candidate drilling pressure and candidate rotation speed are generated based on the drilling pressure action distribution and rotation speed action distribution. When the safety margin of the second derivative of the spline is greater than the first safety margin threshold, the average drilling pressure and the average rotation speed are used as candidate drilling pressure and candidate rotation speed. Based on the action boundary markers, the candidate drilling pressure and candidate rotation speed are checked to obtain the target drilling pressure and target rotation speed. The target drilling pressure and target rotation speed are subjected to first-order low-pass filtering and rate limiting to obtain the executable target drilling pressure and executable target rotation speed. The executable target speed is written into the speed setting command of the frequency converter and sent to the electric drive frequency converter; the executable target drilling pressure is written into the drilling pressure adjustment command and sent to the drilling pressure control execution unit. Within the current control cycle, mechanical drilling speed, torque, and riser pressure are collected to form a control execution feedback sequence, and a reward signal is generated based on the control execution feedback sequence. The control execution feedback sequence, reward signal, and drilling smoothing state characteristics of the next control cycle are written into the experience buffer. The KAN-Critic value network and Actor policy network are then embedded based on the sample update mechanism in the experience buffer.

[0016] The beneficial effects of this invention are: (1) This invention solves the problem that existing electric drive drilling parameter optimization is only adjusted based on real-time wellhead data and the downhole drill bit state is not synchronized with the control data by inputting drilling state sample frames into a Smith predictor based on linear torsional equations and simultaneously compensating for the execution delay of the electric drive frequency converter and the propagation delay of the drill string elastic wave. As a result, it can generate a downhole aligned state that is closer to the actual downhole response, reduce the lag judgment in the adjustment of drilling pressure and rotation speed, and enable the subsequent reinforcement learning strategy to make decisions based on the aligned downhole state. This has important engineering significance for the electric drive closed-loop control of deep wells, long drill strings and complex well sections. (2) This invention generates continuously differentiable drilling smooth state features through a KAN state encoder, and constructs physical residual trend terms for electric-driven drilling based on the drill string torsional fluctuation equation and the rock breaking energy equation. This solves the problem that ordinary mechanical energy optimization or conventional neural networks cannot characterize torsional fluctuations, rock breaking energy consumption, and continuous changes in formation transition. This processing unifies the rotational speed response, torque response, downhole drilling pressure, mechanical drilling speed, and lithological changes into residual trend information that can participate in constraints, improving the stability of state characterization under scenarios of abrupt formation changes, torque fluctuations, and abnormal rock breaking efficiency. (3) This invention embeds the KAN-Critic value network into a mechanism, matching the set of second derivatives of the Q-value surface with the trend term of the physical residuals in electric drilling, generating a penalty term for the second derivative of the physical residuals, and dynamically adjusting the exploration variance of drilling pressure and rotation speed in the Actor policy network based on the safety margin of the second derivative of the spline. This solves the problems of overexploration, sudden action changes, and weak physical constraints that are common in existing reinforcement learning drilling control. This scheme can actively shrink the exploration range in high residual well sections and smoothly issue the target drilling pressure and target rotation speed after first-order low-pass filtering and rate limiting, which is beneficial to improving the safety, continuity, and adaptive optimization capability of electric drilling control. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of an adaptive optimization method for electric drilling parameters based on reinforcement learning proposed in this invention; Figure 2 This is a schematic diagram of the Smith predictor based on the linear torsion equation in the adaptive optimization method for electric drilling parameters based on reinforcement learning proposed in this invention. Figure 3 This is a framework diagram of the KAN-Critic value network in the adaptive optimization method for electric drilling parameters based on reinforcement learning proposed in this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0019] refer to Figures 1-3 An adaptive optimization method for electric drilling parameters based on reinforcement learning includes the following steps: Step 1: Collect operating data of the electric drive top drive frequency converter, wellhead sensor data and logging data, perform data preprocessing, and generate drilling status sample frames; Step 2: Input the drilling state sample frame into the Smith predictor based on the linear torsion equation to compensate for the execution delay of the electric drive frequency converter and the propagation delay of the drill string elastic wave, and generate the downhole alignment state. Step 3: Input the downhole alignment state into the KAN state encoder, and generate continuous and differentiable drilling smooth state features through the spline edge activation function; Step 4: Generate the physical residual trend term of electric drilling based on the drill string torsional wave equation and the rock fracture energy equation; Step 5: Embed the drilling smoothing state characteristics, drilling pressure action, and rotation speed action input mechanism into the KAN-Critic value network, and generate smoothing value evaluation results based on the constraints of the time series difference error and the trend term of the electric drilling physical residual on the second derivative of the Q-value surface. Step 6: Input the smoothed value assessment results into the Actor strategy network to generate the strategy distribution of drilling pressure and rotation speed, and dynamically adjust the exploration variance of the strategy distribution according to the safety margin of the second derivative of the spline. Step 7: Generate target drilling pressure and target rotation speed based on the strategy distribution, and then send them out for execution after first-order low-pass filtering and rate limiting processing. The KAN-Critic value network and Actor strategy network are embedded based on the feedback update mechanism of mechanical drilling speed, torque and standpipe pressure.

[0020] In this embodiment, step one specifically includes: At the electric drive drilling site, the operating data of the electric drive top drive frequency converter is read from the electric drive top drive control cabinet, the wellhead sensor data is read from the wellhead sensor group, and the logging data is read from the logging system. The operating data of the electric drive top drive frequency converter includes the top drive motor speed, top drive output torque, frequency converter load rate and operating status mark. The wellhead sensor data includes drilling pressure, standpipe pressure, well depth and mechanical drilling speed. The logging data includes lithology type, formation position, well depth mark and drilling condition mark. Using the sampling timescale of the electric drive top drive frequency converter operation data as the main timescale, the wellhead sensor data is incorporated into the main timescale sequence according to the sampling timescale, and the logging data is incorporated into the main timescale sequence of the corresponding well depth interval according to the well depth identifier, forming the original drilling state sequence. The original drilling state sequence consists of the electric drive top drive frequency converter operation data, wellhead sensor data, and logging data arranged in time. The original drilling status sequence is preprocessed to convert continuous sampling data into continuous variables with uniform dimensions. Lithology, formation position, operational status markers, and drilling condition markers are converted into discrete coded variables. Confidence markers are written for sampling points with missing data, equipment malfunctions, or logging delays. Confidence markers include normal markers and low confidence markers. Specifically, the confidence weight for normal markers is 1.0, and the confidence weight for low confidence markers is 0.3. The original drilling state sequence after data preprocessing is slid segmented according to the preset time window. Continuous variables, discrete coded variables, well depth intervals and confidence markers within the same time window are written into the same frame of data in chronological order to generate drilling state sample frames. The drilling state sample frames include drilling pressure sequence, rotation speed sequence, torque sequence, standpipe pressure sequence, mechanical drilling speed sequence, lithology coding sequence and confidence marker sequence.

[0021] In this embodiment, step two specifically includes: Read the rotation speed sequence, torque sequence, drilling pressure sequence, standpipe pressure sequence, mechanical drilling rate sequence, well depth range and confidence marker sequence from the drilling status sample frame, and read the drill string assembly parameters for the current well section. The drill string assembly parameters include drill string length, drill string shear modulus, drill string material density, drill string equivalent moment of inertia, drill string equivalent torsional stiffness and drill string equivalent damping coefficient. The execution time delay of the electric drive inverter is determined by the difference between the time scale of the control command issued by the electric drive inverter and the time scale of the top drive motor speed starting to change. The propagation velocity of the torsional elastic wave in the drill string is calculated based on the drill string length, drill string shear modulus, and drill string material density. Then, the propagation time delay of the elastic wave in the drill string is determined based on the drill string length and the propagation velocity of the torsional elastic wave in the drill string. The execution time delay of the electric drive frequency converter and the propagation time delay of the drill string elastic wave are added together to obtain the total time delay, and the total time delay is written into the prediction time parameter of the Smith predictor. The top drive end speed, downhole drill bit end speed, torsional angle difference between the top drive end and the downhole drill bit end, downhole torque, downhole drilling pressure and standpipe pressure are used as state variables, and the speed setting change and drilling pressure setting change are used as control variables. A linear torsional state equation is established according to the drill string equivalent moment of inertia, drill string equivalent torsional stiffness and drill string equivalent damping coefficient. The linear torsional state equation is discretized according to the sampling period to form the state prediction equation; Based on the state prediction equation, a time-delay-free prediction branch and a time-delay prediction branch are established. The time-delay-free prediction branch calculates the downhole prediction state after the total time delay forward from the current sampling time, while the time-delay prediction branch calculates the time-delay prediction state at the same time scale as the wellhead measured data. The difference between the measured state at the wellhead and the predicted state with time delay is used to obtain the prediction error. The prediction error is then weighted and corrected according to the confidence level. Specifically, when the confidence level is normal, the prediction error correction weight is 1.0, and when the confidence level is low, the prediction error correction weight is 0.3. The prediction error is multiplied by the corresponding prediction error correction weight to obtain the weighted prediction error. The downhole predicted state obtained from the time-delay prediction branch is superimposed with the weighted and corrected prediction error to generate the downhole aligned state. The downhole aligned state includes the downhole rotation speed estimate, downhole torque estimate, downhole drilling pressure estimate, standpipe pressure estimate, mechanical drilling rate estimate, alignment timescale, well depth range, and confidence marker.

[0022] In this embodiment, step three specifically includes: Read the estimated downhole rotation speed, estimated downhole torque, estimated downhole drilling pressure, estimated standpipe pressure, estimated mechanical drilling rate, well depth range, and confidence marker from the downhole alignment status, and read the lithology code corresponding to the same well depth range from the drilling status sample frame; The rate of change of rotational speed is obtained by dividing the difference between the estimated downhole rotational speeds at two adjacent aligned time points by the time interval between the two aligned time points; the rate of change of torque is obtained by dividing the difference between the estimated downhole torques at two adjacent aligned time points by the time interval between the two aligned time points; the rate of change of drilling pressure is obtained by dividing the difference between the estimated downhole drilling pressures at two adjacent aligned time points by the time interval between the two aligned time points; the rate of change of standpipe pressure is obtained by dividing the difference between the estimated standpipe pressures at two adjacent aligned time points by the time interval between the two aligned time points; the rate of change of mechanical drilling speed is obtained by dividing the difference between the estimated mechanical drilling speeds at two adjacent aligned time points by the time interval between the two aligned time points. The well depth interval number is determined according to the starting well depth of the current well section and the preset well depth step size, and the well depth interval number is written into the well depth interval code; specifically, the preset well depth step size is 10m, and the current well depth interval number is obtained by dividing the difference between the current well depth and the starting well depth of the current well section by 10m and then rounding down. The lithology categories are converted into lithology codes according to the preset lithology coding table; specifically, sandstone, mudstone, limestone and coal seam correspond to different discrete coding values; Normal tags are converted to normal confidence weights, and low confidence tags are converted to low confidence weights; specifically, the normal confidence weight is 1.0, and the low confidence weight is 0.3. Write the downhole rotation speed estimate, downhole torque estimate, downhole drilling pressure estimate, standpipe pressure estimate, mechanical drilling rate estimate, well depth interval code, lithology code, rotation speed change rate, torque change rate, drilling pressure change rate, standpipe pressure change rate, mechanical drilling rate change rate, and confidence weight under the same aligned time scale into the KAN input vector according to the preset field order. The KAN state encoder consists of a state input layer, a channel grouping layer, a spline edge mapping layer, a derivative extraction layer, and a state feature output layer. The state input layer receives the KAN input vector according to a preset field order; The channel grouping layer splits the KAN input vector into an electric drive response state channel, a rock breaking state channel, and a sample weight channel through a field index table. The electric drive response state channel contains the estimated downhole rotation speed, estimated downhole torque, estimated downhole drilling pressure, estimated standpipe pressure, and corresponding rate of change. The rock breaking state channel contains the estimated downhole drilling pressure, estimated mechanical drilling rate, well depth interval code, lithology code, rate of change of drilling pressure, and rate of change of mechanical drilling rate. The sample weight channel contains the confidence weight. The spline edge mapping layer performs edge mapping on each channel field through spline edge activation functions composed of B-spline basis functions, spline nodes, spline control points, and spline coefficients. Each input field corresponds to at least one spline edge mapping channel. Spline nodes are divided according to the field value range. Spline control points record the response amplitude of each node interval. Spline coefficients record the weight of each spline control point in the current mapping. When the input field falls into the corresponding spline node interval, the spline edge mapping layer reads the adjacent spline control points and corresponding spline coefficients of the interval, calculates the edge mapping value of the input field in the current node interval, and weights and sums the multiple edge mapping values ​​received by the same output node to obtain the node response value. The derivative extraction layer extracts the first and second derivative responses of the node response values ​​relative to the input field by analytically differentiating the spline edge activation function. The node response values ​​are used as the state smoothing response, the first derivative response is used as the state change trend, and the second derivative response is used as the state curvature change. The state feature output layer generates smooth drilling state features through weighted summation within channels and splicing between channels. Specifically, the smoothed state response, first-order derivative response, and second-order derivative response of the electric drive response state channel are weighted and summed to generate smoothed electric drive response features. The smoothed state response, first-order derivative response, and second-order derivative response of the rock breaking state channel are weighted and summed to generate smoothed rock breaking process features. The confidence weights in the sample weight channels participate in the above weighted summation. Specifically, the response values ​​corresponding to normal confidence weights participate in the weighted summation by a factor of 1.0, and the response values ​​corresponding to low confidence weights participate in the weighted summation by a factor of 0.3, thus generating smooth drilling state features.

[0023] In this embodiment, step four specifically includes: Read the smooth characteristics of electric drive response, smooth characteristics of rock breaking process, trend of state change, change of state curvature, well depth interval code and lithology code in the smooth state characteristics of drilling, and read the drill bit diameter, equivalent rotational inertia of drill string, equivalent torsional stiffness of drill string and equivalent damping coefficient of drill string corresponding to the current well depth interval. Based on the speed response, torque response, and state curvature change in the smooth characteristics of electric drive response, the torsional fluctuation response of the drill string is formed. The theoretical torsional response for the current well depth range is calculated based on the drill string's equivalent moment of inertia, equivalent torsional stiffness, and equivalent damping coefficient. The drill string torsional fluctuation residual is generated by subtracting the drill string torsional fluctuation response from the theoretical torsional response. The actual rock-breaking specific energy is calculated based on the downhole torque and downhole speed in the smooth characteristics of the electric drive response, and the downhole drilling pressure, mechanical drilling speed, drill bit diameter and lithology code in the smooth characteristics of the rock-breaking process. Read the benchmark rock breaking energy corresponding to the current lithology code, and calculate the difference between the actual rock breaking energy and the benchmark rock breaking energy to generate the rock breaking energy residual. Based on the continuous arrangement of drill string torsional fluctuation residuals and rock fracturing energy residuals within well depth intervals, the residual change direction, residual change amplitude, and residual change rate of adjacent well depth intervals are calculated. Specifically, the residual value of the next well depth interval is subtracted from the residual value of the previous well depth interval to obtain the signed residual change amount. The sign of the signed residual change amount is taken as the residual change direction, the absolute value of the signed residual change amount is taken as the residual change amplitude, and the residual change amplitude is divided by the distance between adjacent well depth intervals to obtain the residual change rate. When the lithology code changes between adjacent well depths, a formation transition marker is written, and the formation transition marker is associated with the direction, magnitude, and rate of change of the residual. Specifically, when the lithology code changes, the formation transition marker is set to 1, and when the lithology code does not change, the formation transition marker is set to 0. The dimensions of the drill string torsional fluctuation residual, rock breaking energy residual, residual change rate, and formation transition marker are unified and the amplitude is normalized to generate the electric drive drilling physical residual trend term. The electric drive drilling physical residual trend term includes the torsional fluctuation residual component, rock breaking energy residual component, residual change rate component, and formation transition component.

[0024] In this embodiment, step five specifically includes: Read drilling smoothing state characteristics, electric drilling physical residual trend terms, drilling pressure actions and rotation speed actions. Drilling pressure actions and rotation speed actions include historical executed actions in the experience buffer or candidate actions in the current training round. The drilling pressure action is converted into a normalized value according to the allowable adjustment range of drilling pressure, and the rotation speed action is converted into a normalized value according to the allowable adjustment range of rotation speed. The drilling smoothing state characteristics, the normalized value of drilling pressure action, and the normalized value of rotation speed action are concatenated according to the preset field order to generate a state action joint vector. The mechanism embedded in the KAN-Critic value network includes a state-action input layer, a KAN value mapping layer, a curvature extraction layer, a physical residual constraint layer, a Q-value output layer, and a loss update layer. The state-action input layer receives the joint state-action vector; The KAN value mapping layer performs nonlinear mapping on the joint vector of state and action through a spline edge activation function composed of B-spline basis functions. Each state field and action field corresponds to a set of spline nodes, spline control points and spline coefficients. The KAN value mapping layer reads the spline control points and spline coefficients according to the node interval of each field in the joint vector of state and action, calculates the value edge mapping value, and generates the value intermediate feature through multi-node weighted summation. The Q-value output layer generates the current Q-value by weighted summation of intermediate value features through fully connected output nodes, and generates the target Q-value based on the drilling smoothing state features, drilling pressure action, rotation speed action and reward signal at the next sampling time. The difference between the target Q-value and the current Q-value is then used as the time-series difference error. The curvature extraction layer extracts the second derivative of the current Q value with respect to the drilling smoothing state by performing second-order analytical differentiation on the spline edge activation function in the KAN value mapping layer, generating a set of second derivatives of the Q value surface. The set of second derivatives of the Q value surface includes the second derivative of the current Q value with respect to the smoothing characteristics of the electric drive response, the second derivative of the current Q value with respect to the smoothing characteristics of the rock breaking process, and the change in the second derivative of the Q value surface within adjacent well depth intervals. The physical residual constraint layer matches the set of second derivatives of the Q-value surface with the physical residual trend term of electric drive drilling through a field matching table. Specifically, the second derivative of the current Q-value relative to the smoothing feature of the electric drive response matches the torsional fluctuation residual component, the second derivative of the current Q-value relative to the smoothing feature of the rock breaking process matches the rock breaking specific energy residual component, and the change in the second derivative of the Q-value surface within adjacent well depth intervals matches the residual change rate component and the formation transition component. The physical residual constraint layer subtracts the second derivative of the matched Q-value surface from the corresponding electric drilling physical residual trend term to generate a physical residual constraint deviation, and then squares the physical residual constraint deviation to generate a physical residual second derivative penalty term. The loss update layer adds the weighted terms of the temporal difference squared error term and the physical residual second derivative penalty term to generate the value network training loss; specifically, the weight of the temporal difference squared error term is 0.7, and the weight of the physical residual second derivative penalty term is 0.3. The spline control points, spline coefficients, and Q-value output layer parameters in the KAN value mapping layer are updated via backpropagation. The updated mechanism is embedded into the current Q-value, value gradient, set of second derivatives of the Q-value surface, physical residual constraint bias, and value assessment confidence level output by the KAN-Critic value network. These are written into the same result record according to the current sampling time scale and well depth interval to generate a smooth value assessment result. Specifically, the value assessment confidence level is jointly determined by the normalized value of the time series difference error and the normalized value of the physical residual constraint bias. Specifically, the normalized value of the time series difference error is multiplied by 0.5, the normalized value of the physical residual constraint bias is multiplied by 0.5, the two are added together to obtain the confidence level, and then the confidence level is obtained by subtracting the confidence level from 1. When the calculation result is less than 0, it is taken as 0, and when the calculation result is greater than 1, it is taken as 1.

[0025] In this embodiment, step six specifically includes: Read the current Q value, value gradient, set of second derivatives of Q value surface, physical residual constraint bias and value assessment credibility from the smoothing value assessment results, and read the drilling smoothing state characteristics; The Actor policy network consists of a state input layer, a KAN policy mapping layer, an action mean output layer, an action variance output layer, a safety margin adjustment layer, and an action boundary mapping layer. The state input layer receives drilling smoothing state characteristics; The KAN policy mapping layer maps drilling smoothing state features to a policy using spline edge activation functions composed of B-spline basis functions. Each state field corresponds to a set of spline nodes, spline control points, and spline coefficients. The KAN policy mapping layer calculates the policy edge mapping value based on the field values ​​of the drilling smoothing state features and updates the spline control points and spline coefficients in the KAN policy mapping layer according to the value gradient. The intermediate policy features are generated by weighted summation of nodes. The action mean output layer generates the average drilling pressure and average rotation speed by weighted summing of the intermediate features of the strategy through the first output node group; The action variance output layer performs a weighted summation of the intermediate features of the strategy through the second output node group, and generates the drilling pressure exploration variance and rotation speed exploration variance through Softplus non-negative mapping; The safety margin adjustment layer extracts the spline second derivatives of the mean drilling pressure and mean rotation speed with respect to the characteristics of the drilling smooth state by performing second-order analytical differentiation on the spline edge activation function in the KAN strategy mapping layer. The absolute value of the second derivative of the spline, the set of second derivatives of the Q-value surface, and the deviation of the physical residual constraint are normalized and then weighted and summed to generate the safety margin of the second derivative of the spline. Specifically, the weight of the normalization term of the absolute value of the second derivative of the spline is 0.4, the weight of the normalization term of the set of second derivatives of the Q-value surface is 0.3, and the weight of the normalization term of the deviation of the physical residual constraint is 0.3. The safety margin of the second derivative of the spline is compared with the first safety margin threshold and the second safety margin threshold. The first safety margin threshold is less than the second safety margin threshold. Specifically, the first safety margin threshold is 0.35 and the second safety margin threshold is 0.70. When the safety margin of the second derivative of the spline is less than or equal to the first safety margin threshold, the variance shrinkage coefficient is set to 1.0 to keep the drilling pressure exploration variance and rotation speed exploration variance at their current values. When the safety margin of the second derivative of the spline is greater than the first safety margin threshold and less than or equal to the second safety margin threshold, a variance shrinkage coefficient is generated according to the proportion of the difference between the safety margin of the second derivative of the spline and the first safety margin threshold to the length of the interval between the first safety margin threshold and the second safety margin threshold. Specifically, the variance shrinkage coefficient varies linearly between 1.0 and 0.3, and the drilling pressure exploration variance and the rotation speed exploration variance are multiplied by the variance shrinkage coefficient respectively. When the safety margin of the second derivative of the spline is greater than the second safety margin threshold, the variance shrinkage coefficient is set to 0.1, and the drilling pressure exploration variance and rotation speed exploration variance are adjusted to 10% of the initial exploration variance, respectively. Based on the torsional fluctuation residual component, rock breaking energy residual component, residual change rate component, and formation transition component in the physical residual trend term of electric drilling, the variance shrinkage coefficient is corrected. Specifically, when the normalized value of any component is greater than 0.6 and less than or equal to 0.8, the variance shrinkage coefficient is multiplied by 0.7; when the normalized value of any component is greater than 0.8, the variance shrinkage coefficient is multiplied by 0.5. When the credibility of the value assessment is lower than the preset credibility threshold, the variance of drilling pressure exploration and the variance of rotation speed exploration are multiplied by 0.5 respectively, and the mean of drilling pressure and the mean of rotation speed are smoothed back according to the proportion of the effective action value at the previous sampling time as 0.7 and the mean of current drilling pressure or current rotation speed as 0.3; specifically, the preset credibility threshold is 0.6; The action boundary mapping layer performs boundary checks on the mean drill pressure and mean rotation speed based on the allowable adjustment range of drill pressure and rotation speed. After completing the variance dynamic adjustment, the drill pressure action distribution, rotation speed action distribution, mean drill pressure, mean rotation speed, drill pressure exploration variance, rotation speed exploration variance, spline second derivative safety margin, variance shrinkage coefficient, and action boundary marker are written into the same strategy record to generate the strategy distribution of drill pressure and rotation speed.

[0026] In this embodiment, step seven specifically includes: Read the drilling pressure action distribution, rotation speed action distribution, mean drilling pressure, mean rotation speed, spline second derivative safety margin, variance shrinkage coefficient, and action boundary markers from the strategy distribution, and read the effective drilling pressure and effective rotation speed issued in the previous control cycle; When the safety margin of the second derivative of the spline is less than or equal to the first safety margin threshold, candidate drilling pressure and candidate rotation speed are generated based on the drilling pressure action distribution and rotation speed action distribution. When the safety margin of the second derivative of the spline is greater than the first safety margin threshold, the average drilling pressure and the average rotation speed are used as candidate drilling pressure and candidate rotation speed. Based on the action boundary markers, the candidate drilling pressure and candidate rotation speed are checked to obtain the target drilling pressure and target rotation speed. The target drilling pressure and target rotation speed are subjected to a first-order low-pass filter to ensure that the target drilling pressure and target rotation speed change continuously relative to the effective drilling pressure and effective rotation speed of the previous control cycle. Specifically, the first-order low-pass filter generates the filtered target drilling pressure and target rotation speed according to the ratio of 0.7 of the effective action value of the previous control cycle and 0.3 of the current target action value. The target drill pressure and target rotation speed after low-pass filtering are rate-limited. Based on the current control cycle length, the allowable change in drill pressure per unit time, and the allowable change in rotation speed per unit time, the maximum change in target drill pressure and target rotation speed within the current control cycle is limited to obtain the executable target drill pressure and executable target rotation speed. Specifically, the allowable change in drill pressure per unit time is 1 kN / s, and the allowable change in rotation speed per unit time is 2 r / min / s. According to the data format of the electric drive drilling control interface, the executable target speed is written into the inverter speed setting command, the executable target drilling pressure is written into the drilling pressure adjustment command, the inverter speed setting command is sent to the electric drive inverter, and the drilling pressure adjustment command is sent to the drilling pressure control execution unit. Within the current control cycle, mechanical drilling speed, torque, and riser pressure are collected to form a control execution feedback sequence; A reward signal is generated based on the control execution feedback sequence. Specifically, the positive normalized value of the rate of change of mechanical drilling speed is multiplied by 0.5, the normalized value of torque fluctuation is multiplied by 0.2, the normalized value of riser pressure fluctuation is multiplied by 0.2, the normalized value of execution deviation is multiplied by 0.1, and the corresponding items of torque fluctuation, riser pressure fluctuation and execution deviation are subtracted from the corresponding item of mechanical drilling speed to obtain the reward signal. The control execution feedback sequence, reward signal, and drilling smoothing state characteristics of the next control cycle are written into the experience buffer. The time difference error and physical residual second derivative penalty terms are calculated based on the samples in the experience buffer, and the spline control points, spline coefficients, and Q-value output layer parameters in the mechanism embedded KAN-Critic value network are updated. Based on the updated smoothed value assessment results and value gradient, update the spline control points, spline coefficients, action mean output parameters, and action variance output parameters in the Actor policy network; When torque fluctuation exceeds the torque fluctuation threshold, riser pressure fluctuation exceeds the riser pressure fluctuation threshold, or mechanical drilling rate decrease exceeds the mechanical drilling rate decrease threshold in the control execution feedback sequence, an anomaly masking flag is written for the corresponding sample, and the sample weight of the corresponding sample in the network update is set to 0.2; when the control execution feedback sequence does not have the above situations, the sample weight of the corresponding sample in the network update is set to 1.0. The executable target drilling pressure, executable target rotation speed, control execution feedback sequence, reward signal, network update parameters, anomaly masking flag, current sampling time scale and well depth range are written into the current control cycle record to complete the smooth execution of target drilling pressure and target rotation speed, as well as the feedback update of the mechanism embedded in the KAN-Critic value network and Actor strategy network.

[0027] Example 1: To verify the feasibility of this invention in practice, it was applied to the drilling control scenario of a three-section well in an electrically driven top-drive horizontal well. The well depth was 2860m to 3160m, the drill bit diameter was 215.9mm, the formation consisted mainly of alternating sandstone and mudstone, and the drill string assembly length was 1820m. On-site, the top-drive motor speed, top-drive output torque, and inverter load rate were read from the electric drive top-drive control cabinet; drill pressure, standpipe pressure, well depth, and mechanical drilling rate were read from the wellhead sensor group; and lithology and drilling condition markers were read from the logging system. The sampling period was set to 1s, the sliding time window to 30s, the well depth interval step size to 10m, the normal confidence weight to 1.0, and the low confidence weight to 0.3.

[0028] In application, drilling status sample frames are input into the Smith predictor based on the linear torsional equation. The execution delay of the electric drive inverter is determined by the control command issuance time and the start of the top drive motor speed change time, which is statistically 1.6s in the field. The drill string elastic wave propagation delay is determined to be 3.8s based on the drill string length, drill string shear modulus, and drill string material density. The total delay is written into the prediction time parameter of the Smith predictor. The KAN state encoder uses a spline edge activation function composed of B-spline basis functions, with a first safety margin threshold of 0.35 and a second safety margin threshold of 0.70. The weight of the temporal differential error square term is 0.7, and the weight of the physical residual second derivative penalty term is 0.3. In the first-order low-pass filter, the effective action value of the previous control cycle accounts for 0.7, the current target action value accounts for 0.3, the allowable change in drilling pressure per unit time is 1kN / s, and the allowable change in rotational speed per unit time is 2r / min / s. The same well section was compared using the proposed method, method A, method B, and method C respectively. Method A is a VFD parameter optimization method based on mechanical specific energy, method B is a conventional Actor-Critic drilling parameter control method, and method C is a KAN-Critic method without adding a physical residual second derivative penalty term and spline second derivative safety margin adjustment.

[0029] Table 1. Comparison of different control methods in the same electrically driven drilling section

[0030] As shown in Table 1, the average mechanical drilling rate of this method reaches 11.3 m / h, which is approximately 29.9% higher than that of Method A, approximately 20.2% higher than that of Method B, and approximately 11.9% higher than that of Method C. This result indicates that while relying solely on mechanical specific energy or ordinary reinforcement learning can adjust drilling pressure and rotational speed, control actions are still prone to lag when there are time delays in downhole conditions and significant formation alternation. This method incorporates downhole alignment state and drilling smoothing state features into the decision-making process, making the state input obtained by the Actor policy network closer to the actual downhole response.

[0031] In terms of stability, the torque fluctuation rate of this method is 8.9%, and the riser pressure fluctuation is 1.4 MPa, both significantly lower than other methods. The number of drill pressure / speed exceedances is reduced to 1 per 100m, and the average execution deviation is reduced to 4.6%. This indicates that the penalty term of the second derivative of the physical residual can constrain the change of the Q-value surface, so that the value assessment not only pursues short-term mechanical drilling speed, but also considers the drill string torsional fluctuation residual and the rock fracturing energy residual. Although Method C uses KAN-Critic, it lacks the constraint of the second derivative of the Q-value surface by the physical residual trend term of electric drilling, and still suffers from the problem of large operation fluctuations in high residual well sections.

[0032] Regarding recovery capability in high residual well sections, this method achieves a recovery time of 39 seconds, lower than Method A's 96 seconds, Method B's 82 seconds, and Method C's 64 seconds. This is because when the safety margin of the spline's second derivative exceeds a threshold, this method reduces the variance of drilling pressure and rotational speed exploration, and smoothly distributes target drilling pressure and rotational speed through first-order low-pass filtering and rate limiting, thereby avoiding overexploration during formation transitions or torque anomalies. Therefore, this invention can reduce torque fluctuations, standpipe pressure fluctuations, and execution deviations while ensuring increased mechanical drilling speed, demonstrating significant practical value for the safe and adaptive optimization of electric drilling parameters in complex formations.

[0033] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An adaptive optimization method for electric drilling parameters based on reinforcement learning, characterized in that, Includes the following steps: Step 1: Collect operating data of the electric drive top drive frequency converter, wellhead sensor data and logging data, perform data preprocessing, and generate drilling status sample frames; Step 2: Input the drilling state sample frame into the Smith predictor based on the linear torsion equation to compensate for the execution delay of the electric drive frequency converter and the propagation delay of the drill string elastic wave, and generate the downhole alignment state. Step 3: Input the downhole alignment state into the KAN state encoder, and generate continuous and differentiable drilling smooth state features through the spline edge activation function; Step 4: Generate the physical residual trend term of electric drilling based on the drill string torsional wave equation and the rock fracture energy equation; Step 5: Embed the drilling smoothing state characteristics, drilling pressure action, and rotation speed action input mechanism into the KAN-Critic value network, and generate smoothing value evaluation results based on the constraints of the time series difference error and the trend term of the electric drilling physical residual on the second derivative of the Q-value surface. Step 6: Input the smoothed value assessment results into the Actor strategy network to generate the strategy distribution of drilling pressure and rotation speed, and dynamically adjust the exploration variance of the strategy distribution according to the safety margin of the second derivative of the spline. Step 7: Generate target drilling pressure and target rotation speed based on the strategy distribution, and then send them out for execution after first-order low-pass filtering and rate limiting processing. The KAN-Critic value network and Actor strategy network are embedded based on the feedback update mechanism of mechanical drilling speed, torque and standpipe pressure.

2. The adaptive optimization method for electric drilling parameters based on reinforcement learning according to claim 1, characterized in that, The electric drive top drive frequency converter operating data includes top drive motor speed, top drive output torque, frequency converter load rate and operating status mark; the wellhead sensor data includes drilling pressure, standpipe pressure, well depth and mechanical drilling speed; the logging data includes lithology type, formation position, well depth mark and drilling condition mark. Using the sampling timescale of the electric drive top drive frequency converter operation data as the main timescale, the wellhead sensor data is incorporated into the main timescale sequence according to the sampling timescale, and the logging data is incorporated into the main timescale sequence of the corresponding well depth interval according to the well depth identifier to form the original drilling status sequence. The original drilling status sequence is preprocessed to convert continuous sampling data into continuous variables with uniform dimensions, and to convert lithology, formation position, operating status markers and drilling condition markers into discrete coded variables. Confidence markers are written for sampling points with missing data, equipment malfunctions or logging delays. The original drilling state sequence after data preprocessing is slid segmented according to the preset time window. Continuous variables, discrete coded variables, well depth intervals and confidence markers within the same time window are written into the same frame of data in chronological order to generate drilling state sample frames.

3. The adaptive optimization method for electric drilling parameters based on reinforcement learning according to claim 1, characterized in that, Step two specifically includes: Read the rotation speed sequence, torque sequence, drilling pressure sequence, standpipe pressure sequence, mechanical drilling rate sequence, well depth range and confidence marker sequence from the drilling status sample frame, and read the drill string assembly parameters for the current well section; The execution time delay of the electric drive frequency converter is determined based on the control response process of the electric drive frequency converter, and the propagation time delay of the drill string elastic wave is determined based on the drill string assembly parameters. The execution time delay of the electric drive frequency converter and the propagation time delay of the drill string elastic wave are added together to obtain the total time delay, and the total time delay is written into the prediction time parameter of the Smith predictor. A linear torsional state equation is established using the top drive end speed, downhole drill bit end speed, torsional angle difference between the top drive end and the downhole drill bit end, downhole torque, downhole drilling pressure and standpipe pressure as state variables, and the speed setting change and drilling pressure setting change as control variables. The linear torsional state equation is discretized according to the sampling period to form the state prediction equation; Based on the state prediction equation, a time-delay-free prediction branch and a time-delay prediction branch are established. The time-delay-free prediction branch calculates the downhole prediction state after the total time delay forward from the current sampling time, while the time-delay prediction branch calculates the time-delay prediction state at the same time scale as the wellhead measured data. The difference between the measured state at the wellhead and the predicted state with time delay is used to obtain the prediction error, and the prediction error is weighted and corrected according to the confidence level. The downhole predicted state obtained from the time-delay prediction branch is superimposed with the weighted and corrected prediction error to generate the downhole aligned state.

4. The adaptive optimization method for electric drilling parameters based on reinforcement learning according to claim 3, characterized in that, The determination of the electric drive inverter execution time delay based on the control response process of the electric drive inverter, and the determination of the drill string elastic wave propagation time delay based on the drill string assembly parameters, specifically include: The drill string assembly parameters include drill string length, drill string shear modulus, drill string material density, drill string equivalent moment of inertia, drill string equivalent torsional stiffness, and drill string equivalent damping coefficient. The execution time delay of the electric drive inverter is determined by the difference between the time scale of the control command issued by the electric drive inverter and the time scale of the top drive motor speed starting to change. The propagation velocity of the torsional elastic wave in the drill string is calculated based on the drill string length, drill string shear modulus, and drill string material density. Then, the propagation time delay of the elastic wave in the drill string is determined based on the drill string length and the propagation velocity of the torsional elastic wave in the drill string. A linear torsional state equation is established based on the drill string's equivalent moment of inertia, equivalent torsional stiffness, and equivalent damping coefficient. In the Smith predictor, the no-delay prediction branch generates the downhole prediction state based on the total time delay, while the time-delay prediction branch generates the time-delay prediction state with the same time scale as the wellhead measured data. The prediction error is obtained by subtracting the wellhead measured state from the time-delay prediction state. When the confidence level is marked as low confidence, the correction weight for prediction error is reduced; when the confidence level is marked as normal confidence, the correction weight for prediction error is maintained.

5. The adaptive optimization method for electric drilling parameters based on reinforcement learning according to claim 1, characterized in that, Step three specifically includes: Read the estimated downhole rotation speed, estimated downhole torque, estimated downhole drilling pressure, estimated standpipe pressure, estimated mechanical drilling rate, well depth range, and confidence marker in the downhole alignment status, and read the lithology code corresponding to the same well depth range; Based on the differences and time intervals between the estimated downhole rotation speed, estimated downhole torque, estimated downhole drilling pressure, estimated standpipe pressure, and estimated mechanical drilling speed under two adjacent aligned time scales, the rates of change of rotation speed, torque, drilling pressure, standpipe pressure, and mechanical drilling speed are generated respectively. Based on the starting well depth and preset well depth step size of the current well section, the well depth interval is converted into a well depth interval code, and the confidence mark is converted into a confidence weight; Write the downhole rotation speed estimate, downhole torque estimate, downhole drilling pressure estimate, standpipe pressure estimate, mechanical drilling rate estimate, well depth range code, lithology code, rotation speed change rate, torque change rate, drilling pressure change rate, standpipe pressure change rate, mechanical drilling rate change rate, and confidence weight into the KAN input vector. The KAN input vector is split into an electric drive response state channel, a rock breaking state channel, and a sample weight channel using a field index table. The electric drive response state channel includes downhole rotation speed estimates, downhole torque estimates, downhole drill pressure estimates, standpipe pressure estimates, and corresponding rates of change. The rock breaking state channel includes downhole drill pressure estimates, mechanical drilling rate estimates, well depth interval codes, lithology codes, drill pressure change rates, and mechanical drilling rate change rates. The sample weight channel includes confidence weights. Edge mapping is performed on each channel field using spline edge activation functions composed of B-spline basis functions, spline nodes, spline control points, and spline coefficients to generate node response values; The spline edge activation function is analytically differentiated to generate the first-order and second-order state derivative responses, and the node response values ​​are used as the state smoothing responses. The smoothed state response, first-order state response, and second-order state response corresponding to the electric drive response state channel are fused into the smoothed electric drive response feature. The smoothed state response, first-order state response, and second-order state response corresponding to the rock breaking state channel are fused into the smoothed rock breaking process feature. Drilling smoothed state features are generated based on the confidence weight.

6. The adaptive optimization method for electric drilling parameters based on reinforcement learning according to claim 1, characterized in that, Step four specifically includes: Read the smooth characteristics of electric drive response, smooth characteristics of rock breaking process, trend of state change, change of state curvature, well depth interval code and lithology code in the smooth state characteristics of drilling, and read the drill bit diameter, equivalent rotational inertia of drill string, equivalent torsional stiffness of drill string and equivalent damping coefficient of drill string corresponding to the current well depth interval. Based on the rotational speed response, torque response, and state curvature change in the smooth characteristics of the electric drive response, the torsional fluctuation response of the drill string is formed, and the theoretical torsional response is calculated based on the equivalent rotational inertia of the drill string, the equivalent torsional stiffness of the drill string, and the equivalent damping coefficient of the drill string. The drill string torsional fluctuation residual is generated by subtracting the drill string torsional fluctuation response from the theoretical torsional response. The actual rock-breaking specific energy is calculated based on the downhole torque and downhole speed in the smooth characteristics of the electric drive response, and the downhole drilling pressure, mechanical drilling speed, drill bit diameter and lithology code in the smooth characteristics of the rock-breaking process. Read the benchmark rock breaking energy corresponding to the current lithology code, and calculate the difference between the actual rock breaking energy and the benchmark rock breaking energy to generate the rock breaking energy residual. Based on the residuals of torsional fluctuation of the drill string and the residuals of rock breaking energy in a continuous arrangement of well depth intervals, calculate the direction of residual change, the magnitude of residual change, and the rate of residual change in adjacent well depth intervals; When the lithology code changes between adjacent well depths, a formation transition marker is written, and the formation transition marker is associated with the direction of residual change, the magnitude of residual change, and the rate of residual change. The dimensions of the drill string torsional fluctuation residual, rock breaking energy residual, residual change rate, and formation transition marker are unified and the amplitude is normalized to generate an electric-drive drilling physical residual trend term. The electric-drive drilling physical residual trend term includes torsional fluctuation residual components, rock breaking energy residual components, residual change rate components, and formation transition components.

7. The adaptive optimization method for electric drilling parameters based on reinforcement learning according to claim 1, characterized in that, Step five specifically includes: Read drilling smoothing state characteristics, electric drilling physical residual trend terms, drilling pressure actions and rotation speed actions, wherein the drilling pressure actions and rotation speed actions include historical executed actions in the experience cache or candidate actions of the current training round; Convert the drilling pressure action into a normalized value according to the allowable adjustment range of drilling pressure, and convert the rotation speed action into a normalized value according to the allowable adjustment range of rotation speed. The drilling smoothing state characteristics, normalized values ​​of drilling pressure and rotation speed are concatenated according to the preset field order to generate a joint vector of state and action. The joint state-action vector is input into the KAN value mapping layer. The KAN value mapping layer performs nonlinear mapping on the joint state-action vector through the spline edge activation function composed of B spline basis functions, and reads the spline control points and spline coefficients according to the node intervals where each field in the joint state-action vector is located, and calculates the value edge mapping value. Multiple value edge mapping values ​​are weighted and summed to generate intermediate value features. The intermediate value features are then weighted and summed through the Q-value output layer to generate the current Q-value. By performing second-order analytical differentiation on the spline edge activation function in the KAN value mapping layer through the curvature extraction layer, the second derivative of the current Q value with respect to the drilling smooth state feature is extracted, and a set of second derivatives of the Q value surface is generated. The target Q value is generated based on the drilling smoothing state characteristics, drilling pressure action, rotation speed action, and reward signal at the next sampling time, and the difference between the target Q value and the current Q value is used as the time series difference error. The value network training loss is generated based on the temporal difference error, the trend term of the physical residual of electric drilling, and the set of second derivatives of the Q-value surface. The KAN-Critic value network is then embedded according to the value network training loss update mechanism to generate a smooth value assessment result.

8. The adaptive optimization method for electric drilling parameters based on reinforcement learning according to claim 7, characterized in that, The update mechanism embedded in the KAN-Critic value network specifically includes: The second derivative set of Q-value surface is matched with the physical residual trend term of electric drive drilling through the field matching table. Specifically, the second derivative of the current Q-value relative to the smoothing feature of the electric drive response is matched with the torsional fluctuation residual component, the second derivative of the current Q-value relative to the smoothing feature of the rock breaking process is matched with the rock breaking specific energy residual component, and the change of the second derivative of Q-value surface within adjacent well depth intervals is matched with the residual change rate component and the formation transition component. The second derivative of the matched Q-value surface is subtracted from the corresponding trend term of the physical residual of electric drilling to generate the physical residual constraint deviation. The physical residual constraint deviation is then squared to generate the physical residual second derivative penalty term. The weighted terms of the temporal difference error squared term and the physical residual second derivative penalty term are added together to generate the value network training loss; The spline control points, spline coefficients, and Q-value output layer parameters in the KAN value mapping layer are updated via backpropagation. The updated current Q value, value gradient, set of second derivatives of Q value surface, physical residual constraint bias, and value assessment credibility are written into the same result record according to the current sampling time scale and well depth interval to generate a smooth value assessment result.

9. The adaptive optimization method for electric drilling parameters based on reinforcement learning according to claim 1, characterized in that, Step six specifically includes: Read the value gradient, Q-value surface second derivative set, physical residual constraint bias and value assessment credibility from the smoothing value assessment results, and input the drilling smoothing state characteristics into the KAN strategy mapping layer; The KAN policy mapping layer maps drilling smoothing state features to a policy using spline edge activation functions composed of B-spline basis functions, and updates spline control points and spline coefficients in the KAN policy mapping layer according to the value gradient to generate intermediate policy features. The action mean output layer generates the mean drill pressure and mean rotation speed based on the intermediate features of the strategy, and the action variance output layer generates the drill pressure exploration variance and rotation speed exploration variance based on the intermediate features of the strategy and through non-negative mapping. The safety margin adjustment layer extracts the spline second derivatives of the mean drilling pressure and mean rotation speed with respect to the characteristics of the drilling smooth state by performing second-order analytical differentiation on the spline edge activation function in the KAN strategy mapping layer. The absolute value of the second derivative of the spline, the set of second derivatives of the Q-value surface, and the deviation of the physical residual constraint are normalized and then weighted and summed to generate the safety margin of the second derivative of the spline. The safety margin of the second derivative of the spline is compared with the first safety margin threshold and the second safety margin threshold. When the safety margin of the second derivative of the spline is less than or equal to the first safety margin threshold, the drilling pressure exploration variance and the rotation speed exploration variance are maintained. When the safety margin of the second derivative of the spline is greater than the first safety margin threshold and less than or equal to the second safety margin threshold, a variance shrinkage coefficient is generated based on the difference between the safety margin of the second derivative of the spline and the second safety margin threshold, and the drilling pressure exploration variance and rotation speed exploration variance are reduced according to the variance shrinkage coefficient. When the safety margin of the second derivative of the spline is greater than the second safety margin threshold, the exploration variance of drilling pressure and the exploration variance of rotation speed will be adjusted to the preset minimum exploration variance. Based on the torsional fluctuation residual component, rock breaking energy residual component, residual change rate component and formation transition component in the physical residual trend term of electric drilling, the variance shrinkage coefficient is corrected. When the credibility of the value assessment is lower than the preset credibility threshold, the drilling pressure exploration variance and rotation speed exploration variance are reduced, and the mean drilling pressure and mean rotation speed are smoothly rolled back to the effective action value at the previous sampling time. Based on the allowable adjustment range of drilling pressure and rotation speed, the mean values ​​of drilling pressure and rotation speed are checked for boundaries. The drilling pressure action distribution, rotation speed action distribution, mean value of drilling pressure, mean value of rotation speed, exploration variance of drilling pressure, exploration variance of rotation speed, safety margin of spline second derivative, variance shrinkage coefficient, and action boundary marker after the variance dynamic adjustment are written into the same strategy record to generate the strategy distribution of drilling pressure and rotation speed.

10. The adaptive optimization method for electric drilling parameters based on reinforcement learning according to claim 1, characterized in that, Step seven specifically includes: Read the drilling pressure action distribution, rotation speed action distribution, mean drilling pressure, mean rotation speed, spline second derivative safety margin, variance shrinkage coefficient, and action boundary markers from the strategy distribution, and read the effective drilling pressure and effective rotation speed issued in the previous control cycle; When the safety margin of the second derivative of the spline is less than or equal to the first safety margin threshold, candidate drilling pressure and candidate rotation speed are generated based on the drilling pressure action distribution and rotation speed action distribution. When the safety margin of the second derivative of the spline is greater than the first safety margin threshold, the average drilling pressure and the average rotation speed are used as candidate drilling pressure and candidate rotation speed. Based on the action boundary markers, the candidate drilling pressure and candidate rotation speed are checked to obtain the target drilling pressure and target rotation speed. The target drilling pressure and target rotation speed are subjected to first-order low-pass filtering and rate limiting to obtain the executable target drilling pressure and executable target rotation speed. The executable target speed is written into the speed setting command of the frequency converter and sent to the electric drive frequency converter; the executable target drilling pressure is written into the drilling pressure adjustment command and sent to the drilling pressure control execution unit. Within the current control cycle, mechanical drilling speed, torque, and riser pressure are collected to form a control execution feedback sequence, and a reward signal is generated based on the control execution feedback sequence. The control execution feedback sequence, reward signal, and drilling smoothing state characteristics of the next control cycle are written into the experience buffer. The KAN-Critic value network and Actor policy network are then embedded based on the sample update mechanism in the experience buffer.

Images