Rotary steering drilling trajectory prediction method based on deep learning and knowledge distillation

Abstract

The invention discloses a rotary steerable drilling trajectory prediction method based on deep learning and knowledge distillation, and belongs to the technical field of drilling trajectory prediction, and the method comprises the following steps: collecting rotary steerable drilling site ground data and while-drilling well logging data, and carrying out data preprocessing; constructing a teacher model and training; the teacher model comprises a well drilling feature enhancement module and a sequence change prediction module; a teaching assistant model is constructed and trained, and the teacher model is assisted to carry out multi-interlayer data adaptive compression; a composite knowledge distillation framework and a student model are built, the student model is trained, and the trained student model is a lightweight borehole trajectory prediction model; during field application, ground data and logging-while-drilling data are preprocessed and then sent into the lightweight well track prediction model, a well track prediction value in the rotary steering drilling process is obtained, and therefore drilling operation parameters are adjusted in time according to a prediction result. According to the method, high-precision and light-weight well track prediction is realized.

Description

Technical Field

[0001] This invention belongs to the field of drilling trajectory prediction technology, specifically relating to a rotary steering drilling trajectory prediction method based on deep learning and knowledge distillation. Background Technology

[0002] Wellbore trajectory refers to the irregular three-dimensional spatial path formed by the drill bit as it travels between the surface and the reservoir during the drilling process. Its completion quality and efficiency significantly impact the cost and cycle time of the entire exploration and development process. Rotary Steerable Systems (RSS), as the most advanced directional drilling systems, are currently a hot topic in the oil and gas exploration and development field. They typically consist of a downhole rotary control tool subsystem, a downhole measurement and control subsystem, and a surface-to-downhole two-way communication subsystem. Upon receiving control commands, the RSS can flexibly control the drill bit direction by pushing the drill bit or changing the bend angle of the build-up tool structure, thereby precisely controlling the wellbore trajectory in any well section.

[0003] In rotary steerable drilling, accurately predicting wellbore trajectory changes and dynamically adjusting drilling parameters are crucial for achieving precise guidance and efficient drilling. However, current common prediction methods struggle to balance prediction accuracy and model complexity. Pursuing only accuracy at the expense of an overly large model, or using a low-precision model to accommodate downhole deployment limitations, cannot truly meet the edge computing requirements of RSS downhole closed-loop control. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes a method for predicting wellbore trajectories in rotary steerable drilling based on deep learning and knowledge distillation. This method first combines graph representation learning with time series prediction to construct a high-precision teacher model. Then, a novel isomorphic knowledge distillation method called multi-intermediate-layer feature adaptive fusion supervised knowledge distillation is employed to extract intermediate-layer information from the teacher model and perform knowledge transfer. Finally, the intermediate-layer knowledge is combined with the output-layer knowledge to form a composite knowledge distillation architecture to train a lightweight student model. The training process reconstructs the knowledge distillation loss function and optimizes it through a dynamic weight allocation mechanism. The lightweight prediction model obtained by this method performs excellently on the test set, achieving high-precision and lightweight wellbore trajectory prediction in rotary steerable drilling to a certain extent, providing theoretical guidance for the development of downhole closed-loop control in the RSS (Real-Side Surface Mount) field.

[0005] The technical solution of the present invention is as follows: A method for predicting rotary steered drilling trajectories based on deep learning and knowledge distillation includes the following steps: Step 1: Collect surface data and logging-while-drilling data from the rotary steerable drilling site, and perform data preprocessing; Step 2: Construct and train the teacher model; the teacher model includes a drilling feature enhancement module and a sequence change prediction module; Step 3: Build and train the teaching assistant model to assist the teacher model in adaptive compression of multi-intermediate layer data; Step 4: Build a composite knowledge distillation architecture and student model, and train the student model. The trained student model is the lightweight wellbore trajectory prediction model. Step 5: In field application, the ground data and logging-while-drilling data are preprocessed and then fed into the lightweight wellbore trajectory prediction model to obtain the predicted wellbore trajectory value of the rotary steerable drilling process, so as to adjust the drilling operation parameters in a timely manner based on the prediction results.

[0006] Furthermore, the specific process of step 1 is as follows: Step 1.1: Obtain raw drilling data samples including 16 drilling features through surface data recording and measurement-while-drilling tools; clean the raw drilling data samples, sort the cleaned samples according to measurement depth, and stitch together samples from different well sections to form the raw dataset. ,in, For the first in the original dataset One sample; Number of drilling features; The 16 drilling characteristics include drilling pressure, rotation speed, steering force, tool face angle, drilling fluid flow rate, drilling fluid density, equivalent circulating density, standpipe pressure, drilling speed, torque, well inclination angle, azimuth angle, measurement depth, drilling vertical depth, formation temperature, and formation drillability index. Step 1.2: The original dataset is divided into sliding window segments along the measurement depth using the sliding window algorithm; the sliding window is divided into history windows. and prediction window ,pass The continuous variation characteristics of the upper well section were obtained by segmenting the original dataset; through... The original dataset is segmented to obtain the continuous variation features of the lower well section; the continuous variation features of the upper and lower well sections are combined to obtain a serialized dataset containing the continuous variation features of both the upper and lower well sections. ;in, For the serialized dataset, the first Serialized samples; The formula for calculating the number of serialized samples is: (1); in, This represents the sampling interval for the sliding window algorithm. Step 1.3: Divide the serialized dataset into training set, validation set and test set according to the proportion.

[0007] Furthermore, the specific process of step 2 is as follows: Step 2.1: Construct the drilling feature enhancement module; the input to the drilling feature enhancement module is the serialized dataset. The drilling feature enhancement module, as an upstream data enhancement algorithm, is used to capture the external correlations between different drilling parameters at the same location and characterize these external correlations as data patterns that the downstream sequence variation prediction module can recognize. The drilling feature enhancement module is constructed using convolutional neural networks or graph neural networks. When the drilling feature enhancement module uses a graph neural network as its core algorithm, the PyTorch Geometric toolkit is used to serialize the dataset. The tensor format is converted into a graph data format to meet the input data format requirements of the graph neural network, forming the training set, validation set, and test set required for the teacher model; Step 2.2: Construct the sequence variation prediction module. The sequence variation prediction module consists of an attention mechanism, a sequence prediction algorithm, and a linear layer. Specifically, the attention mechanism adopts the dot product attention mechanism. The sequence prediction algorithm is used to receive the drilling data after it has been enhanced by the upstream module and further capture the internal sequence dependencies of the drilling parameters in the upper and lower well sections. The linear layer is used to map the high-dimensional feature tensor output by the sequence prediction algorithm to the dimension required for the prediction target. The input of the sequence variation prediction module is the output of the drilling feature enhancement module, and the output is the predicted value of the wellbore trajectory parameters. Step 2.3: Module Assembly and Model Pre-training; After the drilling feature enhancement module and sequence change prediction module are constructed, the two are assembled according to the specific prediction requirements by selecting either a sequential stacking structure or an encoder-decoder structure network connection method to form a teacher model; when the drilling feature enhancement module uses a graph neural network as the core algorithm, the training set, validation set, and test set formed in Step 2.1 are used to pre-train and evaluate the performance of the teacher model; when the drilling feature enhancement module uses a convolutional neural network or other deep learning algorithms that require vector / tensor input data formats, the training set, validation set, and test set divided in Step 1.3 are directly used to pre-train and evaluate the performance of the teacher model.

[0008] Furthermore, the specific process of step 3 is as follows: Step 3.1: Read and record the hidden information tensors of different intermediate layers of the teacher model to obtain... The shape is Two-dimensional tensor , This represents the number of intermediate layers in the time series prediction algorithm of the teacher model. The number of hidden features in the time series prediction algorithm of the teacher model; the two-dimensional tensor Stacking along the number of intermediate layers forms a shape of 3D intermediate layer hidden information tensor ; Step 3.2: Construct the teaching assistant model; the teaching assistant model consists of a composite attention mechanism module and a convolutional neural network, used to read the 3D hidden information tensor extracted by the teacher model. And adaptive compression of the three-dimensional hidden information tensor; Step 3.3: Train the teaching assistant model.

[0009] Furthermore, in step 3.2, the composite attention mechanism module employs a composite attention module that simultaneously includes channel attention and spatial attention mechanisms; the channel attention mechanism compresses information through global average pooling and global max pooling. The lack of structured spatial relationships Dimensional information is assigned to the first initial compression weights. The calculation formula is as follows: (2); (3); (4); in, To The scalar obtained by global average pooling of the dimension; To The scalar obtained by global max pooling of the dimension; This refers to the time step number; This refers to the intermediate layer number; For the first The hidden feature is in the first The first intermediate layer The value at each time step; This is an operation performed on the Sigmoid activation function. This refers to the computational process of a two-layer shared perceptron. Obtain the first initial compression weight Then, combine it with The channel recalibration tensor is obtained by element-wise multiplication. , (5); in, For element-wise multiplication; The spatial attention mechanism in the composite attention mechanism module is: The structured arrangement relationship and Dimensional information is assigned to the second initial compression weights. The calculation formula is as follows: (6); (7); (8); in, To A two-dimensional tensor obtained by global average pooling of dimensions; To The two-dimensional tensor obtained by global max pooling of the dimension; For the first Each channel recalibrated tensor; This is a convolution operation; Obtain the second initial compression weight Then, combine it with After element-wise multiplication, the three-dimensional hidden information is obtained, and the final output tensor is obtained. : (9); Second initial compression weight Second initial compression weight It is continuously optimized during the training process of the teaching assistant model; in receiving... Subsequently, in the teaching assistant model, a two-dimensional convolution and pooling process is performed using a convolutional neural network to process the tensor. Adaptive compression is performed to generate a 3D information compression tensor with the same shape as the student model. Used for subsequent student model training. and These represent the number of hidden features and the number of intermediate layers in the time series prediction algorithm of the student model, respectively.

[0010] Furthermore, in step 3.3, different training approaches are adopted based on the different connection methods of the drilling feature enhancement module and the sequence change prediction module; When the drilling feature enhancement module and sequence change prediction module of the teacher model are connected using an encoder-decoder architecture, the corresponding teaching assistant models should be trained based on the teacher model encoder and decoder respectively. For the encoder teaching assistant model, the training objective is to extract a bridging vector that approximates the output of the teacher model encoder from its compressed hidden feature fusion tensor. The loss function is calculated as follows: (10); in, The loss function for the encoder teaching assistant model; Indicates the input number The complete hidden states of the last time step of the tail layer in the fusion hidden layer tensor output by the encoder teaching model for a drilling sequence sample; Indicates the input number The bridging vector output by the teacher model encoder for each drilling sequence sample; For the decoder teaching assistant model, the training optimization objective is to extract sequence generation patterns and prediction results that approximate the teacher model decoder from the compressed hidden feature fusion tensor; the loss function is calculated as follows: (11); in, The loss function for the decoder teaching assistant model; Indicates the input number For each drilling sequence sample, the decoder teaching model outputs the complete hidden states of the last time step of the tail layer in the fused hidden layer tensor; Indicates the input number The final hidden state of the teacher model decoder before the input linear layer for each drilling sequence sample; When the drilling feature enhancement module and sequence change prediction module of the teacher model are connected in a sequential stack, the teaching assistant model is trained only for the sequence change prediction module of the teacher model, and the loss function calculation formula refers to formula (10); in specific calculation, Replace with , Indicates the input number For each drilling sequence sample, the teacher model sequence change prediction module outputs all hidden states at the last time step.

[0011] Furthermore, the specific process of step 4 is as follows: Step 4.1: Build a composite knowledge distillation architecture for knowledge transfer; the composite knowledge distillation architecture includes four parts: output layer knowledge distillation, intermediate layer knowledge distillation, real error propagation, and weight optimization allocation. The output layer knowledge distillation process is as follows: After obtaining the pre-trained teacher model in step 2, the output layer knowledge distillation extracts the output layer output of the pre-trained teacher model and performs error calculation and backpropagation with the output layer output of the student model. The error calculation uses the mean squared error formula. The loss function of the output layer knowledge distillation is as follows: (12); in, The loss function for the knowledge distillation part of the output layer; and The output layers of the teacher model and student model respectively output the first... One predicted value; The process of knowledge distillation in the intermediate layer is as follows: The intermediate layer information of the teacher model is extracted and adaptively fused using the training teaching assistant model obtained in step 3 to obtain a 3D information compressed tensor. This compressed tensor is then compared with the 3D tensor formed by stacking the hidden states extracted from the intermediate layer of the student model. Error calculation and backpropagation are performed using the mean squared error formula. The loss function of the intermediate layer knowledge distillation is as follows: (13); in, The loss function for the intermediate layer knowledge distillation part; For input number The three-dimensional information compression tensor output by the teaching assistant model for each drilling sequence sample; For input number The three-dimensional tensor formed by stacking the hidden states of the intermediate layers of the student model along the hierarchical sequence when a drilling sequence sample is used; The working process of the true error propagation part is as follows: Error calculation and backpropagation are performed between the actual values ​​recorded by the downhole measuring equipment and the surface equipment and the predicted values ​​output by the student model's output layer. The error calculation uses the mean square error formula. The loss function of the true error propagation part is as follows: (14); in, The loss function is the one used for the propagation of the true error. The first record for downhole measuring equipment and surface equipment One true value; The first output layer of the student model One predicted value; the actual value index corresponds to the predicted value index; The process of weight optimization and allocation is as follows: assign influence weights to the three types of loss functions calculated by formulas (12)-(14) to construct the total loss function. as follows: + (15); in, , and The influence weights are assigned to the three types of loss functions; Step 4.2: Build and train the student model.

[0012] Furthermore, the specific process of step 4.2 is as follows: The student model building process follows the teacher model building process in step 2. During the building process, the drilling feature enhancement module is deleted. In terms of the sequence change prediction module, it maintains an isomorphic relationship with the teacher model, that is, it does not change the core algorithm type and basic model framework, but only compresses the model size by modifying the model parameters. The model parameters include the number of intermediate layers and the number of hidden features. After the student model is built, the composite knowledge distillation architecture and loss function formulas (12)-(15) are used, and the model is trained, optimized and tested based on the training set, validation set and test set in step 1.3.

[0013] The beneficial technical effects brought about by this invention are as follows.

[0014] (1) An accurate and lightweight wellbore trajectory prediction model for rotary steerable drilling is proposed. A high-precision teacher model is constructed by combining graph neural networks and time series prediction (TSF) algorithms. The teacher model is lightweighted through knowledge distillation to obtain a student model that is more suitable for downhole edge calculation as the final prediction model.

[0015] (2) A novel intermediate layer knowledge distillation method suitable for TSF tasks is proposed. This method can extract the hidden features of each intermediate layer of the time series prediction network (algorithm) in the teacher model layer by layer, and perform high-dimensional feature adaptive fusion according to the prediction target under the guidance of the attention mechanism. This solves the problem of "isomorphic but different parameters" between the teacher model and the student model, while effectively reducing knowledge transfer loss.

[0016] (3) The knowledge distillation loss function was reconstructed. During the training of the student model, the task loss, output loss and intermediate layer loss were combined to form a composite knowledge distillation architecture. A dynamic weight allocation mechanism was introduced to balance the influence weights of the various sub-loss functions according to the input samples of different well sections, thereby greatly improving the knowledge distillation effect. Attached Figure Description

[0017] Figure 1 This is a flowchart of the rotary steered drilling trajectory prediction method based on deep learning and knowledge distillation according to the present invention.

[0018] Figure 2 This is a flowchart illustrating the framework of the rotary steered drilling trajectory prediction method based on deep learning and knowledge distillation of this invention. Detailed Implementation

[0019] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: like Figure 1 and Figure 2As shown, to achieve rapid dynamic prediction of wellbore trajectory during RSS drilling, this invention proposes a lightweight sequence prediction method based on deep learning networks and a composite knowledge distillation architecture. This method first constructs a teacher model that combines feature enhancement algorithms and sequence prediction algorithms. The intermediate layer information of the teacher model is extracted, compressed, and optimized through a teaching assistant model. Finally, the output layer information and intermediate layer information of the teacher model are transferred to the student model through a composite knowledge distillation architecture. The student model is used to realize edge calculation and rapid prediction of wellbore trajectory during drilling.

[0020] This invention includes the following steps: Step 1: Collect surface data and logging-while-drilling data from the rotary steerable drilling site, and perform data preprocessing. This step processes the raw RSS drilling records collected from the site to form the datasets required for training various models in subsequent stages. This process can be summarized as data preprocessing model, and the specific steps are as follows: Step 1.1: Using ground data recording and measurement while drilling tools, obtain RSS raw drilling data samples including 16 drilling characteristics (see Table 1) at a sampling interval of 1m. The prediction target of this invention is the well deviation change rate.

[0021] The raw drilling data samples undergo data cleaning processes such as noise reduction and outlier removal to reduce the impact of measurement or recording errors.

[0022] The cleaned samples were sorted according to the measurement depth, and samples from different well sections were spliced ​​together to form the original dataset. ,in, For the first in the original dataset One sample; This represents the effective drilling sample size after data cleaning. This represents the number of drilling characteristics.

[0023] Table 1 Selection of Drilling Parameters (Characteristics) .

[0024] Step 1.2: Use sampling intervals The 1m sliding window algorithm divides the original dataset along the measurement depth into sliding window segments. Specifically, the sliding window is formed by the history window... and prediction window Composition, through The original dataset is segmented to obtain the continuous variation characteristics of the upper well section. After further processing, this information is used as model input. This process determines the range of the upper well section referenced by subsequent prediction models (including teacher and student models). The original dataset is segmented to obtain the continuous variation features of the lower well section (actual lower well section data), which are then compared with the predicted sequence to calculate the prediction error. This process determines the prediction range for subsequent wellbore trajectories. The continuous variation features of the upper and lower well sections are combined to obtain a serialized dataset; that is, this invention uses a sliding window algorithm to transform the original dataset, composed of single points from drilling data, into a serialized dataset. Transformed into a serialized dataset containing continuous variation features of the upper and lower well sections. .in, For the serialized dataset, the first Serialized samples; The formula for calculating the number of serialized samples is: (1); For example, when the original dataset Sliding window size ( , When the value is set to (20, 5), the serialized dataset is... .

[0025] Step 1.3: Divide the serialized dataset into training, validation, and test sets proportionally. The dataset is divided into training, validation, and test sets in an 8:1:1 ratio. The training set is used for training the teacher and student models, the validation set is used for parameter tuning of the teacher and student models, and the test set is used for performance evaluation of the teacher and student models.

[0026] Step 2, Teacher Model Construction and Training, can be summarized as a teacher model analysis module. This step uses a more complex but more accurate predictive model as the teacher model. This model can be deployed on a drilling platform or information center, or other surface facilities with good computing power and data storage conditions. It analyzes a large amount of regional well history data to capture the sequence changes in wellbore trajectory during rotary steerable drilling. The teacher model consists of a drilling feature enhancement module and a sequence change prediction module. The specific process is as follows: Step 2.1: Construction of the drilling feature enhancement module. The input to the drilling feature enhancement module is the serialized dataset. The drilling feature enhancement module, as an upstream data enhancement algorithm, is used to capture the external correlations between different drilling parameters at the same location and characterize these external correlations as data patterns recognizable by the downstream sequence variation prediction module. The drilling feature enhancement module can be constructed using algorithms such as convolutional neural networks and graph neural networks. Specifically, when constructing the drilling feature enhancement module with a graph neural network as its core, step 1.2 should use the PyTorch Geometric toolkit or other tools / methods to serialize the dataset. The tensor format is converted into a graph data format to meet the input data format requirements of the graph neural network, forming the training set, validation set, and test set required for the teacher model.

[0027] Step 2.2: Construction of the Sequence Variation Prediction Module. The sequence variation prediction module consists of an attention mechanism, a core sequence prediction algorithm, and a linear layer. Specifically, the attention mechanism employs a dot product attention mechanism. The sequence prediction algorithm receives drilling data enhanced by the upstream module and further captures the internal sequence dependencies of drilling parameters in the upper and lower well sections. The linear layer maps the high-dimensional feature tensor output by the sequence prediction algorithm to the dimension required for the prediction target. The input to this module is the output of the drilling feature enhancement module, and the output is the predicted wellbore trajectory parameter values ​​within a certain range (determined by the prediction window size setting), thus avoiding the information lag problem that may be caused by single-point prediction when the drilling speed is high. To ensure the smooth completion of subsequent steps, the sequence prediction algorithm involved in this step should employ recurrent neural networks and variant networks with similar operational logic, such as long short-term memory neural networks, gated recurrent unit networks, or composite models based on the above algorithms; sequence prediction algorithms that do not rely on hidden states for information transmission, such as temporal convolutional networks, should not be used.

[0028] Step 2.3: Module Assembly and Model Pre-training. After the drilling feature enhancement module and sequence change prediction module are constructed, they are assembled according to specific prediction requirements using network connection methods such as sequential stacking structure and encoder-decoder structure to form the teacher model. When the drilling feature enhancement module uses a graph neural network, the teacher model is pre-trained and its performance evaluated using the training set, validation set, and test set formed in Step 2.1. When the drilling feature enhancement module uses a convolutional neural network or other deep learning algorithms that require vector / tensor input data, the teacher model is directly pre-trained and its performance evaluated using the training set, validation set, and test set divided in Step 1.3. Evaluation can use metrics such as the coefficient of determination and mean absolute error.

[0029] The following are examples of teacher model network structures for reference: The embodiment employs a graph attention network as the core algorithm to construct a drilling feature enhancement module, and a long short-term memory neural network as the core algorithm to construct a sequence change prediction module. The modules are connected and assembled using an encoder-decoder structure. (Serialized dataset) The serialized samples were transformed into 14,247 drilling data maps after graph data (construction), and input into the teacher model at a batch size of 18 maps per batch. The drilling feature enhancement module (encoder) of the teacher model aggregates and enhances the node information of the original features provided by the drilling data maps through a graph attention network, and inputs the enhanced features into a long short-term memory network for information encoding and compression to form a bridging vector (context vector). The sequence change prediction module (decoder) of the teacher model reads the hidden features and bridging vectors of the encoder at different time steps, enhances the information of key time steps through an attention mechanism (i.e., enhances the information of key well sections), and then inputs it into a long short-term memory network for information decoding and recovery. Finally, the output data is transformed into the dimension of the prediction target requirement through a linear layer. The output at this time is the predicted value of the teacher model. The specific network structure and detailed parameters are shown in Table 2.

[0030] Table 2 Parameter Setting Table for Example Model .

[0031] Step 3: Construct and train an assistant model to assist the teacher model in adaptive compression of multi-intermediate layer data. After the teacher model is pre-trained, its different intermediate layers contain multiple levels of prior knowledge, effectively complementing the output layer knowledge. However, since the student model is developed for downhole deployment, its parameter settings and model structure are simplified compared to the teacher model, making it impossible to directly perform loss calculation and error backpropagation during subsequent knowledge distillation. This step solves the above problem by constructing an assistant model that can assist the teacher model in extracting and adaptively aggregating multi-intermediate layer data. This process can be summarized as an assistant model compression module, the specific process of which is as follows: Step 3.1: Extraction of Hidden Information Tensors from Intermediate Layers. Read and record the hidden information tensors of different intermediate layers of the teacher model to obtain... The shape is Two-dimensional tensor , This represents the number of intermediate layers in the time series prediction algorithm of the teacher model. Let be the number of hidden features in the time series prediction algorithm of the teacher model. The above two-dimensional tensor is stacked along the dimension of the number of intermediate layers to form a shape... 3D intermediate layer hidden information tensor In particular, when the number of hidden features differs among the different intermediate layers of the teacher model, average pooling can be used to unify the shape of all two-dimensional tensors.

[0032] Step 3.2, Teaching Assistant Model Construction. The teaching assistant model consists of a composite attention mechanism module and a convolutional neural network, used to read the 3D hidden information tensor extracted by the teacher model. The tensor is compressed with low precision loss to make it compatible with the 3D hidden information tensor output by the student model training process in step 4. Maintain consistency in shape. The composite attention mechanism module should employ a composite attention module that simultaneously incorporates channel attention and spatial attention mechanisms, such as the Convolutional Block Attention Module (CBAM). It receives input from the teacher model. Subsequently, the channel attention mechanism compresses information through global average pooling and global max pooling, and... The lack of structured spatial relationships Dimensional information is assigned to the first initial compression weights. The calculation formula is as follows: (2); (3); (4); in, To The scalar obtained by global average pooling of the dimension. To Both the scalar obtained by global max pooling of dimensions and the scalar obtained by global max pooling compress dimensions with structured arrangement relationships. And retained Dimensions are used for generation ; It is the time step number, which is Dimensional feature index; It is the intermediate layer number, which is Dimensional feature index; For the first The hidden feature is in the first The first intermediate layer The value at each time step; This is an operation performed on the Sigmoid activation function. The two-layer shared perceptron operation process involves first reducing the dimensionality of the input through a first linear layer. times ( Here, the default value of 16 is used to obtain the dimensionality-reduced input. The dimensionality-reduced input is then non-linearly activated using the ReLU function to obtain the activated input. Finally, the activated input is restored to the input dimension through the second linear layer operation.

[0033] Obtain the first initial compression weight Then, combine it with The channel recalibration tensor is obtained by element-wise multiplication. , (5); in, For element-wise multiplication; The spatial attention mechanism in the composite attention mechanism module is: The structured arrangement relationship and Dimensional information is assigned to the second initial compression weights. The calculation formula is as follows: (6); (7); (8); in, To A two-dimensional tensor obtained by global average pooling of dimensions; To The two-dimensional tensor obtained by global max pooling of the dimension; The index number is used when summing the channel recalibration tensors across all dimensions. Dimensional feature index; For the first Each channel recalibrated tensor; For convolution operations, that is... and Along The dimensions are concatenated, and a convolutional kernel of a certain size (7 is chosen here) is used. 7) Perform convolution operation.

[0034] Obtain the second initial compression weight Then, combine it with After element-wise multiplication, the three-dimensional hidden information is obtained, and the final output tensor is obtained. : (9); The aforementioned second initial compression weight Second initial compression weight During the training process of the teaching assistant model, it is continuously optimized to highlight the influence of key intermediate layers, key time steps, and key hidden features of the teacher model, thereby reducing information compression loss. In receiving... Subsequently, in the teaching assistant model, a two-dimensional convolution and pooling process is performed using a convolutional neural network to process the tensor. Adaptive compression is performed to generate a 3D information compression tensor with the same shape as the student model. Used for subsequent student model training. and These represent the number of hidden features and the number of intermediate layers in the time series prediction algorithm of the student model, respectively.

[0035] Step 3.3, Teaching Assistant Model Training. The overall goal of training the teaching assistant model is to reduce the dimensionality of the intermediate layers of the sequence change prediction module in the teacher model while fully preserving the high-precision hidden features obtained through optimization by the drilling feature enhancement module, thereby transferring the feature extraction and sequence generation patterns of the teacher model to the single-layer or shallow network of the student model. This invention employs different training approaches depending on the different connection methods of the drilling feature enhancement module and the sequence change prediction module, as detailed below: (1) When the drilling feature enhancement module and sequence change prediction module of the teacher model are connected by an encoder-decoder architecture, the corresponding teaching assistant models should be trained based on the encoder and decoder of the teacher model respectively; for the encoder teaching assistant model, the training objective is to extract a bridging vector close to the output of the encoder of the teacher model from its compressed hidden feature fusion tensor. The formula for calculating the loss function is as follows: (10); in, The loss function for the encoder teaching assistant model; Indicates the input number The fused hidden layer tensor output by the encoder teaching model for each drilling sequence sample All hidden states at the last time step of the middle tail layer; Indicates the input number The bridging vector output by the teacher model encoder for each drilling sequence sample; For input index.

[0036] For the decoder teaching assistant model, the training optimization objective is to extract sequence generation patterns and prediction results that approximate those of the teacher model decoder from the compressed hidden feature fusion tensor; the loss function is calculated as follows: (11); in, The loss function for the decoder teaching assistant model; Indicates the input number The fused hidden layer tensor output by the decoder teaching model for each drilling sequence sample. The complete hidden state of the last time step in the middle tail layer; Indicates the input number The final hidden state of the teacher model decoder before the input linear layer for each drilling sequence sample.

[0037] (2) When the drilling feature enhancement module and the sequence change prediction module of the teacher model are connected in a sequential stack, the teaching assistant model can be trained only for the sequence change prediction module of the teacher model. The loss function calculation formula is the same as formula (10). In specific calculation, Replace with , Indicates the input number For each drilling sequence sample, the teacher model sequence change prediction module outputs all hidden states at the last time step.

[0038] Step 4: Construction and Training of the Composite Knowledge Distillation Architecture and Student Model. This step will use the pre-trained teacher and assistant models obtained in Steps 1-3 to train a lightweight student model for downhole deployment and prediction while drilling in the RSS drilling process, employing a composite knowledge distillation method. This process can be summarized as a knowledge distillation module, as follows: Step 4.1: Building the Composite Knowledge Distillation Architecture. This step involves constructing a composite knowledge distillation architecture to complete knowledge transfer. The composite knowledge distillation architecture consists of four parts: output layer knowledge distillation, intermediate layer knowledge distillation, real error propagation, and weight optimization allocation. Each part is described below: (1) Output layer knowledge distillation. The pre-trained teacher model is obtained through step 2. The output layer output of this model is extracted and compared with the output layer output of the student model for error calculation and backpropagation. The mean squared error formula is used for error calculation, and the loss function is as follows: (12); in, The loss function for the knowledge distillation part of the output layer; and The output layers of the teacher model and student model respectively output the first... One predicted value; This is the index of the output sequence of the output layer for the teacher model and the student model.

[0039] (2) Intermediate layer knowledge distillation. The intermediate layer information of the teacher model is extracted and adaptively fused from the training teaching assistant model obtained in step 3 to obtain the three-dimensional information compression tensor. , and the three-dimensional tensor formed by extracting and stacking the hidden states from the intermediate layers of the student model. Error calculation and backpropagation are performed. The mean square error formula is used for error calculation. The loss function for the intermediate layer knowledge distillation part is as follows: (13); in, The loss function for the intermediate layer knowledge distillation part; For input number The three-dimensional information compression tensor output by the teaching assistant model for each drilling sequence sample; For input number The three-dimensional tensor formed by stacking the hidden states of the intermediate layers of the student model along the hierarchical sequence when a drilling sequence sample is used; In particular, when the teacher model adopts an encoder-decoder structure, formula (13) needs to be executed twice; the first execution Corresponding input number The 3D information compression tensor output by the encoder teaching model for each drilling sequence sample yields the encoder intermediate layer knowledge distillation loss function. Second execution Corresponding input number The 3D information compression tensor output by the decoder teaching model for each drilling sequence sample yields the knowledge distillation loss function of the decoder's intermediate layer. ; (3) Propagation of true error. This involves verifying the true values ​​recorded by downhole measuring equipment and surface equipment. Error calculation and backpropagation are performed between the predicted values ​​output by the student model's output layer and the actual error propagation part. The mean squared error formula is used for error calculation. The loss function for the actual error propagation part is as follows: (14); in, The loss function is the one used for the propagation of the true error. The first record for downhole measuring equipment and surface equipment One true value; The first output layer of the student model One predicted value; the actual value index corresponds to the predicted value index; (4) Weight Optimization Allocation. After calculating the loss functions of the three types of knowledge according to formulas (12)-(14), the influence weights need to be reasonably allocated to obtain a more accurate student model. Since the constraint effect of the three types of knowledge on the student model may vary in different well sections, setting fixed weights based on field experience or prior analysis during the knowledge distillation process is not effective. Therefore, a weight optimization allocation mechanism is adopted in this step to optimize the weights. , and The trainable parameters of the model are incorporated into the optimizer so that they follow the total loss function. Gradient descent process optimization: + (15); In particular, when the teacher model adopts an encoder-decoder structure, in formula (15) Should be replaced with and And assign influence weights respectively. and Then, it is summed together with other losses for calculation.

[0040] Step 4.2: Student Model Building, Training, and Evaluation. The student model building process follows the teacher model building process in Step 2. In particular, to fully compress the model size, the drilling feature enhancement module is removed from the student model; in terms of the sequence change prediction module, it should maintain an isomorphic relationship with the teacher model, that is, without changing the core algorithm type and the basic model framework, only by modifying the model parameters (such as the number of intermediate layers and the number of hidden features) to compress the model size. After the student model is built, the composite knowledge distillation architecture and loss function formulas (12)-(15) are used, and the model is trained, optimized, and tested based on the training set, validation set, and test set in Step 1.3 until the model prediction performance meets the requirements of field application. The model performance evaluation in this step includes two aspects: predictive performance and compression. Predictive performance evaluation aims to determine whether the student model can meet the accuracy requirements for predicting wellbore trajectories while rotating steered drilling. Predictive metrics include the coefficient of determination and mean absolute error. Compression evaluation aims to determine the extent to which the student model is compressed compared to the teacher model while maintaining field accuracy requirements. This metric can be used to predict whether the student model can meet the downhole deployment requirements during the prediction while drilling process. The compression metric uses the parametric compression ratio. And calculate compression ratio The calculation formula is as follows: (16); (17); in, and These represent the number of floating-point operations for the teacher model and the student model, respectively. and These are the structural parameters of the teacher model and the student model, respectively. The output is a student model with good evaluation performance, which is the required lightweight wellbore trajectory prediction model.

[0041] After completing steps 5 and 1 through 4, a trained lightweight wellbore trajectory prediction model (student model) is obtained. In field applications, preprocessed surface data and logging-while-drilling data are fed into the lightweight wellbore trajectory prediction model to obtain subsequent data within a certain range (determined by parameters). The wellbore trajectory prediction value (determined by the set value) is used to achieve early prediction, rapid judgment, and optimization control of the wellbore trajectory by changing operating parameters such as drilling pressure, rotation speed, and steering force. The overall operation flow of the method designed in this invention is as follows: Figure 1 .

[0042] In addition, to demonstrate the feasibility and superiority of the present invention, the following comparative experiments are provided.

[0043] Regarding the teacher model, the structure set in the embodiment of step 2 (see Table 2) was adopted. Simultaneously, three common sequence prediction models—convolutional neural networks, bidirectional long short-term memory neural networks, and temporal convolutional networks—were selected after parameter tuning, along with two baseline models—long short-term memory neural networks and gated recurrent unit networks—as comparison models. Each model was tested five times on the test set, and the average value of each evaluation metric (coefficient of determination R0) was calculated. 2 The mean absolute error (MAE) was compared and analyzed, and the evaluation results are shown in Table 3.

[0044] Table 3. Comparison of prediction performance among different models .

[0045] Table 3 shows that, under different parameter settings, the teacher model's prediction performance on the test set was significantly higher than other comparative models; the mean absolute error (MAE) decreased by 18.8% to 41.2%, indicating that the teacher model's overall prediction curve was closer to the true prediction curve than other models; the coefficient of determination R... 2 The improvement of 2.4% to 9.6% indicates that the teacher model achieves more effective control over local data errors compared to other models. These performance optimizations demonstrate the feasibility and technical superiority of the high-precision teacher model proposed in this invention, formed by the drilling feature enhancement module and the sequence variation prediction module.

[0046] For the student model, a 3×3 crossover experiment was conducted to evaluate its performance, using the number of hidden features and the number of intermediate layers as variables. The number of hidden features was set to 64, 32, and 16, and the number of intermediate layers was set to 3, 2, and single layers. The nine models were run five times on the test set, and the average value of each evaluation metric (R²) was calculated. 2 The mean absolute error (MAE), parametric compression ratio (PCR), and calculated compression ratio (CCR) were compared and analyzed. The evaluation results are shown in Table 4.

[0047] As shown in Table 4, the student model performed relatively stably under different parameter settings. While its prediction accuracy was somewhat lower than the teacher model's best performance, the mean absolute error (MAE) increased by 15.6% to 35.9%, and the coefficient of determination (R²) decreased. 2While the accuracy decreased by 2.1% to 4.3%, the overall prediction accuracy remained close to that of other prediction models in Table 3, basically meeting the accuracy requirements of engineering sites. Meanwhile, with sufficient compression, the student model's parameter compression ratio (PCR) and computational compression ratio (CCR) reached 98.07% and 98.07%, respectively. This means that the student model requires only about 2% of the storage and computing power of the original teacher model, under the premise of controllable accuracy loss. This demonstrates that the composite knowledge distillation architecture can effectively transfer knowledge and compress the model to adapt to downhole deployment constraints. The student model produced by the method proposed in this invention can meet the requirements for wellbore trajectory prediction during rotary steerable drilling. Furthermore, to demonstrate the superiority of the composite knowledge distillation architecture proposed in this invention, three sets of comparative experiments were designed: output layer knowledge distillation, intermediate layer knowledge distillation, and knowledge-free distillation (trained only with real data). The experiments were repeated five times on the test set, and the optimal values ​​for each evaluation index were selected. The comparison results are shown in Table 5.

[0048] Table 4 Comparison of model performance under different parameter settings .

[0049] Table 5. Performance Comparison of Models under Different Knowledge Distillation Architectures .

[0050] Table 5 shows that the composite knowledge distillation architecture designed in this invention has a significant advantage in terms of accuracy loss compared with other conventional knowledge distillation architectures. The improvement in the mean absolute error index is particularly significant, with the average MAE decreasing by 19.7% to 36.2% compared with other architectures. This demonstrates that knowledge transfer through the composite knowledge distillation architecture can enable the obtained lightweight wellbore trajectory prediction model to have higher overall prediction accuracy.

[0051] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A method for predicting rotary steered drilling trajectory based on deep learning and knowledge distillation, characterized in that, Includes the following steps: Step 1: Collect surface data and logging-while-drilling data from the rotary steerable drilling site, and perform data preprocessing; Step 2: Construct and train the teacher model; the teacher model includes a drilling feature enhancement module and a sequence change prediction module; Step 3: Build and train the teaching assistant model to assist the teacher model in adaptive compression of multi-intermediate layer data; Step 4: Build a composite knowledge distillation architecture and student model, and train the student model. The trained student model is the lightweight wellbore trajectory prediction model. Step 5: In field application, the ground data and logging-while-drilling data are preprocessed and then fed into the lightweight wellbore trajectory prediction model to obtain the predicted wellbore trajectory value of the rotary steerable drilling process, so as to adjust the drilling operation parameters in a timely manner based on the prediction results.

2. The rotary steered drilling trajectory prediction method based on deep learning and knowledge distillation according to claim 1, characterized in that, The specific process of step 1 is as follows: Step 1.1: Obtain raw drilling data samples including 16 drilling features through surface data recording and measurement-while-drilling tools; clean the raw drilling data samples, sort the cleaned samples according to measurement depth, and stitch together samples from different well sections to form the raw dataset. ,in, For the first in the original dataset One sample; Number of drilling features; The 16 drilling characteristics include drilling pressure, rotation speed, steering force, tool face angle, drilling fluid flow rate, drilling fluid density, equivalent circulating density, standpipe pressure, drilling speed, torque, well inclination angle, azimuth angle, measurement depth, drilling vertical depth, formation temperature, and formation drillability index. Step 1.2: The original dataset is divided into sliding window segments along the measurement depth using the sliding window algorithm; the sliding window is divided into history windows. and prediction window ,pass The continuous variation characteristics of the upper well section were obtained by segmenting the original dataset; through... The original dataset is segmented to obtain the continuous variation features of the lower well section; the continuous variation features of the upper and lower well sections are combined to obtain a serialized dataset containing the continuous variation features of both the upper and lower well sections. ;in, For the serialized dataset, the first Serialized samples; The formula for calculating the number of serialized samples is: (1)； in, This represents the sampling interval for the sliding window algorithm. Step 1.3: Divide the serialized dataset into training set, validation set and test set according to the proportion.

3. The rotary steered drilling trajectory prediction method based on deep learning and knowledge distillation according to claim 2, characterized in that, The specific process of step 2 is as follows: Step 2.1: Construct a drilling feature enhancement module; The input to the drilling feature enhancement module is a serialized dataset. The drilling feature enhancement module, as an upstream data enhancement algorithm, is used to capture the external correlation between different drilling parameters at the same location and characterize the external correlation as a data pattern that the downstream sequence change prediction module can recognize. The drilling feature enhancement module is constructed using convolutional neural networks or graph neural networks. When the drilling feature enhancement module uses graph neural networks as the core algorithm, the PyTorch Geometric toolkit is used to serialize the dataset. The tensor format is converted into a graph data format to meet the input data format requirements of the graph neural network, forming the training set, validation set, and test set required for the teacher model; Step 2.2: Construct the sequence variation prediction module. The sequence variation prediction module consists of an attention mechanism, a sequence prediction algorithm, and a linear layer. Specifically, the attention mechanism adopts the dot product attention mechanism. The sequence prediction algorithm is used to receive the drilling data after it has been enhanced by the upstream module and further capture the internal sequence dependencies of the drilling parameters in the upper and lower well sections. The linear layer is used to map the high-dimensional feature tensor output by the sequence prediction algorithm to the dimension required for the prediction target. The input of the sequence variation prediction module is the output of the drilling feature enhancement module, and the output is the predicted value of the wellbore trajectory parameters. Step 2.3: Module Assembly and Model Pre-training; After the drilling feature enhancement module and sequence change prediction module are constructed, the two are assembled according to the specific prediction requirements by selecting either a sequential stacking structure or an encoder-decoder structure network connection method to form a teacher model; when the drilling feature enhancement module uses a graph neural network as the core algorithm, the training set, validation set, and test set formed in Step 2.1 are used to pre-train and evaluate the performance of the teacher model; when the drilling feature enhancement module uses a convolutional neural network or other deep learning algorithms that require vector / tensor input data formats, the training set, validation set, and test set divided in Step 1.3 are directly used to pre-train and evaluate the performance of the teacher model.

4. The rotary steered drilling trajectory prediction method based on deep learning and knowledge distillation according to claim 3, characterized in that, The specific process of step 3 is as follows: Step 3.1: Read and record the hidden information tensors of different intermediate layers of the teacher model to obtain... The shape is Two-dimensional tensor , This represents the number of intermediate layers in the time series prediction algorithm of the teacher model. The number of hidden features in the time series prediction algorithm in the teacher model; Two-dimensional tensor Stacking along the number of intermediate layers forms a shape of 3D intermediate layer hidden information tensor ; Step 3.2: Construct the teaching assistant model; the teaching assistant model consists of a composite attention mechanism module and a convolutional neural network, used to read the 3D hidden information tensor extracted by the teacher model. And adaptive compression of the three-dimensional hidden information tensor; Step 3.3: Train the teaching assistant model.

5. The rotary steered drilling trajectory prediction method based on deep learning and knowledge distillation according to claim 4, characterized in that, In step 3.2, the composite attention mechanism module employs a composite attention module that simultaneously includes channel attention and spatial attention mechanisms; the channel attention mechanism compresses information through global average pooling and global max pooling. The lack of structured spatial relationships Dimensional information is assigned to the first initial compression weights. The calculation formula is as follows: (2)； (3)； (4)； in, To The scalar obtained by global average pooling of the dimension; To The scalar obtained by global max pooling of the dimension; This refers to the time step number; This refers to the intermediate layer number; For the first The hidden feature is in the first The first intermediate layer The value at each time step; This is an operation performed on the Sigmoid activation function. This refers to the computational process of a two-layer shared perceptron. Obtain the first initial compression weight Then, combine it with The channel recalibration tensor is obtained by element-wise multiplication. , (5)； in, For element-wise multiplication; The spatial attention mechanism in the composite attention mechanism module is: The structured arrangement relationship and Dimensional information is assigned to the second initial compression weights. The calculation formula is as follows: (6)； (7)； (8)； in, To A two-dimensional tensor obtained by global average pooling of dimensions; To The two-dimensional tensor obtained by global max pooling of the dimension; For the first Each channel recalibrated tensor; This is a convolution operation; Obtain the second initial compression weight Then, combine it with After element-wise multiplication, the three-dimensional hidden information is obtained, and the final output tensor is obtained. : (9)； Second initial compression weight Second initial compression weight It is continuously optimized during the training process of the teaching assistant model; in receiving... Subsequently, in the teaching assistant model, a two-dimensional convolution and pooling process is performed using a convolutional neural network to process the tensor. Adaptive compression is performed to generate a 3D information compression tensor with the same shape as the student model. Used for subsequent student model training. and These represent the number of hidden features and the number of intermediate layers in the time series prediction algorithm of the student model, respectively.

6. The rotary steered drilling trajectory prediction method based on deep learning and knowledge distillation according to claim 5, characterized in that, In step 3.3, different training approaches are adopted based on the different connection methods of the drilling feature enhancement module and the sequence change prediction module; When the drilling feature enhancement module and sequence change prediction module of the teacher model are connected using an encoder-decoder architecture, the corresponding teaching assistant models should be trained based on the teacher model encoder and decoder respectively. For the encoder teaching assistant model, the training objective is to extract a bridging vector that approximates the output of the teacher model encoder from its compressed hidden feature fusion tensor. The loss function is calculated as follows: (10)； in, The loss function for the encoder teaching assistant model; Indicates the input number The complete hidden states of the last time step of the tail layer in the fusion hidden layer tensor output by the encoder teaching model for a drilling sequence sample; Indicates the input number The bridging vector output by the teacher model encoder for each drilling sequence sample; For the decoder teaching assistant model, the training optimization objective is to extract sequence generation patterns and prediction results that approximate the teacher model decoder from the compressed hidden feature fusion tensor; the loss function is calculated as follows: (11) ； in, The loss function for the decoder teaching assistant model; Indicates the input number For each drilling sequence sample, the decoder teaching model outputs the complete hidden states of the last time step of the tail layer in the fused hidden layer tensor; Indicates the input number The final hidden state of the teacher model decoder before the input linear layer for each drilling sequence sample; When the drilling feature enhancement module and sequence change prediction module of the teacher model are connected in a sequential stack, the teaching assistant model is trained only for the sequence change prediction module of the teacher model, and the loss function calculation formula refers to formula (10); in specific calculation, Replace with , Indicates the input number For each drilling sequence sample, the teacher model sequence change prediction module outputs all hidden states at the last time step.

7. The rotary steered drilling trajectory prediction method based on deep learning and knowledge distillation according to claim 6, characterized in that, The specific process of step 4 is as follows: Step 4.1: Build a composite knowledge distillation architecture for knowledge transfer; the composite knowledge distillation architecture includes four parts: output layer knowledge distillation, intermediate layer knowledge distillation, real error propagation, and weight optimization allocation. The output layer knowledge distillation process is as follows: After obtaining the pre-trained teacher model in step 2, the output layer knowledge distillation extracts the output layer output of the pre-trained teacher model and performs error calculation and backpropagation with the output layer output of the student model. The error calculation uses the mean squared error formula. The loss function of the output layer knowledge distillation is as follows: (12)； in, The loss function for the knowledge distillation part of the output layer; and The output layers of the teacher model and student model respectively output the first... One predicted value; The process of knowledge distillation in the intermediate layer is as follows: The intermediate layer information of the teacher model is extracted and adaptively fused using the training teaching assistant model obtained in step 3 to obtain a 3D information compressed tensor. This compressed tensor is then compared with the 3D tensor formed by stacking the hidden states extracted from the intermediate layer of the student model. Error calculation and backpropagation are performed using the mean squared error formula. The loss function of the intermediate layer knowledge distillation is as follows: (13)； in, The loss function for the intermediate layer knowledge distillation part; For input number The three-dimensional information compression tensor output by the teaching assistant model for each drilling sequence sample; For input number The three-dimensional tensor formed by stacking the hidden states of the intermediate layers of the student model along the hierarchical sequence when a drilling sequence sample is used; The working process of the true error propagation part is as follows: Error calculation and backpropagation are performed between the actual values ​​recorded by the downhole measuring equipment and the surface equipment and the predicted values ​​output by the student model's output layer. The error calculation uses the mean square error formula. The loss function of the true error propagation part is as follows: (14)； in, The loss function is the one used for the propagation of the true error. The first record for downhole measuring equipment and surface equipment One true value; The first output layer of the student model One predicted value; the actual value index corresponds to the predicted value index; The process of weight optimization and allocation is as follows: assign influence weights to the three types of loss functions calculated by formulas (12)-(14) to construct the total loss function. as follows: + （15）； in, , and The influence weights are assigned to the three types of loss functions; Step 4.2: Build and train the student model.

8. The rotary steered drilling trajectory prediction method based on deep learning and knowledge distillation according to claim 7, characterized in that, The specific process of step 4.2 is as follows: The student model building process follows the teacher model building process in step 2. During the building process, the drilling feature enhancement module is deleted. In terms of the sequence change prediction module, it maintains an isomorphic relationship with the teacher model, that is, it does not change the core algorithm type and basic model framework, but only compresses the model size by modifying the model parameters. The model parameters include the number of intermediate layers and the number of hidden features. After the student model is built, the composite knowledge distillation architecture and loss function formulas (12)-(15) are used, and the model is trained, optimized and tested based on the training set, validation set and test set in step 1.3.

Images