Construction method and application of well site real-time data pre-training model

Abstract

The invention provides a construction method and application of a well site real-time data pre-training model, and belongs to the crossing field of petroleum engineering and artificial intelligence. Carrying out discrete marking; the method comprises the following steps: constructing a self-supervised time sequence model based on a Transform architecture, and fusing position coding and an embedding mechanism; designing a self-supervised task, and setting multiple task targets such as mask prediction, comparative learning, sequence judgment and future state prediction; performing end-to-end training on the model through a joint loss function, and extracting deep time sequence features; deploying the pre-training model in an actual well site environment, and executing tasks such as anomaly detection, working condition recognition and trend prediction; and carrying out model optimization and iteration. The invention further provides application of the method in the field of intelligent monitoring of oil and gas drilling engineering. According to the method, the problems of high dependence on manual labeling, weak model generalization ability, insufficient real-time adaptability and the like of a traditional method are solved, and the method has good engineering deployment and cross-well-site popularization value.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of petroleum engineering and artificial intelligence, and in particular to a method for constructing and applying a real-time data pre-training model for well sites. Background Technology

[0002] With the emergence of concepts such as intelligent drilling and digital oilfields, the large amount of real-time data generated during well site operations has become a crucial basis for improving drilling efficiency and safety. Traditional parametric modeling methods often rely on manual empirical rules or static regression models, which are difficult to adapt to the field environment with frequent changes in operating conditions and drastic parameter fluctuations, and have limited support capabilities for tasks such as anomaly identification and trend prediction. Meanwhile, continuous time-series data is affected by noise interference and complex time-series structures during the modeling process, and direct supervised learning training often relies on a large number of manual labels, making large-scale deployment difficult. In recent years, although deep models such as Transformer have made progress in sequence modeling, they still lack optimized structures suitable for complex well site scenarios. Summary of the Invention

[0003] In view of this, the present invention aims to propose a method for constructing and using a pre-trained model for real-time well site data. By combining discrete labeling, self-supervised multi-task design and Transformer temporal modeling, it can achieve efficient modeling, anomaly detection and operating condition prediction of multi-source well site data, and improve the generalization and field adaptability of the model without the need for manual labeling.

[0004] To achieve the above objectives, the technical solution of the present invention is implemented as follows: a method for constructing a pre-trained model for real-time well site data, comprising the following steps:

[0005] Step S1, Data Acquisition and Preprocessing: Acquire real-time continuous time-series data generated during well site drilling and preprocess the real-time continuous time-series data;

[0006] Step S2, Discrete Labeling: Divide the preprocessed continuous time series data into multiple numerical intervals and assign a unique discrete label to each interval to generate a discrete label sequence;

[0007] Step S3, Model Building: Construct a time series prediction model based on the Transformer architecture, using discrete label sequences as input, to model the time dependency and state change patterns of drilling parameters;

[0008] Step S4, Self-supervised task design: Design self-supervised learning tasks for pre-training, including mask label prediction task, state comparison learning task, sequence judgment task, and future state prediction task, which use the current label sequence and historical label sequence as input to predict the label state at future time.

[0009] Step S5, Model pre-training: The time series prediction model is pre-trained in a self-supervised manner using the discretely labeled data, and the model parameters are updated by optimizing the prediction accuracy and feature representation ability.

[0010] Step S6, Application and Validation: Deploy the pre-trained model at the well site to achieve real-time anomaly detection, working condition identification and future state prediction of drilling parameters, and validate the model effect based on field data;

[0011] Step S7, Model Optimization and Iteration: Based on the validation results, the pre-trained model is retrained and optimized, and the model structure or parameters are updated to improve the model's adaptability and prediction accuracy under actual working conditions.

[0012] Furthermore, in step S1, the well site system uses various industrial sensors deployed on the surface and underground to collect real-time continuous time-series data during the drilling process, including:

[0013] Mechanical parameters: drill bit depth, mechanical drilling speed, hook load, drilling pressure, rotational speed, torque, pump speed;

[0014] Hydraulic parameters: flow rate, moving pool volume, pump pressure, drilling fluid density;

[0015] Temperature parameters: estimated values ​​of wellhead temperature and bottom hole temperature;

[0016] Control parameters: pump status, valve opening, operating mode;

[0017] All data are sampled synchronously with a uniform time step of 1Hz-5Hz and uploaded to the data server via the SCADA system or OPC-UA communication protocol / Modbus communication protocol to form a structured multi-channel time series dataset, which forms the basis for subsequent modeling.

[0018] The collected real-time continuous time series data needs to be standardized and preprocessed before being input into the model, including denoising, outlier removal, linear interpolation padding, Z-score normalization, and time alignment, to finally form a training sample set with consistent format and complete fields.

[0019] Furthermore, in step S2, the discrete labeling uses K-means clustering to divide the intervals and assigns a unique integer code label to each interval;

[0020] To adapt to the self-supervised modeling framework, continuous time-series data is transformed into discrete label sequences. For each continuous variable, a weighted sliding window K-Means clustering algorithm is used to construct data distribution centers. The discrete label encoding function is as follows:

[0021]

[0022] Where, x t Represents the original input value at time t; For x t The corresponding discrete label number, such as 0, 1, 2; This represents the k-th cluster center within that time window.

[0023] Furthermore, in step S3, multivariate time series modeling based on the Transformer structure is adopted. The input sequence used for modeling is extracted from the continuous multivariate discrete label sequence through a sliding window with a fixed step size. The input sequence is constructed as follows:

[0024]

[0025] in, Using time step t as the current moment and backtracking T, past The multivariate discrete label input sequence matrix formed by each time step. It is the multivariate discrete label vector at time step t, derived from the clustering and encoding output in step S2; T past The time window length is n; the number of variable dimensions is n.

[0026] To adapt to the Transformer structure, each discrete label vector is embedded and superimposed with sine and cosine positional encodings to construct the final input tensor:

[0027]

[0028] Among them, Z t This refers to the multivariate input feature tensor formed within the historical window at time step t after embedding and positional encoding. d model This represents the embedding dimension of the Transformer model.

[0029] Furthermore, in step 4, the four tasks—mask label prediction, state comparison learning, sequence judgment, and future state prediction—are trained in parallel, collectively forming the final training objective function:

[0030]

[0031] in, This represents the joint self-supervised loss function for multiple tasks. This represents the loss for mask label prediction; Indicates the contrast learning loss; Indicates the order of loss assessment. Let λ1, λ2, and λ3 represent the weighting coefficients of the task loss.

[0032] Furthermore, in step S5, the pre-training stage takes the structured discrete label sequence as input and performs end-to-end model optimization under the self-supervised task drive. The model training objective is to maximize the predictive and representational capabilities of various tasks.

[0033] A multi-task loss joint optimization mechanism is introduced, and the following update formula is set:

[0034]

[0035] Where, θ (t+1) θ represents the parameter vector of the model at the (t+1)th iteration; (t) This represents the parameter vector of the model at the t-th iteration; η is the base learning rate set during the pre-training phase of the model. For multi-task joint self-supervised loss function; This indicates the gradient calculation of the parameters.

[0036] Furthermore, in step S6, the pre-trained model is deployed on the edge computing device at the well site. The computing device performs online inference and processing on the real-time acquired well site data. Simultaneously, the deployed model integrates with the on-site SCADA system via a data interface or communication protocol to achieve automatic data transmission and result feedback. The deployed model is used to perform online processing and feedback verification of the real-time acquired data. The online processing and feedback verification include:

[0037] Anomaly detection: Utilizing mask recovery capabilities to determine deviations in the current data.

[0038]

[0039] Where, δ i,t This represents the anomaly indicator value of the i-th variable at time step t. If the predicted value is inconsistent with the true label, it is 1; if the predicted value is consistent with the true label, it is 0. An indicator function that checks if the condition inside the parentheses is true. If the condition is true, the function returns 1; otherwise, it returns 0. This represents the discrete label value of the i-th variable predicted by the model at time t; This represents the true discrete label value; when the cumulative anomaly score exceeds the threshold θ alert When this happens, the system will trigger an alarm or provide an operational intervention prompt;

[0040] Working condition identification: Unsupervised clustering is performed through the intermediate layer representation of the model to automatically identify working conditions such as drilling, single joint connection, and tripping.

[0041] Trend prediction: Predicting the state at several future moments based on the current sliding window, using the following formula:

[0042]

[0043] in, This represents a sequence of labels for several future time steps predicted by the model. This represents a pre-trained model with parameter θ; θ pretrained This represents the set of model parameters after self-supervised training and optimization. This represents the sliding window input sequence before time t.

[0044] Furthermore, in step S7, to address the performance degradation caused by changes in well site data distribution over time, an online model optimization mechanism is introduced; the latest data is collected through a sliding window. Using the pre-trained model θ0 as initial parameters, and performing low-frequency fine-tuning based on a self-supervised task, the optimized online update formula is as follows:

[0045]

[0046] Where θ0 represents the current model parameters; θ online This represents the deployed model after fine-tuning and updating; α is the fine-tuning learning rate, which is much smaller than that in the initial training phase. This is a set of self-supervised tasks, corresponding to the four types of training tasks defined in step S4; The dynamic weight of task j at time step t; This represents the loss function corresponding to task j; This is the latest collected discretized dataset.

[0047] This invention also provides an application of the above-described construction method in the field of intelligent monitoring of oil and gas drilling projects.

[0048] Furthermore, the method for constructing a pre-trained model for real-time well site data can perform parameter anomaly detection, operational condition identification, and drilling trend prediction based on real-time time-series data collected by well site sensors.

[0049] Compared with existing technologies, the method for constructing and applying a real-time well site data pre-training model described in this invention has the following advantages:

[0050] (1) This invention does not rely on manually labeled data, but uses a self-supervised multi-task mechanism to achieve efficient learning of drilling data, thereby reducing the cost and time of model deployment;

[0051] (2) This invention effectively improves the model’s ability to handle continuous variables and its robustness against noise by introducing the sliding window K-Means discrete labeling method;

[0052] (3) This invention combines a Transformer-based time series model structure to enhance the model’s ability to model multivariate and long-term dependencies, and adapt to the complex changes in well site operating conditions.

[0053] (4) This invention improves the feature extraction capability and multi-objective expression capability of the model by designing a variety of self-supervised tasks such as mask prediction, contrastive learning, sequence judgment and future state prediction.

[0054] (5) This invention introduces an online model optimization and sliding window update mechanism to enable the model to continuously adapt and evolve in multiple well sites and at multiple stages, and has stronger cross-well promotion capability and long-term operational stability.

[0055] (6) The overall process of this invention can be integrated into a drilling platform or edge device, with good engineering feasibility and real-time deployment value, and is suitable for a variety of intelligent drilling scenarios. Attached Figure Description

[0056] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0057] Figure 1 A flowchart illustrating a method for constructing a pre-trained model for real-time well site data;

[0058] Figure 2 This is a schematic diagram showing the SPP pressure values ​​and their discrete ranges.

[0059] Figure 3 Here is a diagram of the Transformer architecture;

[0060] Figure 4 Deployment flowchart for the model. Detailed Implementation

[0061] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0062] In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0063] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0064] like Figures 1-4 As shown, this invention relates to a method for intelligent modeling of well site data based on self-supervised learning, specifically a method for constructing a pre-trained model for real-time well site data. The aim is to achieve real-time modeling, anomaly detection, and operational condition prediction of key parameters in the drilling process, thereby improving the real-time analysis and adaptive capabilities of the drilling system. The method includes the following steps:

[0065] Step S1, Data Acquisition and Preprocessing: Acquire real-time continuous time-series data generated during well site drilling and preprocess the real-time continuous time-series data;

[0066] Step S2, Discrete Labeling: Divide the preprocessed continuous time series data into multiple numerical intervals and assign a unique discrete label to each interval to generate a discrete label sequence, so as to adapt to the modeling requirements of the Transformer model structure and the self-supervised training task.

[0067] Step S3, Model Building: Construct a time series prediction model based on the Transformer architecture, using discrete label sequences as input, to model the time dependency and state change patterns of drilling parameters, and to provide a model foundation with strong expressive power and stable structure for subsequent self-supervised learning tasks;

[0068] Step S4, Self-supervised task design: Design self-supervised learning tasks for pre-training, including mask label prediction task, state comparison learning task, sequence judgment task, and future state prediction task, which use the current label sequence and historical label sequence as input to predict the label state at future time.

[0069] Step S5, Model pre-training: The time series prediction model is pre-trained in a self-supervised manner using the discretely labeled data, and the model parameters are updated by optimizing the prediction accuracy and feature representation ability.

[0070] Step S6, Application and Validation: Deploy the pre-trained model at the well site to achieve real-time anomaly detection, working condition identification and future state prediction of drilling parameters, and validate the model effect based on field data;

[0071] Step S7, Model Optimization and Iteration: Based on the validation results, the pre-trained model is retrained and optimized, and the model structure or parameters are updated to improve the model's adaptability and prediction accuracy under actual working conditions.

[0072] In step S1, the well site system uses various industrial sensors deployed on the surface and downhole to collect real-time continuous time-series data during the drilling process. This data includes, but is not limited to:

[0073] Mechanical parameters: Drill depth (Depth), ROP (Rolling Rate), Hook Load (HKLD), WOB (Whole Wire Load), RPM (Rotation Speed), Tr (Torque), SPM (Surge Speed);

[0074] Hydraulic parameters: flow rate (Q), volumetric volume (PVT), pump pressure (SPP), drilling fluid density (MW);

[0075] Temperature parameters: estimated values ​​of wellhead temperature and bottom hole temperature;

[0076] Control parameters: pump status, valve opening, operating mode (manual / automatic), etc.

[0077] The collected raw multivariate time series data are preprocessed to improve data quality and enhance model trainability. The preprocessing process includes the following steps:

[0078] To reduce high-frequency fluctuations and occasional interference during the field sensor acquisition process, this invention uses moving average filtering for noise reduction, uses a fixed-length sliding window to smooth the original signal, and can perform linear interpolation to repair or remove severely distorted or erroneous data points.

[0079] To ensure that different physical quantities are on the same numerical scale and to facilitate model training, this invention employs the Z-score normalization method as a normalization strategy.

[0080] Since the parameters may be obtained by different sampling modules, there may be sampling delays or omissions. In order to unify the timing, the master clock reference interpolation method is used to synchronize the data of each channel, and the sampling gaps are filled by linear or sliding interpolation.

[0081] Finally, all parameter data are synchronously sampled at a uniform time step (1Hz-5Hz is recommended) and uploaded to the data server through the well site SCADA system or a compatible industrial communication protocol (such as OPC-UA, Modbus) to build a structured multi-channel time series dataset, providing a standardized data foundation for subsequent discrete labeling, model building and training tasks.

[0082] In step S2, the discrete labeling uses K-means clustering to divide the data into intervals, and assigns a unique integer encoding label to each interval. Specifically, this invention uses a weighted time-series window K-means clustering algorithm to adaptively discretely partition each variable, as follows:

[0083] (1) Sliding window temporal clustering structure

[0084] Considering that the well site data is a dynamic, non-stationary sequence, a sliding time window W is used. t (Window length can be set to 5-10 minutes) Perform local clustering and dynamically obtain the status label mapping relationship for each time period.

[0085] (2) Sample credibility weighting mechanism

[0086] To reduce the interference of outliers (such as pump shocks, pressure loss, etc.) on the boundaries of discrete labels, a sample confidence weight w is introduced. i ∈(0,1], construct a weighted clustering objective function, the function formula is:

[0087]

[0088] Where, x i w represents the value of the i-th original sample; i Indicates sample x i Weights; W t C represents the time window that slides forward from the current time t; k This represents the k-th cluster; This indicates that the k-th class sample is in window W. t The cluster center (mean) within the cluster; K represents the discrete label type.

[0089] (3) Discrete Tag Encoding Function

[0090]

[0091] Where, x t Represents the original input value at time t; For x t The corresponding discrete label number (e.g., 0, 1, 2); This represents the k-th cluster center within that time window.

[0092] The optimization of this formula lies in using "K-Means cluster centers within a sliding window". Compared to general static clustering, it demonstrates temporal locality and online adaptability; the discrete labels are specifically designed for Transformer input.

[0093] Taking pressure parameters as an example, they can be divided into three categories: "low pressure," "medium pressure," and "high pressure," corresponding to label values ​​of 0, 1, and 2, respectively. This discretization process preserves the original physical meaning while avoiding the error amplification problem caused by direct regression.

[0094] For strongly correlated variable pairs (such as drill pressure (WOB) and mechanical drilling rate (ROP), or multiple correlated parameters, joint cluster analysis can be performed using a joint modeling approach for multidimensional clustering. The formula is as follows:

[0095]

[0096] Where, x t Let be the two-dimensional joint eigenvector at time t, consisting of two continuous variables; For the joint feature x t The corresponding discrete label number; μ k Let be the k-th cluster center in the joint space.

[0097] Taking drilling pump pressure (SPP) as an example, we set K=3 for clustering:

[0098] Low pressure range, SPP ≤ 30 Mpa, corresponding to label 0;

[0099] Medium pressure range, 30 MPa < SPP ≤ 60 Mpa, corresponding to label 1;

[0100] High pressure range, SPP > 60 Mpa, corresponding to label 2.

[0101] The final generated label sequence is like {0, 0, 1, 2, 1, 0...}, which is used as the model input or the prediction target for the self-supervised task.

[0102] The input at each time step t is represented as a multi-dimensional discrete label vector:

[0103]

[0104] All time steps form the input label matrix: [[ID=SI19]]

[0105] <00003SO>

[0106] Among them, represents the discrete label value of the i-th variable at time step t; T is the total number of time steps in the time series; n is the number of parameters of the input variable; X (L) is the final discretized label sequence matrix and is the main input of the pre-trained model.

[0107] In step S3, the input of the model is the multi-variable discrete label sequence matrix processed in step S2:

[0108]

[0109] Among them, represents the discrete label vector at time step t; T is the sequence length; n is the number of dimensions of the input variable.

[0110] To enable the model to effectively capture the temporal dependence relationship, the input sequence is usually constructed by means of a sliding window with a fixed step size. Let the current time step be t and the historical window length be T past , then the input sequence is defined as:

[0111]

[0112] Among them, is the multi-variable discrete label input sequence matrix formed by taking time step t as the current moment and looking back T past time steps, is the multi-variable discrete label vector at time step t, which comes from the output result after clustering encoding in step S2; T past is the time window length; n is the number of variable dimensions.

[0113] To adapt to the Transformer structure, each discrete label vector is embedded and superimposed with sine and cosine positional encodings to construct the final input tensor:

[0114]

[0115] Among them, Z t This refers to the multivariate input feature tensor formed within the historical window at time step t after embedding and positional encoding. d model This represents the embedding dimension of the Transformer model.

[0116] The main body of the model constructed in this invention is a multi-layer Transformer encoder structure, and each layer contains the following modules:

[0117] A multi-head self-attention mechanism is used to capture the state dependencies and mode transitions between time steps in the input sequence. The calculation process is as follows:

[0118]

[0119] Where Q, K, and V represent the query, key, and value matrices, respectively, generated from the input vector through a linear transformation; d k is the dimension of the key vector, used as a scaling factor; the Attention result is used to represent the dependency between the current time step and other positions.

[0120] Each sub-layer in the Transformer uses a combination of residual connections and layer normalization to stabilize the model training process. The initial residual structure is as follows:

[0121] z l =LayerNorm(z l +Attention l (·))

[0122] Where l is the layer number of the current Transformer encoder; z l The input features for this layer (i.e., the outputs before the l-th layer); Attention l (·) represents the multi-head attention output of this layer; z l +Attention l (·) is a residual connection operation that preserves the original feature information; LayerNorm(·) is a layer normalization operation that makes the output have a uniform distribution and improves training stability.

[0123] After connecting to the feedforward neural network module, a second residual structure is formed:

[0124] H l=LayerNorm(z l +FFN(z l ))

[0125] Among them, Z l The output of the previous sublayer (the normalized Attention result); FFN(·) is the feedforward neural network module, containing two layers of linear transformations and activation functions; Z l +FFN(·) represents residual connections, enhancing the model's nonlinear expressive power; H l This is the final output of this layer, which is then passed to the next layer.

[0126] To achieve effective modeling of multidimensional time-series well site data under unsupervised conditions, this invention reserves a self-supervised training interface mechanism in the model structure design. This mechanism allows the introduction of various self-supervised task modules during the model training phase, including but not limited to mask label prediction, state comparison learning, and sequence judgment tasks.

[0127] The output branches, attention representations, and embedding layer interfaces required for various task objectives are all fully configured in the model architecture, enabling automatic integration with task samples during training. Through these modules, the model can not only complete pre-training without manual annotation but also possesses a certain degree of transfer learning and task-specific generalization capabilities.

[0128] In step 4, the self-supervised task design, based on multivariate discrete label sequences, designs a set of self-supervised training mechanisms oriented towards state recognition, trend modeling, and time structure perception. This enables the model to effectively learn representations through task-driven learning even in the absence of manual annotation, and to possess multi-condition generalization and trend prediction capabilities. To achieve label-free training, this invention summarizes the well site discrete label modeling task into four core self-supervised tasks: (1) mask label prediction task (recovering missing states); (2) state comparison learning task (reducing the representation of working conditions); (3) sequence judgment task (structural sequence perception); and (4) future state prediction task (trend modeling capability), which are used to enhance the model's time sequence perception and working condition discrimination capabilities.

[0129] (1) Masked Label Prediction Task: Randomly mask some label positions in the input sequence, denoted as [MASK], and require the model to recover them based on contextual reasoning. The loss function is defined as:

[0130]

[0131] in, This represents the loss for mask label prediction; This represents the set of all masked label positions; This represents the model's predicted output for the masked location; x i,trepresents the original true label; CE(·) represents the cross-entropy loss function.

[0132] (2) State Comparison Learning Task: Samples belonging to the same working condition or similar stage are brought closer together in the representation space, enabling the model to learn the "representational similarity" of working conditions. Two segments with similar working conditions are extracted from the same well site time series as positive sample pairs; negative sample pairs are constructed from data from different time periods or different wells. The representation is compressed into a low-dimensional feature vector z using a projection head. i The contrast loss is defined as:

[0133]

[0134] in, z represents the contrastive learning loss. i ,z j For positive sample pairs; z k is the candidate negative sample representation; sim(·) is the cosine similarity; τ is the temperature coefficient.

[0135] (3) Sequence judgment task: Select a piece of data from the original sequence. The order of the sequences is randomly shuffled, and a binary classification task is constructed to determine whether the current sequence order is correct.

[0136]

[0137] in, Indicates the loss based on the order of judgment, y order ∈{0,1} represents the true order label; Let y be the model output probability; BCE(·) is the binary cross-entropy loss function. order A value of 1 indicates that the order is correct, while 0 indicates that the order has been disrupted.

[0138] (4) Future state prediction task: Predict the discrete state of several future time steps through historical state windows, giving the model "trend reasoning ability".

[0139] Model input: History window X past ;

[0140] Model output: Future label sequence

[0141] Real tag: X future ;

[0142] The loss function is as follows:

[0143]

[0144] in, T represents the loss from predicting future states.future This represents the length of the future prediction time step; n is the dimension of the variable. This represents the label of the i-th variable predicted by the model in the j-th step. This is a real label.

[0145] The parallel training of the above four types of tasks together constitutes the final training objective function:

[0146]

[0147] in, This represents the joint self-supervised loss function for multiple tasks. This represents the loss for mask label prediction; Indicates the contrast learning loss; Indicates the order of loss assessment. λ1, λ2, and λ3 represent the weighting coefficients of the task loss (the recommended initial value is 1, which can be adjusted).

[0148] The self-supervised training process can be summarized as follows: construct multi-task training samples (masking, perturbation, fragment pairs, history windows); input the data into the constructed Transformer model; calculate the loss term for each output through the corresponding task header; and drive backpropagation with the total loss to complete end-to-end training.

[0149] In step S5, the pre-training process uses a structured discrete-label dataset and employs a multi-task self-supervised mechanism to train the model end-to-end. The goal is to maximize the accuracy of various prediction tasks, drive model parameter optimization, and enhance the model's ability to represent and generalize to complex well site conditions. Model pre-training includes the following:

[0150] (1) Model pre-training process

[0151] This invention employs a label-free, self-supervised pre-training strategy, and the core process is as follows:

[0152] Data preparation: A structured discrete label dataset is constructed from the multivariate time-series data collected from the well site (see step S2 for details). The data is standardized and synchronously sampled to form... Training samples in the specified format.

[0153] Multi-task sample construction: Combining the four types of tasks (mask prediction, contrastive learning, sequence judgment, and future prediction) defined in step S4, the input sequence is processed into tasks to generate multi-dimensional training objectives.

[0154] Model initialization and forward propagation: Based on the Transformer structure built in step S3, the model will receive the embedded discrete label sequence, model its temporal dependencies through the attention mechanism, and output the representation or prediction results.

[0155] Loss function-driven training: The training objective is to minimize the joint loss function across multiple tasks.

[0156]

[0157] The definitions of the parameters in the loss function above are consistent with the final training objective function in step S4, which can be found in the "Parameter Explanation" section of step S4.

[0158] Parameter update mechanism: The AdamW optimizer is used to update the model parameters θ. The update formula is as follows:

[0159]

[0160] Where, θ (t+1) θ represents the parameter vector of the model at the (t+1)th iteration; (t) This represents the parameter vector of the model at the t-th iteration; η is the base learning rate set during the pre-training phase of the model. The multi-task joint self-supervised loss function is defined (see step S4). This indicates the gradient calculation of the parameters.

[0161] (2) Training parameter configuration and convergence strategy

[0162] To improve training efficiency and stability, this invention recommends the settings shown in Table 1:

[0163] Table 1 Parameter Configuration Table

[0164] parameter meaning Recommended range d Univariate Embedding Dimension 32-128 L Transformer layer number 3-6 h Number of attention heads 4-8 τ Comparison of task temperature coefficients 0.05-0.2 <![CDATA[λ1,λ2,λ3]]> Multi-task weight The default value is 1, which can be adjusted. learningrate Learning rate Initially 1e-4, supports cosine annealing batchsize Batch size 64-256

[0165] The criteria for model convergence include: a gradual decrease in total loss; stable multi-task accuracy on the validation set; and a smooth single-task loss curve with no obvious oscillations.

[0166] (3) Model validation and deployment optimization

[0167] After pre-training, the model will be evaluated on the independent validation set for performance on four types of tasks, and the model with the best overall performance will be selected for deployment.

[0168] (4) Online fine-tuning and rolling optimization mechanism

[0169] This invention supports online model evolution capabilities, collecting new data through a rolling window to achieve long-term adaptive behavior after deployment. The formula is as follows:

[0170]

[0171] Where, θ online This is the set of model parameters after online fine-tuning and optimization, where θ0 represents the historical model parameters; α represents the newly sampled data for the rolling cycle; α is the fine-tuning learning rate.

[0172] In step S6, after completing model construction, pre-training and deployment, the pre-trained model is applied to the real-time analysis of actual well site drilling data. By utilizing its deep feature extraction and time series modeling capabilities obtained through self-supervised learning, it enables anomaly detection, state identification and trend prediction for key working conditions, supporting intelligent drilling decision-making.

[0173] The intelligent well site modeling system based on a self-supervised pre-trained model constructed in this invention can be widely applied to various real-time analysis and decision support scenarios. This model can fully utilize the input multi-dimensional discrete label sequence to complete tasks such as drilling parameter anomaly detection, operating condition identification, and future state trend prediction in real time, supporting intelligent drilling management.

[0174] The pre-trained model is deployed on edge computing devices at the well site, enabling online inference and processing of real-time acquired well site data. Simultaneously, the deployed model integrates with the on-site SCADA system via data interfaces or communication protocols (such as OPC-UA and Modbus) to achieve automatic data transmission and result feedback. The deployed model is then used for online processing and feedback verification of the real-time acquired data. Specific application methods are as follows:

[0175] The model can be used for drilling parameter anomaly detection. Based on the mask prediction task logic, a portion of the real-time input data is selected as a "mask region," and the model recovers the labels. If the predicted result differs significantly from the true label, it is considered a potential anomaly. The anomaly indication function is:

[0176]

[0177] Where, δ i,t This represents the anomaly indicator value of the i-th variable at time step t. If the predicted value is inconsistent with the true label, it is 1; if the predicted value is consistent with the true label, it is 0. An indicator function that checks if the condition inside the parentheses is true. If the condition is true, the function returns 1; otherwise, it returns 0. This represents the discrete label value of the i-th variable predicted by the model at time t; Represents the actual discrete label value;

[0178] When the cumulative abnormal score exceeds the threshold θ alert When this happens, the system will trigger an alarm or provide an operational intervention prompt;

[0179] The model can perform operational condition identification and status judgment. To maintain the consistency of the unsupervised architecture, this invention preferably uses a clustering method to analyze the internal feature representation of the model and identify the operational condition status in different time periods.

[0180] By utilizing the attention distribution and hidden layer representation output from the intermediate layers of the Transformer, combined with a clustering method similar to that used for discrete labeling of continuous variables in step S2, the current work status can be intelligently identified. The work status identification module of this invention possesses the following core capabilities:

[0181] Automatically identify typical operating conditions during drilling operations (such as circulating drilling, connecting single joints, tripping in and out of the hole, etc.), and support further differentiation between normal operation and non-steady-state disturbances;

[0182] Real-time monitoring of operating condition switching status to capture dynamic evolution trends in the drilling process;

[0183] Clustering operations are performed based on the similarity of temporal feature representations to achieve unsupervised division of working conditions.

[0184] In addition, this method does not rely on manual tags, can be adapted to different well sites, blocks and process environments, and has the ability to be quickly deployed and migrated, demonstrating good engineering practicality and system scalability.

[0185] Furthermore, the model supports short-term trend prediction. By activating the future state prediction task module, the model can predict the evolution trend of state labels several time steps in the future based on the sliding window input sequence.

[0186]

[0187] in, This represents a sequence of labels for several future time steps predicted by the model. This represents a pre-trained model with parameter θ; θ pretrained This represents the set of model parameters after self-supervised training and optimization. This represents the sliding window input sequence before time t.

[0188] This function can be used to determine whether drilling parameters are approaching high-risk conditions; to provide early warning of operational trends; and to provide prediction-based auxiliary parameter adjustment suggestions.

[0189] The deployed model can be validated using the methods shown in Table 2:

[0190] Table 2 Verification Methods

[0191] Task type Validation metrics Method Description Mask recovery Accuracy, Error Rate Comparison with real labels Operating condition identification F1 score, precision Comparison with manual annotations / logs Trend Forecast Match rate, Top-1 accuracy Align with subsequent actual trends

[0192] To adapt to the model evaluation needs at different stages and in different scenarios, this system supports the following two verification modes:

[0193] Offline verification mode: Suitable for the evaluation stage after model training. It calculates the accuracy of various tasks (such as mask recovery, trend prediction, etc.) by performing batch evaluation tasks on historical drilling datasets, and is used for tuning and structural comparison analysis.

[0194] Online sliding window evaluation mode: In actual deployment, the system can perform real-time inference and result comparison on the input data based on a fixed sliding window (such as the most recent 30 time steps), and dynamically evaluate the model's operating status by combining label feedback (if available) or trend delay confirmation mechanism, to assist in maintenance and alarm adjustment.

[0195] In step S7, after model deployment, a model optimization and iteration mechanism is introduced to continuously improve the model's adaptability in different periods and well site environments, thus constructing a drilling intelligent system with long-term evolutionary characteristics. This mechanism combines real-time data updates, self-supervised task feedback, and historical evaluation results, employing a sliding update and multi-task fine-tuning strategy to periodically optimize model parameters and structure.

[0196] In actual deployment, well site conditions and data distribution may fluctuate over time, leading to decreased model prediction performance or identification bias. To address this "non-stationary evolution" problem, this invention designs a sliding window-based data update method. The system collects the latest time-series data at fixed time intervals (e.g., every 24 hours) and performs discrete tagging processing to form an updated dataset. Its structure is consistent with the pre-training stage, ensuring uniformity in input format and model compatibility between new and old data. The sliding window can be defined as:

[0197]

[0198] Among them, T win The length of the sliding window. It is a multivariate discrete label vector at time step t, with the same structure as step S2.

[0199] Based on the updated dataset, the model parameters will be initialized with the current version θ0 and updated using a low learning rate fine-tuning method while maintaining the model structure. To enhance the dynamic adjustment capability of self-supervised tasks and demonstrate the advantages of the multi-task structure of this invention, the following optimized online update formula is proposed:

[0200]

[0201] Where θ0 represents the current model parameters; θ onlineThis represents the deployed model after fine-tuning and updating; α is the fine-tuning learning rate, which is much smaller than that in the initial training phase. This is a set of self-supervised tasks, corresponding to the four types of training tasks defined in step S4; The dynamic weight of task j at time step t; This represents the loss function corresponding to task j; This is the latest collected discretized dataset.

[0202] Through the above optimized structure, the system can dynamically adjust the model's focus based on actual working conditions. For example, it can automatically increase λ when the anomaly detection accuracy decreases. mask When trend prediction results are unstable, enhance λ forecast This improves the model's target performance.

[0203] To ensure the continuity and security of system operation, this invention further integrates model version management and rollback mechanisms. Each round of model optimization records the parameter version and timestamp. If performance degrades or the false positive rate increases within multiple consecutive sliding window cycles, the system can automatically roll back to the previous validated optimal model version, ensuring the safety and reliability of drilling operations.

[0204] This invention also provides an application of the above-described construction method in the field of intelligent monitoring of oil and gas drilling engineering. The method for constructing a pre-trained model for real-time well site data can perform parameter anomaly detection, operational condition identification, and drilling trend prediction based on real-time time-series data collected by well site sensors, thereby enabling intelligent early warning and parameter optimization decision-making to support the drilling system.

[0205] Example 1

[0206] According to the method for constructing a real-time well site data pre-training model as described in this invention, it was applied in a development block of the Bohai Oilfield. This block consists of deep sandstone reservoirs at a depth of approximately 3500m-4200m, with high formation pressure coefficients and poor wellbore stability, requiring high levels of real-time drilling monitoring and parameter prediction. This embodiment uses the horizontal well "A24-1-A1" in this block as the object for full-process deployment and testing.

[0207] During the drilling operation phase at depths of 2630m to 3950m, edge computing acquisition terminals were deployed to collect and upload drilling parameters in real time, including: weight on bit (WOB), rotational speed (RPM), torque (TQ), rate of drilling (ROP), pump pressure (SPP), standpipe pressure (SP), inlet / outlet flow rate (FlowIn / Out), drilling fluid density (MW), active tank volume (PVT), casing pressure (CP), and wellhead temperature (TW). The sampling frequency was set to 5Hz, and approximately 470 hours of data were collected, totaling approximately 8.5 million time steps of high-frequency, multivariate data.

[0208] In step S1, after the collected data is uploaded through the SCADA system, the following preprocessing procedures are performed: ① Packet loss interpolation: the maximum gap does not exceed 3s, and the bidirectional sliding window interpolation method is used; ② Standardization and normalization processing: all variables are normalized by Z-score, and the outlier removal range is ±3σ; ③ Time alignment: all data are aligned with the master clock (drilling speed sensor) as the reference, and other variables are aligned to it, with an error not exceeding ±200ms.

[0209] In step S2, the sliding window K-Means clustering method is used to independently model each variable, dividing the continuous variable into 35 state intervals to generate a discrete label set. Taking SPP as an example, the cluster centers correspond to low pressure (<14MPa), medium pressure (14-18MPa), and high pressure (>18MPa), respectively, and are labeled as 0, 1, and 2. Figure 2 As shown, a discrete labeling diagram of the variable within the actual sampling time step is presented, clearly demonstrating the label mapping relationship of the continuous variable in different intervals. The clustering window size is 1500 time steps (approximately 5 minutes), and the update frequency is once every 15 minutes.

[0210] In step S3, an input sequence window of length T is constructed. past =30, meaning the past 30 time steps (corresponding to 6 seconds of data) are used as input to form the input tensor:

[0211]

[0212] Each label embedding has a dimension of 64. After positional encoding, it is fed into a two-layer Transformer structure. Each layer contains a four-head attention mechanism and a feedforward network, maintaining a consistent output dimension. Figure 3 As shown in the Transformer structure diagram, the Transformer-based model structure constructed in this invention integrates embedding layers, positional encoding, and multi-task output mechanisms, and has good sequence modeling and task generalization capabilities.

[0213] In step S4, four self-supervised tasks are set up with a mask ratio of 20, λ1 = 0.6, λ2 = 0.5, and λ3 = 1.0, and the following joint loss optimization objective is adopted: Input the data into the constructed Transformer model; calculate the loss term for each output through the corresponding task header; the total loss drives backpropagation to complete end-to-end training.

[0214] In step S5, the model is pre-trained for 40 epochs on offline training data (the first 300 hours), using the following parameter update strategy: The learning rate is set to 1×10. -4The optimizer is Adam, and the batch size is 128.

[0215] After deployment, execute step S6 to apply the model to the remaining 170 hours of job data. Anomaly detection infers the following metrics through a masking task: When the proportion of outliers exceeds θ within 15 consecutive time steps alert =0.3, the system triggers an early warning. In practical applications, the model successfully identified three abnormal increases in SPP, with an average prediction lead of approximately 45 seconds and an accuracy rate of 93.6%.

[0216] In operational condition identification, by performing K-Means clustering on the Transformer intermediate layer representation vector, the system successfully distinguished six operational states (circular drilling, connecting single joints, tripping, running down, pump stop, and unidentified), achieving a matching accuracy of 91.4% with manual logs. Figure 4 As shown in the model deployment process, the pre-trained model achieves intelligent recognition and dynamic optimization in well site applications through edge deployment and sliding window mechanism, forming a closed loop of "acquisition-prediction-feedback-retraining".

[0217] In step S7, the system performs sliding window fine-tuning training on the latest sampled data every 24 hours. The data volume is approximately 430,000 records, and the following optimization formula is used:

[0218]

[0219] Where α = 5 × 10 -5 Only the upper layer of the Transformer and the output task header are updated; the underlying structure is frozen. Within 14 days of deployment, the system model showed no prediction drift and exhibited good adaptability.

[0220] Verification shows that the method of the present invention has excellent real-time modeling capabilities and deployment adaptability in actual well sites, and is especially suitable for intelligent prediction tasks in high-frequency parameter changes and labelless operation stages, and has good prospects for industrial promotion.

[0221] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for constructing a wellsite real-time data pre-training model, characterized in that, The method comprises the following steps: Step S1, data acquisition and preprocessing: acquiring real-time continuous time series data generated during the drilling process of the well site, and preprocessing the real-time continuous time series data; Step S2, discrete tokenization: dividing the preprocessed continuous time series data into multiple numerical intervals, and assigning a unique discrete label to each interval to generate a discrete label sequence; Step S3, model construction: constructing a time series prediction model based on the Transformer architecture, taking the discrete label sequence as input, to model the time dependence and state change pattern of the drilling parameters; Step S4, self-supervised task design: designing a self-supervised learning task for pre-training, including a mask label prediction task, a state contrast learning task, a sequence judgment task, and a future state prediction task, taking the current label sequence and the historical label sequence as input to predict the label state at the future time; Step S5, model pre-training: using the discretely tokenized data to pre-train the time series prediction model in a self-supervised manner, and updating the model parameters by optimizing the prediction accuracy and feature expression ability; Step S6, application and verification: deploying the pre-trained model on the well site to realize real-time anomaly detection, working condition recognition and future state prediction of the drilling parameters, and verifying the model effect according to the field data; Step S7, model optimization and iteration: based on the verification result, the pre-trained model is optimized and trained again to update the model structure or parameters, so as to improve the adaptability and prediction accuracy of the model in actual working conditions.

2. The method of claim 1, wherein the method comprises: In step S1, the well site system acquires real-time continuous time series data in the drilling process by deploying various industrial sensors on the ground and in the well, including: Mechanical parameters: drill depth, mechanical drilling speed, hook load, drilling pressure, rotation speed, torque, pump speed; Hydraulic parameters: flow rate, active pool volume, pump pressure, drilling fluid density; Temperature parameters: wellhead temperature, bottom hole temperature estimate; Control parameters: pump status, valve opening, operation mode; All data are sampled synchronously at a unified time step of 1Hz-5Hz, uploaded to the data server through SCADA system or OPC-UA communication protocol / Modbus communication protocol, and formed into a structured multi-channel time series data set to serve as the basis for subsequent modeling; The acquired real-time continuous time series data need to be standardized and preprocessed before inputting into the model, including denoising, outlier removal, linear interpolation, Z-score normalization, and time alignment, to finally form a training sample set with consistent format and complete fields.

3. The method of claim 1, wherein the method further comprises: determining a pre-training model based on the real-time data of the well site; and training the pre-training model based on the real-time data of the well site. In step S2, the interval division is performed by K-means clustering, and each interval is assigned a unique integer coded label; To adapt to the self-supervised modeling framework, the continuous time series data are converted into discrete label sequences, and for each continuous variable, a weighted sliding window K-Means clustering algorithm is used to construct data distribution centers, and the discrete label coding function is: wherein x t represents the original input value at time t; is x t corresponding discrete label number, such as 0, 1, 2; is the kth cluster center within the time window.

4. The method of claim 3, wherein the method further comprises: In step S3, a multivariate time series modeling based on the Transformer structure is adopted, and the input sequence used for modeling is obtained by a fixed step sliding window method from the continuous multivariate discrete label sequence, and the input sequence is constructed in the following way: wherein, is a multivariate discrete label input sequence matrix composed of a plurality of time steps t, t-1, t-2, t-T past is a multivariate discrete label input sequence matrix composed of a plurality of time steps t, t-1, t-2, t-T is a multivariate discrete label vector at time step t, derived from the output result after clustering encoding in step S2; T past is a time window length; n is a variable dimension number; In order to adapt to the Transformer structure, each discrete label vector is embedded and superimposed with sine and cosine position encoding to construct the final input tensor: wherein Z t is the multivariate input feature tensor formed after the embedding layer and the positional encoding processing within the history window at time step t, d model is the embedding dimension of the Transformer model.

5. The method of claim 1, wherein the method further comprises: In step 4, the four tasks of mask label prediction, state contrast learning, sequence judgment and future state prediction are trained in parallel to form the final training objective function: wherein, denotes a multi-task joint self-supervised loss function, denotes a mask label prediction loss; denotes a contrastive learning loss; denotes a sequential judgment loss, denotes a future state prediction loss, and λ1, λ2, λ3 are weighting coefficients of the task losses.

6. The method of claim 5, wherein the method further comprises: In step S5, the pre-training phase takes the structured discrete label sequence as input and optimizes the end-to-end model under the driving of the self-supervised task, and the model training objective is to maximize the prediction ability and representation ability of each task; A multi-task loss joint optimization mechanism is introduced, and the following update formula is set: where θ (t+1) denotes the parameter vector of the model at the t+1th iteration; θ (t) denotes the parameter vector of the model at the tth iteration; η is the base learning rate set for the model in the pre-training stage; is a multi-task joint self-supervised loss function; denotes the gradient calculation of the parameters.

7. The method of claim 1, wherein the method further comprises: In step S6, the model completed by pre-training is deployed on the edge computing device of the well site, and the real-time collected well site data is processed and inferred online by the computing device; at the same time, the deployed model is integrated with the on-site SCADA system through a data interface or a communication protocol to realize automatic data transmission and result feedback, and the real-time collected data is processed and verified online using the deployed model, and the online processing and feedback verification content includes: Abnormality detection: deviation judgment is performed on the current data by using the mask recovery ability: wherein, δ i,t represents the anomaly indication value of the ith variable at time step t, 1 if the predicted value is inconsistent with the true label; 0 if the predicted value is consistent with the true label; an indicator function, which judges whether the condition in the parentheses is true or not, 1 if the condition in the parentheses is true, 0 if the condition in the parentheses is not true; represents the discrete label value of the ith variable predicted by the model at time t; represents the true discrete label value; when the cumulative anomaly score exceeds the threshold value θ alert , the system triggers an alarm or gives an operation intervention prompt; Working condition recognition: unsupervised clustering is performed through the intermediate layer representation of the model to automatically identify drilling, single connection, tripping and other working conditions; Trend prediction: the state at future time points is predicted based on the current sliding window, and the formula is: wherein, represents a sequence of future time step labels predicted by the model; represents a pre-trained model parameterized by θ; pretrained represents a set of model parameters optimized through self-supervised training; represents a sliding window input sequence up to time t.

8. The method of claim 6, wherein the method further comprises: determining a pre-training model based on the real-time data of the well site; and training the pre-training model based on the real-time data of the well site. In step S7, an online optimization mechanism is introduced to cope with the performance degradation problem caused by the change of well site data distribution over time; the latest data is collected through a sliding window And taking the pre-trained model θ0 as the initial parameters, based on the self-supervised task, the low-frequency fine-tuning is carried out, and the online update formula after optimization is: wherein θ0represents the current model parameters; θ online denotes the fine-tuned updated deployment model; a is the fine-tuning learning rate, which is much smaller than the initial training phase; is a set of self-supervised tasks, corresponding to the four types of training tasks defined in step S4; is the dynamic weight of task j at time step t; denotes the loss function corresponding to task j; is the latest collected discretized data set.

9. The use of the method for constructing a wellsite real-time data pre-training model according to any one of claims 1-8, characterized in that: It is applied to the field of intelligent monitoring of oil and gas drilling engineering.

10. The application of the method for constructing a wellsite real-time data pre-training model according to claim 9, characterized in that: The construction method of the well site real-time data pre-training model can perform parameter abnormality detection, operation working condition recognition and drilling trend prediction based on real-time time series data collected by well site sensors.

Images