Well leakage warning model construction method based on non-stationary Transformer, well leakage warning method and related device

Abstract

The present invention relates to the technical field of well leakage warning, which is a well leakage warning model construction method based on non-stationary Transformer, a well leakage warning method and related devices. The well leakage warning method comprises obtaining a real-time well leakage data sequence of a single well; pre-processing the real-time well leakage data sequence, intercepting the real-time well leakage data sequence using a sliding window, and obtaining a plurality of window samples to be predicted; inputting the plurality of window samples to be predicted into a well leakage warning model, and obtaining a predicted probability of well leakage; comparing the predicted probability of well leakage with a set threshold, and performing well leakage warning according to the comparison result. The present invention introduces a Transformer network to construct a model, and adopts a combination of self-supervised pre-training and fine-tuning training to perform model training to obtain a well leakage warning model, so that the well leakage warning model effectively extracts the non-stationary information of the well leakage data based on the characteristics of the Transformer network, and improves the accuracy of well leakage warning.

Description

Technical Field

[0001] The present invention relates to the technical field of well leakage warning, and specifically to a well leakage warning model construction method based on a non-stationary Transformer, a well leakage warning method and related devices. Background Art

[0002] In the digital information age, it is becoming more and more a trend to use computers and automation technology to deal with various problems encountered in oil and gas drilling. In the face of downhole leakage problems, improper treatment measures will lead to low plugging success rate, continuous leakage of drilling fluid, increased loss of on-site working time, and even well abandonment. Frequent well leakage problems consume a lot of construction time, and plugging construction will increase the drilling cycle and greatly increase drilling costs.

[0003] At present, the drilling process is accompanied by various random, fuzzy, uncertain and uncontrollable factors. With the extension of the mining scope, the improvement of production capacity and the increase of mining depth, the underground operation situation is becoming more and more complicated, the underground water inflow is further increased, and the leakage problem is more prominent. In the face of the leakage problem, if the treatment measures are not appropriate, it will lead to a low plugging success rate, continuous leakage of drilling fluid, increased loss of on-site working time, and even lead to well abandonment.

[0004] With the development of digital information technology, the use of computers and automation technology to warn of well leakage has become a trend. The existing methods for well leakage warning are mostly to use convolutional neural networks, LSTM and other algorithms for deep learning, build well leakage warning models, and complete well leakage warning. For example:

[0005] The first existing public patent document, with the publication number CN118410701A, discloses an interpretable continuous learning well leakage prediction method based on formation lithology reconstruction, including: S1, constructing a geological engineering data set based on well depth, reconstructing, and obtaining a geological engineering data set based on formation lithology; S2, extracting different lithology formation data according to lithology, and extracting typical well leakage characteristics of different lithology formations by signal time-frequency analysis method; S3, establishing an excitation function of well leakage characteristics of different lithology formations based on typical well leakage characteristics of different lithology formations, and constructing an excitation function adaptive evolution mechanism based on multiple excitation functions, using this mechanism as a continuous learning mechanism of the well leakage prediction model, and verifying and optimizing the well leakage prediction model; S4, inputting the target well design data into the well leakage prediction model to obtain the output result. The present invention solves the problem of poor universality of the traditional machine learning well leakage prediction method and improves the accuracy of well leakage prediction.

[0006] The second existing public patent document, with the publication number CN117540277B, discloses a well leakage warning method based on the WGAN-GP-TabNet algorithm, including: collecting field data; screening characteristic parameters with strong correlation with the leakage flow and deleting parameters with weak correlation in the field data, thereby forming initial parameters; grading the initial parameters according to the leakage flow, and dividing each level into majority class and minority class according to the number of parameters at each level after grading; inputting the initial parameters into the WGAN-GP model to generate minority class data; using the initial parameters and the generated minority class data to train and evaluate the well leakage warning model TabNet; collecting field data and using the trained TabNet model to predict the degree of leakage. The present invention effectively solves the problem of low recall rate and low minority class prediction accuracy caused by class imbalance in deep learning for leak prevention and plugging, and has the advantages of stability and reliability, high accuracy, easy operation, fast response speed, and strong portability.

[0007] There is an existing public patent document three, with publication number CN110766192B, which discloses a drilling well leakage prediction system and method based on deep learning, the method comprising: 1) acquiring drilling production operation data; 2) preprocessing the acquired data to obtain sample data for deep learning; 3) feature extraction and classification training: using the VGG convolutional neural network model for deep learning, and obtaining a trained drilling well leakage prediction model through feature extraction and classification training; 4) result output and result confirmation: using the trained drilling well leakage prediction model to verify the real-time verification data and give a well leakage judgment result.

[0008] The above technologies mainly have the following problems to be solved:

[0009] Long-term dependency of historical well data: There are rapid losses such as micro-loss and formation fracturing during the drilling process. Micro-loss can only be found over a longer time step, but the above model cannot solve the long-term dependency of time steps well, so it is difficult to capture the relevant characteristics of micro-loss;

[0010] Unable to obtain the non-stationary characteristics of well leakage data: Some well leakage data (such as total pool volume) are non-stationary, and the existing model cannot effectively extract non-stationary information, thereby reducing the ability to identify well leakage events;

[0011] Mismatch between well leakage warning and modeling tasks: The binary classification task is to classify well leakage and normal samples. Under ideal circumstances, the existing model can only identify whether a well leakage occurs at a certain moment, and cannot effectively achieve early warning of well leakage. Summary of the invention

[0012] The present invention provides a method for constructing a well leakage warning model based on a non-stationary Transformer, a well leakage warning method and related devices, which overcome the shortcomings of the above-mentioned prior art and can effectively solve the problem that the existing well leakage warning method cannot effectively extract non-stationary features, resulting in inaccurate well leakage warning.

[0013] One of the technical solutions of the present invention is achieved by the following measures: a method for constructing a well leakage warning model based on a non-stationary Transformer, comprising:

[0014] Acquire a first sample set and a second sample set, wherein the first sample set includes a plurality of unlabeled samples, the unlabeled samples are a lost circulation data sequence, and the second sample set includes a plurality of labeled samples, each labeled sample includes a lost circulation sample and a sample label indicating whether lost circulation occurs;

[0015] Using the first sample set to perform self-supervisory pre-training on the first network model to obtain a second network model, wherein the first network model is constructed using a backbone network established using a Transformer network;

[0016] The head network of the second network model is replaced to obtain a third network model, and the third network model is fine-tuned and trained using the second sample set to obtain a well leakage warning model.

[0017] The following are further optimizations and / or improvements to the above technical solutions:

[0018] The process of obtaining the first sample set includes:

[0019] Acquire the historical data of well leakage of multiple wells through frequency sampling, wherein the historical data of well leakage includes various types of well leakage data that show abnormal changes before the well leakage occurs or show obvious changes after the well leakage occurs;

[0020] The time when the well leakage occurs and the time when the well leakage ends are marked in the well leakage history data of each well, and a time label is assigned to each time of the well leakage history data according to the time when the well leakage occurs and the time when the well leakage ends, so as to form a corresponding well leakage data sequence, wherein the time label includes a time label when the well leakage occurs and a time label when the well leakage does not occur;

[0021] Each lost circulation data sequence is preprocessed to obtain a plurality of unlabeled samples to form a first sample set.

[0022] The process of obtaining the second sample set includes:

[0023] Acquire the historical data of well leakage of multiple wells through frequency sampling, wherein the historical data of well leakage includes various types of well leakage data that show abnormal changes before the well leakage occurs or show obvious changes after the well leakage occurs;

[0024] Marking the time when the well leakage occurs and the time when the well leakage ends in the well leakage history data of each well, and assigning a time label to each time of the well leakage history data according to the time when the well leakage occurs and the time when the well leakage ends, wherein the time label includes a time label when the well leakage occurs and a time label when the well leakage does not occur;

[0025] Preprocess the labeled historical data of well leakage;

[0026] Set the size and step length of the sliding window, and use the sliding window to intercept all historical data of well leakage to obtain multiple window samples;

[0027] Calculate the time label mean of all historical data of well leakage in each window sample, retain the window sample with the time label mean of well leakage or no well leakage, and use each window sample as a well leakage sample, and use the time label mean of each window sample as the corresponding sample label to obtain multiple labeled samples to form a second sample set.

[0028] The above preprocessing includes numerical type conversion, Z-score normalization, NaN value filling and time downsampling.

[0029] The process of performing self-supervised pre-training on the first network model using the first sample set is the same as the process of performing fine-tuning training on the third network model using the second sample set, wherein the process of performing self-supervised pre-training on the first network model using the first sample set includes:

[0030] Divide the first sample set into a training set, a validation set and a test set;

[0031] The first network model is trained using the training set, and a loss function is introduced during the training. When the value of the loss function is stable, the training is terminated to obtain the second network model. The first network model is constructed using the backbone network established using the Transformer network, including extracting the destabilization factor from the normalized input data through two multi-layer perceptrons. and , the normalized input data is mapped from low dimension to high dimension through the embedding layer, based on the stationary factor and The high-dimensional input data is encoded through a non-stationary Transformer encoder to obtain high-dimensional semantic features, and the high-dimensional semantic features are mapped back to the semantic features of the original dimension through a multi-layer perceptron;

[0032] The trained second network model is verified and tested using the verification set and the test set respectively, the model parameters of the second network model are optimized, and the second network model that meets the test evaluation requirements is output.

[0033] When fine-tuning the third network model using the second sample set, the loss function used is as follows:

[0034]

[0035] in, is the true result of the labeled sample in the second sample set, is the prediction result of the model;

[0036] Or / and, the head network of the second network model is replaced with a linear layer to obtain a third network model.

[0037] The second technical solution of the present invention is achieved by the following measures: a well leakage warning method based on non-stationary Transformer, comprising:

[0038] Obtain real-time lost circulation data sequence of a single well in a certain period of time;

[0039] Preprocess the real-time lost circulation data sequence, and intercept the real-time lost circulation data sequence using a sliding window to obtain multiple window samples to be predicted;

[0040] Inputting a plurality of samples of the window to be predicted into a well leakage warning model to obtain a predicted probability of well leakage, wherein the well leakage warning model is constructed by the method described in any one of claims 1 to 6;

[0041] The predicted probability of well leakage is compared with the set threshold, and a well leakage warning is issued based on the comparison result.

[0042] The third technical solution of the present invention is achieved by the following measures: a device for constructing a well leakage warning model based on a non-stationary Transformer, comprising:

[0043] A sample set acquisition unit acquires a first sample set and a second sample set, wherein the first sample set includes a plurality of unlabeled samples, the unlabeled samples are a lost circulation data sequence, and the second sample set includes a plurality of labeled samples, each labeled sample includes a lost circulation sample and a sample label indicating whether lost circulation occurs;

[0044] A first training unit performs self-supervised pre-training on a first network model using a first sample set to obtain a second network model, wherein the first network model is constructed using a backbone network established using a Transformer network;

[0045] The second training unit replaces the head network of the second network model to obtain a third network model, and uses the second sample set to fine-tune the third network model to obtain a well leakage warning model.

[0046] The fourth technical solution of the present invention is achieved by the following measures: a well leakage warning device based on non-stationary Transformer, comprising:

[0047] The unit for acquiring data to be predicted acquires a real-time well leakage data sequence of a single well in a certain period of time;

[0048] The data processing unit to be predicted pre-processes the real-time lost circulation data sequence and intercepts the real-time lost circulation data sequence using a sliding window to obtain a plurality of window samples to be predicted;

[0049] A prediction unit, inputting a plurality of samples of the window to be predicted into a well leakage warning model to obtain a predicted probability of well leakage, wherein the well leakage warning model is constructed by the method described in any one of claims 1 to 6;

[0050] The early warning unit compares the predicted probability of well leakage with the set threshold value and issues a well leakage early warning based on the comparison result.

[0051] The fifth technical solution of the present invention is achieved through the following measures: an electronic device, characterized in that it includes a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the steps in the well leakage warning method based on the non-stationary Transformer.

[0052] The present invention introduces a Transformer network to construct a model, and adopts a combination of self-supervised pre-training and fine-tuning training to perform model training to obtain a well leakage warning model. Therefore, the well leakage warning model can effectively extract non-stationary information in well leakage data based on the characteristics of the Transformer network, and improve the accuracy of well leakage warning. In addition, unlabeled samples are used for self-supervised pre-training during model training, and the model can learn common features to improve the expressiveness of the model. On the basis of self-supervised pre-training, labeled samples are used for fine-tuning supervised training to improve the accuracy, reliability and generalization ability of the model's well leakage warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Attached Figure 1 A schematic diagram of an implementation environment provided for an embodiment of the present invention.

[0054] Attached Figure 2 A schematic flow chart of a method for constructing a lost circulation warning model provided by an embodiment of the present invention.

[0055] Attached Figure 3 A schematic diagram of a flow chart of a method for obtaining a first sample set provided by an embodiment of the present invention.

[0056] Attached Figure 4 A schematic diagram of annotating historical data of lost circulation provided by one embodiment of the present invention.

[0057] Attached Figure 5 A schematic flow chart of a method for obtaining a second sample set provided by an embodiment of the present invention.

[0058] Attached Figure 6 A schematic diagram of sliding window capture provided for an embodiment of the present invention.

[0059] Attached Figure 7 A schematic diagram of a self-supervised pre-training method flow chart provided for one embodiment of the present invention.

[0060] Attached Figure 8 A schematic diagram of a first network model structure provided for an embodiment of the present invention.

[0061] Attached Fig. 9 A schematic diagram of a third network model structure provided for an embodiment of the present invention.

[0062] Attached Fig.10 A schematic flow chart of a well leakage warning method provided by an embodiment of the present invention.

[0063] Attached Fig.11 A schematic diagram of the result of reconstructing a test set using a second network model provided in one embodiment of the present invention.

[0064] Attached Fig.12 A schematic diagram of a well leakage warning result provided by an embodiment of the present invention.

[0065] Attached Fig.13 A schematic diagram of the structure of a device for constructing a lost circulation warning model provided by an embodiment of the present invention.

[0066] Attached Fig.14 A schematic structural diagram of a well leakage warning device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The present invention is not limited by the following embodiments, and specific implementation methods can be determined based on the technical solution of the present invention and actual conditions.

[0068] Those skilled in the art will understand that, unless otherwise stated, a "module" or "unit" in the embodiments of the present invention refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as a processing circuit or memory) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0069] In addition, the term “plurality” in the embodiments of the present invention refers to two or more than two, and “first” and “second” are used to distinguish descriptions and should not be understood as implying relative importance.

[0070] The existing methods for early warning of well leakage using computer technology have the following problems:

[0071] Long-term dependency of historical well leakage data: There are rapid losses such as micro-loss and formation fracturing during the drilling process. Micro-loss can only be discovered over a longer time step, but the existing model cannot solve the long-term dependency of time steps well, so it is difficult to capture the relevant characteristics of micro-loss;

[0072] Unable to obtain the non-stationary characteristics of well leakage data: Some well leakage data (such as total pool volume) are non-stationary, and the existing model cannot effectively extract non-stationary information, thereby reducing the ability to identify well leakage events;

[0073] Mismatch between well leakage warning and modeling tasks: The binary classification task is to classify well leakage and normal samples. Under ideal circumstances, the existing model can only identify whether a well leakage occurs at a certain moment, and cannot effectively achieve early warning of well leakage.

[0074] Therefore, in order to effectively characterize the non-stationary information of some well leakage data and improve the accuracy of well leakage warning, the embodiment of the present invention provides a well leakage warning model construction method based on non-stationary Transformer, a well leakage warning method and related devices.

[0075] The embodiment of the present invention provides a well leakage warning method based on non-stationary Transformer, which obtains a real-time well leakage data sequence of a single well in a certain period of time; pre-processes the real-time well leakage data sequence, and uses a sliding window to intercept the real-time well leakage data sequence to obtain multiple window samples to be predicted; inputs the multiple window samples to be predicted into the well leakage warning model to obtain the predicted probability of well leakage; compares the predicted probability of well leakage with a set threshold, and performs well leakage warning according to the comparison result. The construction process of the well leakage warning model includes: obtaining a first sample set and a second sample set, wherein the first sample set includes multiple unlabeled samples, which are well leakage data sequences, and the second sample set includes multiple labeled samples, each labeled sample includes a well leakage sample and a sample label of whether well leakage occurs; uses the first sample set to perform self-supervised pre-training on the first network model to obtain a second network model, wherein the first network model is constructed by using a backbone network established by a Transformer network; replaces the head network of the second network model to obtain a third network model, and uses the second sample set to perform fine-tuning training on the third network model to obtain a well leakage warning model.

[0076] Among them, the method provided by the embodiment of the present invention may involve artificial intelligence (AI) technology, and can be implemented based on artificial intelligence technology, for example, using a deep learning approach and using sample training to obtain a corresponding model.

[0077] Among them, Machine Learning (ML) is a multi-disciplinary interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence.

[0078] Among them, Deep Learning (DL) specifically refers to machine learning based on deep neural network models and methods. It is developed based on statistical machine learning, artificial neural network and other algorithm models, combined with the development of contemporary big data and large computing power. The most important technical feature of deep learning is the ability to automatically extract features.

[0079] The above-mentioned machine learning and deep learning usually include technologies such as neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0080] Among them, the loss function is used in the process of training the neural network. Because we hope that the output of the neural network is as close as possible to the value we really want to predict, we compare the predicted value of the current neural network with the target value, and then update the weight vector of each layer of the neural network according to the difference between the two (there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the neural network), until the neural network can predict the target value or a value very close to the target value. Therefore, in deep learning, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function.

[0081] As attached Figure 1 As shown, it shows a schematic diagram of an implementation environment provided by an embodiment of the present invention. The implementation environment may include: training equipment and use equipment.

[0082] Both the training device and the use device are computer devices; optionally, the computer device is a terminal device, such as a mobile phone, a tablet computer, a PC (Personal Computer) and other electronic devices; or, the computer device is a server, which can be a single server, a server cluster composed of multiple servers, or a cloud computing service center, which is not limited to the embodiments of the present invention.

[0083] The training device refers to a computer device with the ability to train and learn a neural network. Optionally, the training device has the ability to obtain a neural network, and train and learn it according to application requirements. For example, the training device obtains a neural network from other devices through the network, and then trains it through training samples according to application requirements, so that the neural network has the ability to obtain the predicted probability of the occurrence of a well leakage; Optionally, the training device has the ability to construct a neural network, and it can construct a neural network by itself according to application requirements, and then train and learn it. For example, in order to obtain the predicted probability of the occurrence of a well leakage based on a real-time well leakage data sequence, the training device constructs a neural network by itself, and then trains and learns it through samples according to application requirements.

[0084] The device used refers to a computer device that has the need to use a neural network. Optionally, the device uses the neural network according to application requirements and obtains the neural network from other devices through the network. For example, if the device uses the need to predict the probability of a well leakage, it can obtain a neural network that has been trained and learned to predict the probability of a well leakage from other devices through the network, and use the neural network to predict the probability of a well leakage.

[0085] Based on this, the technical solution of the present invention will be introduced and explained in combination with several examples below.

[0086] Embodiment 1: As attached Figure 2 As shown, the embodiment of the present invention discloses a method for constructing a well leakage warning model based on a non-stationary Transformer, comprising:

[0087] Step S110, obtaining a first sample set and a second sample set, wherein the first sample set includes a plurality of unlabeled samples, the unlabeled samples are a lost circulation data sequence, and the second sample set includes a plurality of labeled samples, each labeled sample includes a lost circulation sample and a sample label indicating whether lost circulation occurs.

[0088] In this step, the well leakage data is data that will show obvious changes after the well leakage occurs (for example, the outlet flow rate suddenly decreases) or abnormal values ​​before the well leakage occurs (for example, the instantaneous drilling speed increases). In this embodiment, the well leakage data includes total pool volume, outlet flow rate (percentage), torque, drilling pressure, rotary table speed, standpipe pressure, instantaneous drilling speed, hook load, drilling time, total pump stroke (in actual application, this data corresponds to real-time pump stroke), inlet flow rate and instantaneous drilling time.

[0089] Step S120, using the first sample set to perform self-supervised pre-training on the first network model to obtain a second network model, wherein the first network model is constructed using a backbone network established using a Transformer network.

[0090] Step S130, replacing the head network of the second network model to obtain a third network model, and using the second sample set to fine-tune the third network model to obtain a lost circulation warning model.

[0091] The present invention discloses a method for constructing a well leakage warning model based on a non-stationary Transformer. A Transformer network is introduced to construct a model, and a model training method combining self-supervised pre-training and fine-tuning training is adopted to obtain a well leakage warning model. The well leakage warning model can effectively extract non-stationary information in well leakage data based on the characteristics of the Transformer network, and improve the accuracy of well leakage warning. In addition, unlabeled samples are used for self-supervised pre-training during model training, so that the model can learn common features and improve the expressiveness of the model. On the basis of self-supervised pre-training, labeled samples are used for fine-tuned supervised training, so as to improve the accuracy, reliability and generalization ability of the model for well leakage warning.

[0092] Embodiment 2: The embodiment of the present invention is a further optimization of the above embodiment, wherein the process of obtaining the first sample set and the second sample set is specifically as follows:

[0093] As attached Figure 3 As shown, the process of obtaining the first sample set includes:

[0094] Step S211, obtaining the historical data of well leakage of multiple wells through frequency sampling, wherein the historical data of well leakage includes various types of well leakage data that show abnormal changes before the well leakage occurs or show obvious changes after the well leakage occurs.

[0095] In this step, the historical data of well leakage of multiple wells are obtained by frequency sampling, that is, the well leakage data recorded and stored when each well is sampled at a set frequency during drilling operations. The historical data of well leakage thus obtained are multiple historical well leakage data sequences.

[0096] Step S212, marking the time when the well leakage occurs and the time when the well leakage ends in the well leakage history data of each well, assigning a time label to each moment of the well leakage history data according to the time when the well leakage occurs and the time when the well leakage ends, to form a corresponding well leakage data sequence, wherein the time label includes a time label when the well leakage occurs and a time label when the well leakage does not occur.

[0097] In this step, there may be one or more times of the occurrence of leakage and the end of leakage in the historical data of leakage of each well. It should be noted that the time of leakage occurrence and the end of leakage can be automatically marked according to the historical data of leakage, or can be marked by the staff. Furthermore, after marking, the historical data of leakage near the time of leakage occurrence can be visualized, and then the time of leakage occurrence can be corrected by experts, and the pump stop time can be determined according to the total pump stroke after correction. For example, Figure 4 shown.

[0098] It should also be noted that in order to achieve better training results, the amount of data can also be selected, that is, the historical data of well leakage of each well can be intercepted. The historical data of well leakage from N hours before the occurrence of well leakage to the pump stop time is regarded as an interval in which the data changes are relatively stable. Therefore, the historical data of well leakage in this time period can be intercepted as the well leakage data sequence.

[0099] In this step, a time label is assigned to each moment of the historical data of the well leakage according to the time when the well leakage occurs and the time when the well leakage ends. The time label includes a time label "1" when the well leakage occurs and a time label "0" when the well leakage does not occur as needed.

[0100] Step S213, preprocessing each lost circulation data sequence to obtain a plurality of unlabeled samples to form a first sample set.

[0101] As attached Figure 5 As shown, the process of obtaining the second sample set includes:

[0102] Step S221, obtaining the historical data of well leakage of multiple wells through frequency sampling, wherein the historical data of well leakage includes various types of well leakage data that show abnormal changes before the well leakage occurs or show obvious changes after the well leakage occurs.

[0103] Step S222, marking the time when the well leakage occurs and the time when the well leakage ends in the well leakage history data of each well, and assigning a time label to each moment of the well leakage history data according to the time when the well leakage occurs and the time when the well leakage ends, wherein the time label includes the time label when the well leakage occurs and the time label when the well leakage does not occur.

[0104] Step S223, pre-processing the labeled historical lost circulation data.

[0105] Step S224, setting the size and step length of the sliding window, and using the sliding window to intercept all the historical data of well leakage to obtain multiple window samples.

[0106] The embodiment of the present invention aims to determine the probability of the occurrence of well leakage at the last moment in the window through the data of a window sample. Therefore, in this embodiment, a sliding window is used to intercept all the well leakage history data to obtain multiple window samples;

[0107] Step S225, calculate the time label mean of all the historical data of well leakage in each window sample, retain the window samples with the time label mean that have or have not experienced well leakage, and use each window sample as a well leakage sample, and use the time label mean of each window sample as the corresponding sample label to obtain multiple labeled samples to form a second sample set.

[0108] For example, as attached Figure 6As shown in FIG. 1 , the sliding window size win_size is set to 3, the step size is set to 2, and the sliding window is used to intercept all the historical data of well leakage to obtain multiple window samples, and the time label mean of all the historical data of well leakage in each window sample is calculated, and the window samples with the time label mean of "1" and "0" are retained as the well leakage samples, and the corresponding time label mean is used as the sample label.

[0109] It should also be noted in this embodiment that, since the sample set needs to be divided into a training set, a validation set and a test set during model training, the method of this embodiment can be used to first obtain the first sample set and the second sample set, and then the sample set division can be performed during model training. Alternatively, when obtaining the sample set, the selected multiple wells can be first divided into training wells, validation wells and test wells, and then the method of this embodiment can be implemented to directly determine the training set, validation set and test set corresponding to the labeled samples and the training set, validation set and test set corresponding to the unlabeled samples. By dividing the data based on the well number dimension, data leakage can be avoided to the greatest extent and the robustness of the model can be improved.

[0110] Embodiment 3: The embodiment of the present invention is a further optimization of the above embodiment, wherein the preprocessing includes numerical type conversion, Z-score normalization, NaN value filling and time downsampling, as follows:

[0111] Numeric type conversion: Because the numeric type may be saved as a string when saving data, all data needs to be converted to numeric type;

[0112] Z-score normalization: Since each parameter has different dimensions and distributions, in order to ensure numerical stability and improve the convergence speed of the model, Z-score is used to normalize each data point so that each data point has zero mean and unit variance. This method centralizes the data distribution and has a uniform scale by subtracting the mean of each data point and dividing it by the standard deviation of the data. The specific calculation method is as follows:

[0113]

[0114] in, is the normalized time series data, is a time series of data, and are the mean and standard deviation of the data time series respectively;

[0115] NaN value filling: Due to the lack of records in the original data and to prevent the appearance of NaN values ​​in the z-score, this embodiment adds forward filling of NaN values ​​(for example, filling NaN values ​​after time t with data at time t) to prevent the subsequent model from being unable to process NaN values ​​and interrupting the warning process;

[0116] Time downsampling: Different wells come from different drilling sites, so the sampling frequency of the data is slightly different. Considering the judgment frequency of well leakage warning, this embodiment downsamples the historical data of well leakage at an interval of 20 seconds. The specific implementation can be adjusted according to the actual data situation. This embodiment does not limit it. The sampling steps are as follows: (1) Determine the sampling interval according to the needs and application scenarios. The present invention takes 20 seconds; (2) Divide the time series data into windows with fixed time intervals; (3) Calculate the mean of the data in each window; (4) Use the mean to represent the single data of the last moment of each window, and combine them into new equally spaced time series data.

[0117] Embodiment 4: As attached Figure 7 As shown, the embodiment of the present invention is a further optimization of the above embodiment, wherein the process of performing self-supervisory pre-training on the first network model using the first sample set includes:

[0118] Step S310: divide the first sample set into a training set, a validation set and a test set.

[0119] Step S320, training the first network model using the training set, introducing a loss function during training, and ending the training when the value of the loss function is stable, to obtain a second network model, wherein the first network model is constructed using a backbone network established using a Transformer network, including extracting a destabilization factor from the normalized input data using two multi-layer perceptrons and , the normalized input data is mapped from low dimension to high dimension through the embedding layer, based on the stationary factor and The high-dimensional input data is encoded through a non-stationary Transformer encoder to obtain high-dimensional semantic features, which are then mapped back to semantic features of the original dimension through a multi-layer perceptron.

[0120] The non-stationary Transformer network architecture in this embodiment is an improved version of the Transformer for solving the problem of over-stationarity of time series. Through the sequence stabilization and de-stationarization attention mechanism, the stabilization process is performed with the goal of improving the prediction performance. Since some well leakage data have non-stationarity problems, the present invention uses the Transformer network to better extract the non-stationary information of the data, thereby improving the performance of downstream well leakage classification tasks.

[0121] The first network model constructed by the backbone network established by the Transformer network in this embodiment is shown in the attached figure. Figure 8 As shown, the whole model mainly consists of three parts:

[0122] (1) Data normalization and extraction of destabilization factors and :This part is mainly to normalize the input, so that the windowing method can make the data in each adjacent window have the same mean and variance, so as to eliminate the difference in scale between sequences and increase the distribution stability of input data in time series; at the same time, two multi-layer perceptrons are used to extract the destabilization factor from the non-stationary time series and ;

[0123] (2) Mapping and encoding layer: This part first maps the normalized input from low dimensions to high dimensions through the embedding layer, and then encodes it through the non-stationary Transformer backbone network structure (i.e., the non-stationary Transformer encoder, which is constructed by the Transformer network) to obtain a high-dimensional semantic representation;

[0124] (3) Multilayer Perceptron: This module remaps the mapped high-dimensional semantic features back to the dimensions of the original input data through a multilayer perceptron.

[0125] Therefore, choosing the Transformer network result as the backbone network structure can effectively extract the changing characteristics of rich well leakage data sequences and provide better representation for subsequent pre-training and fine-tuning.

[0126] It should also be noted that self-supervised pre-training of the well leakage data sequence means encoding the original well leakage data sequence, projecting it to a higher-dimensional representation, and then restoring it to the same dimension as the original well leakage data sequence through a multi-layer perceptron. In this process, the mean square error (MSE) of the original well leakage data sequence and the sequence after encoding and restoration can be calculated. The smaller the mean square error (MSE), the better the restoration effect. The specific calculation formula is as follows:

[0127]

[0128] in, is the dimension of the sequence, For the The original lost circulation data sequence of dimensions, is the first A sequence of dimensions.

[0129] Step S330, using the verification set and the test set to respectively verify and test the trained second network model, optimize the model parameters of the second network model, and output the second network model that meets the test evaluation requirements.

[0130] The purpose of using the first sample set to perform self-supervised pre-training on the first network model in the above embodiment is to allow the non-stationary Transformer backbone network structure to fully extract the intrinsic high-order semantics of the well leakage data sequence through non-linear encoding and restoration to the original sequence.

[0131] Embodiment 5: The embodiment of the present invention is a further optimization of the above embodiment, wherein the process of fine-tuning the third network model using the second sample set is the same as the process of self-supervised pre-training the first network model using the first sample set, and will not be repeated.

[0132] It should be noted that the third network model is obtained by replacing the head network of the second network model, and initializing the unstable Transformer backbone network with the weights obtained by pre-training in Example 4. In this embodiment, the head network "multilayer perceptron" of the second network model can be replaced with a "single linear layer", and the corresponding specific structure is shown in the attached figure. Fig. 9 shown.

[0133] Furthermore, the traditional binary classification loss function can effectively guide the iterative optimization of the model, thereby achieving the classification of normal samples and lost circulation samples. The traditional binary classification loss function is as follows:

[0134]

[0135] in, is the true result of the labeled sample in the second sample set, is the prediction result of the model.

[0136] However, the above traditional binary classification loss function can only distinguish the well leakage samples, but cannot realize the early warning of well leakage. Therefore, the present invention modifies the traditional binary classification loss function to meet the goal of early warning of well leakage.

[0137] Specifically, the well leakage warning model needs to provide an early warning before the well leakage occurs. Therefore, if the well leakage warning model predicts a well leakage of 1 (well leakage) within a set time period before the well leakage occurs, although the prediction is wrong (the actual value is 0, not well leakage), it meets the goal of well leakage warning and achieves early warning.

[0138] Therefore, this embodiment determines the time period T according to the actual situation (for example, it can be set to 30 minutes) and optimizes the traditional binary classification loss function as shown below:

[0139]

[0140] in, is the true result of the labeled sample in the second sample set, is the prediction result of the model.

[0141] In the process of fine-tuning the third network model using the second sample set, the optimized binary classification loss function is used to provide early warning of well leakage.

[0142] Embodiment 6: As attached Fig.10 As shown, the embodiment of the present invention discloses a well leakage warning method based on a non-stationary Transformer, comprising:

[0143] Step S410, obtaining a real-time well leakage data sequence of a single well in a certain period of time, the real-time well leakage data sequence including total pool volume, outlet flow (percentage), torque, drilling pressure, rotary table speed, standpipe pressure, instantaneous drilling speed, hook load, drilling time, real-time pump stroke, inlet flow and instantaneous drilling time;

[0144] Step S420, preprocessing the real-time lost circulation data sequence, and intercepting the real-time lost circulation data sequence using a sliding window to obtain a plurality of window samples to be predicted;

[0145] Step S430, inputting a plurality of samples of the window to be predicted into a well leakage warning model to obtain a predicted probability of well leakage, wherein the well leakage warning model is constructed by the methods of Embodiments 1 to 5;

[0146] Step S440, comparing the predicted probability of lost circulation with a set threshold, and issuing a lost circulation warning based on the comparison result.

[0147] Example 7: The well leakage warning model and well leakage warning method disclosed in the present invention are verified through the following specific examples, which are described in detail as follows:

[0148] 12 training wells, 1 verification well and 3 test wells were determined, and the historical data of well leakage of each well was obtained. The first sample set and the second sample set were obtained by the method of the above-mentioned embodiment 2. When obtaining the second sample set, the sliding window size was 180 and the step size was 10. The number of sample sets obtained is shown in Table 1.

[0149] Table 1 Number of sample sets

[0150] .

[0151] The first network model is self-supervised pre-trained using the first sample set to obtain the second network model, where the hyperparameters of the non-stationary Transformer backbone network are shown in Table 2.

[0152] Table 2 Non-stationary Transformer backbone network hyperparameters

[0153] .

[0154] In the specific implementation process, the input dimension of the backbone network model and the number of layers of some networks can be adjusted in a targeted manner according to the number of selected historical well leakage data and the number of constructed sample sets to adapt to the selected data. The present invention does not limit the specific parameter settings.

[0155] The training parameters related to self-supervised pre-training are shown in Table 3.

[0156] Table 3 Pre-training parameter settings

[0157] .

[0158] The results of the second network model reconstructing the test set are shown in the attached figure. Fig.11 shown.

[0159] The head network multilayer perceptron of the second network model is replaced with a linear layer, and the weights of the second network model are used to initialize the backbone network to obtain the third network model, and the third network model is fine-tuned using the second sample set, where the training parameters in the fine-tuning process are shown in Table 4.

[0160] Table 4. Training parameter settings for fine-tuning tasks

[0161] .

[0162] The well leakage warning model obtained through fine-tuning training is used to predict the test set one by one, and the well leakage probability corresponding to each test sample is returned. Preferably, the judgment threshold set in this embodiment is 0.5. If it is greater than 0.5, it is a well leakage sample, otherwise it is a non-well leakage sample. Fig.12 The figure shows the result of a well leakage in the test well. The value of the green point is the proportion of all moments in the sample constructed by the sliding window that are after the well leakage occurs. If the value is greater than 0, the sample is in the well leakage occurrence stage. Fig.12 The time of the well leakage in the case shown is 21:44. The prediction result of the well leakage warning model trained by this embodiment for the sample is shown in black line. Fig.12 The window sample corresponding to the point marked by the green arrow is judged by the model to have a probability of leakage greater than 0.5. At this time, the time corresponding to the last moment of the sample is 20:56, so the leakage warning time is 20:56, which is 48 minutes earlier than the leakage time. Early warning is achieved, which can effectively help on-site staff prepare plugging agents and other leakage prevention and plugging measures to reduce economic losses and risk accidents.

[0163] Embodiment 8: As attached Fig.13 As shown, the embodiment of the present invention discloses a device for constructing a well leakage warning model based on a non-stationary Transformer, comprising:

[0164] A sample set acquisition unit acquires a first sample set and a second sample set, wherein the first sample set includes a plurality of unlabeled samples, the unlabeled samples are a lost circulation data sequence, and the second sample set includes a plurality of labeled samples, each labeled sample includes a lost circulation sample and a sample label indicating whether lost circulation occurs;

[0165] A first training unit performs self-supervised pre-training on a first network model using a first sample set to obtain a second network model, wherein the first network model is constructed using a backbone network established using a Transformer network;

[0166] The second training unit replaces the head network of the second network model to obtain a third network model, and uses the second sample set to fine-tune the third network model to obtain a well leakage warning model.

[0167] Embodiment 9: As attached Fig.14 As shown, the embodiment of the present invention discloses a well leakage warning device based on a non-stationary Transformer, comprising:

[0168] The unit for acquiring data to be predicted acquires a real-time well leakage data sequence of a single well in a certain period of time;

[0169] The data processing unit to be predicted pre-processes the real-time lost circulation data sequence and intercepts the real-time lost circulation data sequence using a sliding window to obtain a plurality of window samples to be predicted;

[0170] A prediction unit, inputting a plurality of samples of the window to be predicted into a well leakage warning model to obtain a predicted probability of well leakage, wherein the well leakage warning model is constructed by the method described in any one of claims 1 to 6;

[0171] The early warning unit compares the predicted probability of well leakage with the set threshold value and issues a well leakage early warning based on the comparison result.

[0172] Embodiment 10: The embodiment of the present invention discloses an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement a well leakage warning method based on a non-stationary Transformer.

[0173] The processor may be a central processing unit (CPU), a general purpose processor, a digital signal processor (DSP), an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. It may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The memory may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory, a mobile hard disk, a magnetic disk or an optical disk.

[0174] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The solutions in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and interpreted scripting language JavaScript, etc.

[0175] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0176] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0177] The above content is only a specific implementation mode of the present invention, which has strong adaptability and implementation effect, but the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope covered by the present invention.

Claims

1. A method for constructing a well leakage warning model based on a non-stationary Transformer, characterized in that: include: Acquire a first sample set and a second sample set, wherein the first sample set includes a plurality of unlabeled samples, the unlabeled samples are a lost circulation data sequence, and the second sample set includes a plurality of labeled samples, each labeled sample includes a lost circulation sample and a sample label indicating whether lost circulation occurs; Using the first sample set to perform self-supervisory pre-training on the first network model to obtain a second network model, wherein the first network model is constructed using a backbone network established using a Transformer network; The head network of the second network model is replaced to obtain a third network model, and the third network model is fine-tuned and trained using the second sample set to obtain a well leakage warning model; The process of performing self-supervisory pre-training on the first network model using the first sample set is the same as the process of performing fine-tuning training on the third network model using the second sample set, wherein the process of performing self-supervisory pre-training on the first network model using the first sample set includes: Divide the first sample set into a training set, a validation set and a test set; The first network model is trained using the training set, and a loss function is introduced during the training. When the value of the loss function is stable, the training is terminated to obtain the second network model. The first network model is constructed using the backbone network established using the Transformer network, including extracting the destabilization factor from the normalized input data through two multi-layer perceptrons. and , the normalized input data is mapped from low dimension to high dimension through the embedding layer, based on the stationary factor and The high-dimensional input data is encoded through a non-stationary Transformer encoder to obtain high-dimensional semantic features, and the high-dimensional semantic features are mapped back to the semantic features of the original dimension through a multi-layer perceptron; The trained second network model is verified and tested using the verification set and the test set respectively, the model parameters of the second network model are optimized, and the second network model that meets the test evaluation requirements is output.

2. The method for constructing a well leakage warning model based on a non-stationary Transformer according to claim 1, characterized in that: The process of obtaining the first sample set includes: Acquire the historical data of well leakage of multiple wells through frequency sampling, wherein the historical data of well leakage includes various types of well leakage data that show abnormal changes before the well leakage occurs or show obvious changes after the well leakage occurs; The time when the well leakage occurs and the time when the well leakage ends are marked in the well leakage history data of each well, and a time label is assigned to each time of the well leakage history data according to the time when the well leakage occurs and the time when the well leakage ends, so as to form a corresponding well leakage data sequence, wherein the time label includes a time label when the well leakage occurs and a time label when the well leakage does not occur; Each lost circulation data sequence is preprocessed to obtain a plurality of unlabeled samples to form a first sample set.

3. The method for constructing a well leakage warning model based on a non-stationary Transformer according to claim 1, characterized in that: The process of obtaining the second sample set includes: Acquire the historical data of well leakage of multiple wells through frequency sampling, wherein the historical data of well leakage includes various types of well leakage data that show abnormal changes before the well leakage occurs or show obvious changes after the well leakage occurs; Marking the time when the well leakage occurs and the time when the well leakage ends in the well leakage history data of each well, and assigning a time label to each time of the well leakage history data according to the time when the well leakage occurs and the time when the well leakage ends, wherein the time label includes a time label when the well leakage occurs and a time label when the well leakage does not occur; Preprocess the labeled historical data of well leakage; Set the size and step length of the sliding window, and use the sliding window to intercept all historical data of well leakage to obtain multiple window samples; Calculate the time label mean of all historical data of well leakage in each window sample, retain the window sample with the time label mean of well leakage or no well leakage, and use each window sample as a well leakage sample, and use the time label mean of each window sample as the corresponding sample label to obtain multiple labeled samples to form a second sample set.

4. The method for constructing a well leakage warning model based on a non-stationary Transformer according to claim 2 or 3, characterized in that: The preprocessing includes numerical type conversion, Z-score normalization, NaN value filling and time downsampling.

5. The method for constructing a well leakage warning model based on a non-stationary Transformer according to claim 1, characterized in that: When fine-tuning the third network model using the second sample set, the loss function used is as follows: in, is the true result of the labeled sample in the second sample set, is the prediction result of the model; Or / and, the head network of the second network model is replaced with a linear layer to obtain a third network model.

6. A well leakage warning method based on non-stationary Transformer, characterized in that: include: Obtain real-time lost circulation data sequence of a single well in a certain period of time; Preprocess the real-time lost circulation data sequence, and intercept the real-time lost circulation data sequence using a sliding window to obtain multiple window samples to be predicted; Inputting a plurality of samples of the window to be predicted into a well leakage warning model to obtain a predicted probability of well leakage, wherein the well leakage warning model is constructed by the method described in any one of claims 1 to 5; The predicted probability of well leakage is compared with the set threshold, and a well leakage warning is issued based on the comparison result.

7. A device for constructing a well leakage warning model based on a non-stationary Transformer using the method according to any one of claims 1 to 5, characterized in that: include: A sample set acquisition unit acquires a first sample set and a second sample set, wherein the first sample set includes a plurality of unlabeled samples, the unlabeled samples are a lost circulation data sequence, and the second sample set includes a plurality of labeled samples, each labeled sample includes a lost circulation sample and a sample label indicating whether lost circulation occurs; A first training unit performs self-supervised pre-training on a first network model using a first sample set to obtain a second network model, wherein the first network model is constructed using a backbone network established using a Transformer network; The second training unit replaces the head network of the second network model to obtain a third network model, and uses the second sample set to fine-tune the third network model to obtain a well leakage warning model; The process of performing self-supervisory pre-training on the first network model using the first sample set is the same as the process of performing fine-tuning training on the third network model using the second sample set, wherein the process of performing self-supervisory pre-training on the first network model using the first sample set includes: Divide the first sample set into a training set, a validation set and a test set; The first network model is trained using the training set, and a loss function is introduced during the training. When the value of the loss function is stable, the training is terminated to obtain the second network model. The first network model is constructed using the backbone network established using the Transformer network, including extracting the destabilization factor from the normalized input data through two multi-layer perceptrons. and , the normalized input data is mapped from low dimension to high dimension through the embedding layer, based on the stationary factor and The high-dimensional input data is encoded through a non-stationary Transformer encoder to obtain high-dimensional semantic features, and the high-dimensional semantic features are mapped back to the semantic features of the original dimension through a multi-layer perceptron; The trained second network model is verified and tested using the verification set and the test set respectively, the model parameters of the second network model are optimized, and the second network model that meets the test evaluation requirements is output.

8. A non-stationary Transformer-based well leakage warning device using the method of claim 6, characterized in that: include: The unit for acquiring data to be predicted acquires a real-time well leakage data sequence of a single well in a certain period of time; The data processing unit to be predicted pre-processes the real-time lost circulation data sequence and intercepts the real-time lost circulation data sequence using a sliding window to obtain a plurality of window samples to be predicted; A prediction unit, inputting a plurality of samples of the window to be predicted into a well leakage warning model to obtain a predicted probability of well leakage, wherein the well leakage warning model is constructed by the method described in any one of claims 1 to 5; The early warning unit compares the predicted probability of well leakage with the set threshold value and issues a well leakage early warning based on the comparison result.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the steps in the method according to claim 6.

Images