Unconventional oil and gas yield prediction method based on space-time Transform

Abstract

The invention discloses an unconventional oil and gas yield prediction method based on space-time Transform, which is characterized in that geological parameters, fracturing construction parameters and production dynamic data are fused, and time sequence dependency and spatial heterogeneity of modeling parameters are synchronized by utilizing a space-time attention mechanism. Firstly, normalization and space-time alignment are carried out on multi-source data, and a joint input matrix is constructed; then, a dynamic change trend of fracturing construction parameters is extracted through a time attention module, a space contribution weight of geological parameters is quantified through a space attention module, and a space-time interaction matrix is introduced to capture a multi-dimensional coupling relation; and finally, outputting a yield prediction result through feature fusion and MLP. According to the method, the dependence of a traditional model on a single data source is broken through, the precision and interpretability of unconventional oil and gas yield prediction are remarkably improved, and reliable technical support is provided for oil and gas field development strategy optimization.

Description

Technical Field

[0001] The present invention belongs to the cross - technical field of oil and gas field development and artificial intelligence, and specifically relates to a method for predicting unconventional oil and gas production based on spatio - temporal Transformer. Background Art

[0002] Unconventional oil and gas resources, especially shale gas, have become an important part of the global energy supply. With the gradual depletion of traditional oil and gas resources, the development of unconventional oil and gas resources has become increasingly crucial. The production of shale gas wells is affected by a combination of various factors, including geological parameters such as porosity and permeability, and fracturing construction parameters such as pump pressure and sand ratio. These factors have complex coupling relationships in time and space. Traditional prediction models are difficult to effectively capture this complexity. Empirical formulas and numerical simulation methods rely on simplified assumptions and are difficult to characterize the spatial heterogeneity of geological parameters and the dynamic time - series correlation of fracturing construction parameters. Traditional machine - learning models such as random forests and support vector machines are limited by their shallow feature - extraction capabilities and cannot effectively capture the spatio - temporal non - linear coupling characteristics of multi - source data. Single - dimensional time - series models only focus on time - series dependencies and ignore the key impact of the spatial distribution characteristics of geological parameters on production. From the above analysis, it can be seen that traditional prediction models have insufficient prediction accuracy and are difficult to support the refined decision - making of oil and gas field development. Therefore, developing a high - precision prediction method that can integrate multi - source data and simultaneously model spatio - temporal coupling relationships is of great significance for optimizing oil and gas field development strategies and reducing investment risks. Summary of the Invention

[0003] In order to overcome the deficiencies of the prior art, the present invention provides a method for predicting unconventional oil and gas production and uncertainty analysis based on spatio - temporal Transformer. This method applies the spatio - temporal attention mechanism in the Transformer architecture to the prediction of unconventional oil and gas production, effectively separating the time - series and spatial dependencies of geological static data and production dynamic data, significantly improving the model's ability to capture complex coupling relationships, breaking through the limitations of traditional models that rely on a single data source, and combining geological parameters, fracturing construction parameters, and production dynamic data to construct a comprehensive high - precision prediction model.

[0004] To achieve the above object, the technical solution of the present invention mainly includes the following steps:

[0005] A. Data pre - processing and construction of spatio - temporal joint input:

[0006] (1) Collect geological static parameters, including: well location coordinates, porosity, permeability, gas saturation, and fracturing construction dynamic parameters, including: pump pressure, sand ratio, liquid injection volume, construction duration, and historical production data.

[0007] (2) Normalize the collected data to ensure that the data are on the same scale to facilitate subsequent model processing.

[0008] (3) Divide the normalized data into training set X train and the test set X test , ensuring that the data is randomly distributed and non-overlapping, in preparation for model training and validation.

[0009] B. Spatiotemporal Transformer model construction:

[0010] (1) Map the input sequence data to the embedding space d of the specified dimension through the Embedding linear projection layer model Add position code Then get the initial embedding vector

[0011] (2) Input the initial embedding vector Z0 into a network model consisting of L layers of self-attention modules. Each layer contains: query (Q), key (K), value (V), and their mappings are:

[0012] Q i =Z l-1 W i Q ,K i =Z l-1 W i K ,V i =Z l-1 W i V

[0013] Where W i V W i Q ,W i K , h is the number of attention heads.

[0014] (3) Use the self-attention mechanism in the Transformer architecture to perform attention calculation and splicing:

[0015]

[0016] Get the multi-head attention output M l Then further perform residual connection and layer normalization:

[0017] M l =LayerNorm(M l +Z l-1 )

[0018] Then a time feature can be obtained to focus on the changing trend of production dynamic data over time.

[0019] (4) Transpose the time feature S T to and map it to the spatial embedding space through a linear projection layer. Then, adopt a multi-head self-attention mechanism similar to the time attention to calculate the spatial contribution weights of geological parameters and output the spatial feature reflecting the distribution of geological parameters at different spatial positions.

[0020] (5) Introduce a learnable interaction matrix and calculate the spatio-temporal joint feature S TS .

[0021] (6) Perform a residual connection on S TS with the original input Z0 to enhance the feature stability.

[0022] (7) Perform global average pooling on the spatio-temporal joint feature S TS in the channel dimension to obtain the feature vector

[0023] (8) Input the feature vector F into an MLP containing two linear layers to obtain the comprehensive feature vector R.

[0024] (9) Generate the production grade probability distribution through the softmax activation function for the comprehensive feature vector R (K is the number of production grades).

[0025] C. Model Training and Validation:

[0026] (1) During the model training process, use the mean square error (MSE) loss function to calculate the difference between the predicted probability and the true label.

[0027] (2) Adopt the Adam optimizer, set the initial learning rate, and adopt the cosine annealing strategy. At the same time, introduce an early stopping mechanism. If the validation set loss does not decrease for 10 consecutive epochs, terminate the training.

[0028] (3) Use the test dataset to perform validation testing on the model, calculate the root mean square error (RMSE) of the model to measure the deviation between the predicted value and the true value, and the coefficient of determination (R 2 ) to evaluate the goodness of fit of the model.

[0029] The beneficial effects of the present invention are as follows: The spatio-temporal attention mechanism synchronously models the spatial heterogeneity of geological parameters and the temporal dynamics of fracturing construction parameters, significantly improving the characterization ability of complex coupling relationships. Compared with the traditional LSTM model, accurate production prediction is realized. At the same time, by uniformly encoding geological static data, fracturing dynamic data, and production data into a spatio-temporal joint tensor, the limitation of relying on a single data source in traditional methods is broken through, end-to-end multi-dimensional information fusion is achieved, and the data utilization rate is effectively improved. In addition, the visualization of spatio-temporal attention weights can quantify the contribution degrees of key parameters (such as permeability and sand ratio) to production, providing a direct basis for optimizing fracturing plans, guiding the scientificity and transparency of engineering decisions, providing reliable technical support for the efficient development of unconventional oil and gas resources, and having significant economic and social values. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is the model structure diagram of the present invention DETAILED DESCRIPTION OF THE INVENTION

[0031] The following further describes the present invention in detail in conjunction with Figure 1 :

[0032] A. Data preprocessing and construction of spatio-temporal joint input:

[0033] (1) Collect geological static data, including: well location coordinates, porosity (%), permeability (mD), gas saturation (%), to form a matrix where N is the number of well location spatial positions, and C is the dimension of geological parameters; collect fracturing construction dynamic data, including: pump pressure (MPa), sand ratio (%), injection volume (m 3 ), construction duration (h), to form a sequence where T is the time step, that is, the fracturing construction cycle, and D is the dimension of dynamic parameters; record the daily oil and gas production (m 3 / d) at the corresponding time step to form a historical production data sequence

[0034] (2) Duplicate the geological static data G T times along the time dimension and expand it to to ensure that each time step contains complete spatial geological information.

[0035] (3) Concatenate G ′ and the fracturing construction data F along the channel dimension to construct a spatio-temporal joint input tensor to realize the spatio-temporal fusion of geological-engineering data.

[0036] (4) Perform min-max normalization on the data x of each channel respectively to eliminate the dimension difference. The formula is as follows:

[0037]

[0038] Among them, i, j, and k represent the time step, well location, and channel index respectively. After normalization, the data range is limited to [0, 1]. For geological parameters such as porosity, permeability, and gas saturation, the original physical meaning is retained to avoid information loss after normalization and ensure the stability of model training.

[0039] (5) Randomly select a preset proportion of well location data and its corresponding time series to form X train , and the remaining well location data forms the test set X test . Ensure that the data is randomly distributed and non-overlapping, and retain the continuity of the time series. At the same time, a small proportion also needs to be divided from the training set as the validation set for the early stopping mechanism.

[0040] B. Construction and training of the spatio-temporal Transformer model:

[0041] (1) Map the input sequence data X to the embedding space d of the specified dimension through the trainable matrix E model to obtain Z embed , and use the sine-cosine function to generate the time position encoding The specific formula is as follows:

[0042]

[0043] t is the time node, and then the position encoding E pos is superimposed on Z embed to obtain the initial embedding vector Z0 = Z embed + E pos .

[0044] (2) Input the initial embedding vector into a network model composed of L layers of self-attention modules, with the number of attention heads being h and the single-head dimension d k = d model / h. First, for each attention head i, generate the query (Q i ), key (K i ), and value (V i ):

[0045]

[0046] Then calculate the scaled dot-product attention through the following formula to obtain the attention scores:

[0047]

[0048] Concatenate the results of each head and then perform projection:

[0049]

[0050] Repeat the above steps L times to output the time features Pay attention to the changing trend of production dynamic data over time.

[0051] (3) Transpose the time feature S T to Rearrange the dimensions to focus on the spatial relationship. Adopt the same multi-head mechanism as the time attention, and input The operation is performed on the well location dimension (N), and the spatial features are output

[0052] (4) Introduce learnable weights Perform the interaction matrix calculation to obtain the spatio-temporal joint feature S TS , and then S TS is residually connected to the original input Z0 to enhance the feature stability. The specific formula is as follows:

[0053] S TS = MatMul(S T , W TS , S S )

[0054] S final = LayerNorm(S TS + Z0)

[0055] (5) Average along the well location dimension (N) for to obtain the feature vector F. The calculation formula is as follows:

[0056]

[0057] (6) Input the feature vector F into the MLP containing two linear layers. In the first layer of the multi-layer perceptron (MLP), the dimension is expanded and the ReLU function is applied for activation: In the second layer, the dimension is compressed, and finally the prediction score is generated: where W1 represents the learnable projection matrix of the inner-layer linear transformation, W2 represents the learnable projection matrix of the outer-layer linear transformation, and b1 and b2 are the respective bias terms.

[0058] (7) The task metric is a classification task. When grading the production, use the softmax activation function to generate the probability distribution:

[0059] p = softmax(R)

[0060] The task metric is a regression task. When directly predicting the production, use a linear output:

[0061]

[0062] C. Model Training and Validation:

[0063] (1) During the model training process, the mean squared error (MSE) loss function is used in the regression task to calculate the difference between the predicted probability and the true label. The specific formula is as follows:

[0064]

[0065] In the classification task, the cross-entropy loss function is used to calculate the difference between the predicted probability and the true label. The specific formula is as follows:

[0066]

[0067] Where BS is the batch size, y i is the true output, is the predicted value, T is the time step, p i is the predicted probability, and K is the number of output grades.

[0068] (2) Use the Adam optimizer, set the initial learning rate η = 10 -4 , β1 = 0.9, β2 = 0.999, the weight decay λ = 10 -5 , and adopt the cosine annealing strategy with a period of 50 epochs. At the same time, introduce the early stopping mechanism to monitor the validation set loss. If it does not decrease for 10 consecutive epochs, save the current optimal model and terminate the training.

[0069] (3) Use the test data set to verify and test the model, and calculate the evaluation metrics of the model, including the root mean square error (RMSE) that measures the deviation between the predicted value and the true value and the coefficient of determination (R 2 ) that evaluates the goodness of fit of the model. The specific calculation formulas are as follows:

[0070]

[0071] Where N test represents the number of samples in the test set.

[0072] (4) Extract the time attention matrix and the spatial attention matrix to draw a heat map to analyze the contribution of key parameters.

[0073] The above embodiments are only used to illustrate the technical solutions of the present invention. Any person skilled in the art may use the technical solutions described above to modify or change them into equivalent examples with equivalent changes. Any simple modification, change, or modification made to the above embodiments based on the technical solutions of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

Claims

1. A method for predicting unconventional oil and gas production based on spatio-temporal Transformer, characterized in that, It includes the following steps: A. Multi-source data fusion and preprocessing: Collect the geological static data matrix, the dynamic sequence of fracturing construction, and the historical production data sequence. Expand the geological static data along the time dimension and splice it with the dynamic data of fracturing construction in the channel dimension to construct a spatio-temporal joint input tensor. Perform channel-level min-max normalization on the spatio-temporal joint input tensor, map the values to the interval [0,1], and divide it into a training set, a validation set, and a test set according to a preset ratio; B. Spatio-temporal attention feature extraction: Map the spatio-temporal joint input tensor to the embedding space through a linear projection layer, stack the time position encoding and then input it into the spatio-temporal Transformer network composed of multiple layers of multi-head self-attention modules. Adopt the multi-head self-attention mechanism in the time dimension to extract the temporal dependence features of fracturing construction parameters, focus on the spatial heterogeneity of geological parameters through transpose operation in the spatial dimension to generate spatial features, introduce a learnable spatio-temporal interaction matrix, calculate the spatio-temporal joint features, and output the final features through residual connection and layer normalization; C. Feature fusion and production prediction: Perform global average pooling on the spatio-temporal joint features in the channel dimension to obtain a feature vector. Input the feature vector into a multi-layer perceptron (MLP), generate a prediction score through linear transformation and ReLU function activation, and use the softmax function to output the probability distribution of production levels; D. Model training and validation: End-to-end training is carried out using the mean squared error (MSE) loss function and the Adam optimizer. The learning rate is dynamically adjusted by combining the cosine annealing strategy. The validation set loss is monitored through the early stopping mechanism. If it does not decrease for a continuous preset number of epochs, the training is terminated, and the optimal model parameters are saved. The root mean square error (RMSE) and the coefficient of determination (R 2 ) are calculated on the test set to measure the deviation between the predicted value and the true value and to evaluate the goodness of fit of the model.

Images