A full waveform inversion method based on gate recurrent unit and attention mechanism
By using a neural network model based on gated recurrent units and attention mechanisms, the problems of inaccurate two-dimensional imaging and weak generalization ability in full waveform inversion are solved, and efficient and accurate geological property interpretation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUADIAN COAL IND GRP DIGITAL INTELLIGENCE TECH CO LTD
- Filing Date
- 2023-02-01
- Publication Date
- 2026-05-19
AI Technical Summary
Existing deep learning-based full waveform inversion methods suffer from problems such as inaccurate two-dimensional imaging, weak generalization ability, poor robustness, and low computational efficiency in underground medium property inversion.
A full waveform inversion method based on gated recurrent units and attention mechanism is adopted. By constructing a neural network model based on gated recurrent units and attention mechanism, the nonlinear fitting ability is improved and important features in the input information are focused on, so as to achieve efficient nonlinear mapping of full waveform inversion.
It significantly improves the accuracy and generalization ability of full waveform inversion, enabling faster and clearer prediction of geological velocity models and accurate interpretation of geological properties.
Smart Images
Figure CN116090352B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and geophysics, specifically to a full waveform inversion method based on gated loop units and attention mechanisms. Background Technology
[0002] Full Waveform Inversion (FWI) plays a crucial role in subsurface characterization in Earth sciences by determining subsurface properties from seismic data and imaging them with high precision and resolution. Typically, FWI is mathematically formulated as a nonlinear inverse problem, and its numerical implementation can occur in either the time or frequency domain. However, FWI computation methods are not only computationally intensive, but the nonlinearity of the inversion problem also leads to convergence to local minima, reducing the applicability and robustness of the algorithms. To alleviate these problems, many traditional methods have been proposed and extended, such as regularization-based techniques, prior information-based methods, multi-scale inversion methods, and preprocessing methods.
[0003] In recent years, with the significant improvement of computing power and the rise of deep neural networks, many deep learning methods have been applied to problems such as geological exploration data processing and inversion. Deep learning-based full waveform inversion methods are mainly divided into three categories: based on deep neural networks, based on convolutional neural networks, and based on recurrent neural networks: 1) Full waveform inversion method based on deep neural networks: Mauricio et al. proposed an accurate deep neural network model that is directly trained on the original data. The prediction model trained by this model maps the relationship between the data space and the velocity model. The final output result has only a small error with the experimental data obtained from the real surface, which shows the research value of deep neural networks in full waveform inversion methods (Mauricio A, Joseph J, Amir A, et al. Deep-learning tomography[J].LeadingEdge.2018,37(1):58-66.); 2) Full waveform inversion method based on convolutional neural networks: Luan et al. used the finite difference method to simulate the actual observation data to construct a training dataset, and used this as input to train a fully convolutional neural network to predict the initial velocity model. The output of this method already has the general details of the real observation data. After accurate inversion by the full waveform inversion method, a high-precision and high-resolution image of the subsurface medium properties can be obtained (Luan C, Peterson NS, Erick GS N. Estimating Initial Velocity Models for the FWI Using Deep Learning[C]. International Congress of the Brazilian Geophysical Society & Expogoef. 2019); 3) Full waveform inversion method based on recurrent neural network: Sun et al. proposed a recurrent neural network that can model one-dimensional or multi-dimensional scalar acoustic earthquake forward propagation and proved that training the network and updating its weights using observation data is equivalent to solving the inverse problem of geographic exploration, namely gradient-based full waveform inversion of earthquake (Sun J, Niu Z, Kristopher AI, et al. A theory-guided deep-learning formulation and optimization of seismic waveform inversion[J]. Geophysics, 2020, 85(2):R87-R99.).Wang et al. demonstrated multi-parameter full waveform inversion in isotropic elastic wave media using recurrent neural networks and proved the equivalence of the full-batch automatic differentiation method and the traditional adjacency state method in inversion in elastic isotropic media (Wang W, McMechan GA, Ma J. Elastic isotropic and anisotropic full waveform inversions using automatic differentiation for gradient calculations in a framework of recurrent neural networks[J]. Geophysics, 2021, 86(6):R795-R810.). However, most existing deep learning-based full waveform inversion methods can only achieve two-dimensional imaging for subsurface medium property inversion and have inaccurate structural characterization. Furthermore, they suffer from weak generalization ability and poor robustness of deep learning models used for full waveform inversion, as well as being easily constrained by the initial model and having low computational efficiency. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention aims to provide a full waveform inversion method based on gated recurrent units and attention mechanisms. This method not only enhances the nonlinear fitting capability of traditional full waveform inversion methods based on recurrent neural networks, but also increases the attention paid to important features in the input information, thereby accelerating the entire process of full waveform inversion.
[0005] To achieve the above objectives, the present invention adopts the following technical solution:
[0006] A full waveform inversion method based on gated recurrent units and attention mechanisms includes the following steps:
[0007] Step 1: Obtain actual working face seismic observation data, and preprocess the obtained actual working face seismic observation data. Using the corresponding geological velocity model as the label set, construct the actual working face seismic observation data into training set and test set through random sampling.
[0008] Step 2: Construct a neural network model for the forward modeling process in the full waveform inversion mapping based on gated recurrent units and attention mechanisms. Input the training set data obtained in Step 1 and its corresponding geological velocity model label set into the neural network model as input data, and set the hyperparameters required for training the neural network model, including the maximum number of iterations, learning rate, and network depth; initialize the parameters of the neural network model and train it; at the preset iteration detection node, test the performance of the neural network model using the test set data, and save the neural network model parameters at this time.
[0009] Step 3: Select the optimal neural network model output results saved by each iterative detection node according to the preset evaluation index, and load the corresponding model parameters into the neural network model to obtain the optimal neural network model. Use the unknown seismic observation data as the input of the optimal neural network model, fix the parameters and perform inversion calculation. Based on the output results, obtain the predicted velocity model, which is the inversion mapping result of the subsurface medium physical property imaging.
[0010] Furthermore, in step 1, the specific process of acquiring actual working face seismic observation data and performing data preprocessing on the obtained actual working face seismic observation data is as follows:
[0011] Based on the actual conditions of the working face, a layout scheme for the location of seismic data detectors is designed to obtain actual seismic observation data of the working face; the obtained actual seismic observation data of the working face is preprocessed, including eliminating bad channels in the signal, removing random impulse noise, and suppressing power frequency interference.
[0012] Furthermore, in step 2, the structure of the neural network model includes two cascaded Att-GRU blocks and a regression layer for feature decoding;
[0013] The forward propagation process of the Att-GRU block is as follows:
[0014] The input data is first encoded by an attention encoding layer, which increases the attention given to important features in the current information. This can be formalized as follows:
[0015]
[0016] Where, x e y and y are the input and output of the attention encoding layer, respectively, and W encod er The weights learned by the attention encoding layer. b represents the attention weights of the attention encoding layer obtained through the attention mechanism. e The bias vector is represented by *, which is the matrix multiplication operator. This is the Hadama product operator;
[0017] The GRU neural network takes the features after passing through the attention encoding layer as input; the GRU neural network controls the flow of input information through reset gates and update gates; its mathematical expression is:
[0018]
[0019] Where, r t To reset the door's control signal, used to control whether the previous hidden state was ignored; u t The gate control signal is used to update the gate and control whether to update the candidate hidden state variables; xt y t h t-1 and h t These are the current input, output, previous state, and current state, respectively. This is the candidate state at the current moment; W o Both are weight matrices, b u b r b h b o All are bias vectors; σ(·) is the sigmoid function; ° is the Hadamard product operator;
[0020] The attention decoding layer decodes features with long short-term memory and enhances the attention given to important information in the global features. Its mathematical expression is:
[0021] y = (x d *W decoder +b d )°W attd (3)
[0022] Where, x d y and y are the input and output of the attention decoding layer, respectively, and W decoder The weights learned by the attention decoding layer. b represents the attention weights of the attention decoding layer obtained through the attention mechanism. d is the bias vector; * is the matrix multiplication operator, and ° is the Hadamard product operator;
[0023] After the input data undergoes feature extraction via the Att-GRU block, it is normalized using a BN layer and activated using the ReLU activation function, the mathematical expression of which is:
[0024] y = ReLU(BN(x)) (4)
[0025]
[0026]
[0027] Where x and y are the input and output data, respectively; γ and β are two hyperparameters, μ B and These are the mean and variance calculated from all values on the same feature map during mini-batch processing; ∈ is a constant value added for the stability of the values.
[0028] The activated feature map is passed through a regression layer to obtain the velocity model corresponding to the final full waveform inversion.
[0029] Furthermore, the specific process of step 3 is as follows:
[0030] The optimal neural network model is obtained by selecting the output results of the neural network model saved at each iterative detection node according to the preset evaluation index obtained in step 2, and loading the corresponding model parameters into the neural network model to obtain the optimal neural network model; the model parameter update is frozen to keep it fixed, and the corresponding geological velocity model is predicted end-to-end using new unknown seismic observation data as input.
[0031] Furthermore, the preset evaluation metric is the mean square error between the predicted velocity model and the corresponding real velocity model in the label set.
[0032] The beneficial effects of this invention are as follows: This invention employs a GRU neural network, which exhibits temporal correlation and efficient nonlinear mapping, to model the forward modeling process in full waveform inversion. The inversion process is learned end-to-end through backpropagation of the neural network. Furthermore, an attention mechanism is introduced into the GRU neural network to emphasize important features in the input information. This invention significantly improves the accuracy of full waveform inversion while reducing computation time and enhancing its generalization ability. As a full waveform inversion method based on gated recurrent units and an attention mechanism, this invention can be widely applied to deep reservoir exploration tasks, enabling faster and clearer prediction of geological velocity models and accurate interpretation of geological properties. Attached Figure Description
[0033] Figure 1 This is a flowchart of the full waveform inversion process based on the attention mechanism and GRU neural network in an embodiment of the present invention;
[0034] Figure 2 This is a schematic diagram of the neural network model in an embodiment of the present invention;
[0035] Figure 3 This is a flowchart illustrating the training and testing steps of the model in an embodiment of the present invention. Detailed Implementation
[0036] The present invention will be further described below with reference to the accompanying drawings. It should be noted that this embodiment is based on the present technical solution and provides detailed implementation methods and specific operation processes, but the protection scope of the present invention is not limited to this embodiment.
[0037] like Figure 1 As shown, this embodiment provides a full waveform inversion method based on gated loop units and attention mechanisms, and its specific implementation process is as follows:
[0038] 1. Data acquisition, preprocessing, and dataset partitioning
[0039] Based on the actual conditions of the working face, a layout scheme for the seismic data detectors was designed, and actual seismic observation data was acquired, totaling 3100 data sets. The acquired data underwent preprocessing to obtain high-quality, high signal-to-noise ratio seismic data reflecting the actual geology. Data preprocessing included removing bad channels from the signal, removing random impulse noise, suppressing power frequency interference, shot gather equalization, and detector gather equalization. The corresponding geological velocity model was used as the label set to divide the data into training and test sets. Random sampling was performed on the preprocessed actual seismic data at a 5:1 ratio to construct the training and test sets, i.e., the training set size was 2600 data sets, and the test set size was 500 data sets.
[0040] 2. Construct the neural network model structure and continue training:
[0041] A neural network model for the forward modeling process in full waveform inversion mapping based on attention mechanism and GRU neural network is constructed, and relevant hyperparameters are set, including maximum number of iterations, learning rate, batch size, network depth and regularization decay factor.
[0042] The Kaiming initialization method initializes the parameters of the neural network model by using data from the training set and their corresponding labels as input data to train the neural network parameters. Figure 2 As shown in (b), the structure of the neural network model includes two cascaded Att-GRU blocks and a regression layer for feature decoding.
[0043] Specifically, for Att-GRU blocks, such as Figure 2 As shown in (a), the forward propagation process is as follows: the input data is first encoded by the attention encoding layer, which increases the attention to important features in the current information. This can be formalized as follows:
[0044] y = (x e *W encoder +b e )°W atte (1)
[0045] Where x e y is the input to the attention encoding layer, and W encoder W represents the weights learned in this layer. atte To obtain the attention weights of this layer through the attention mechanism, b e This is the bias vector. * is the matrix multiplication operator, and ° is the Hadamard product operator.
[0046] The GRU neural network takes attention-encoded features as input. GRU is a highly efficient recurrent neural network with long-term memory and the ability to mitigate the vanishing gradient problem during backpropagation. This network controls the flow of input information through reset and update gates. Its mathematical expression is:
[0047]
[0048] Where r t To reset the door's control signal, used to control whether the previously hidden state was ignored; u t The gate control signal is used to update the door, controlling whether to update the candidate hidden state variables. t y t h t-1 and h t These represent the current input, output, previous state, and current state, respectively. This is the candidate state at the current moment. W * It is a weight matrix, b * It is the bias vector. σ(·) is the sigmoid function. ° is the Hadamard product operator.
[0049] Finally, the attention decoding layer decodes features with long short-term memory and enhances the attention given to important information in the global features. Its mathematical expression is:
[0050]
[0051] Where x d y and y are the input and output of the attention decoding layer, respectively, and W decoder W represents the weights learned in this layer. attd To obtain the attention weights of this layer through the attention mechanism, b d This is the bias vector. * is the matrix multiplication operator, and ° is the Hadamard product operator.
[0052] After the input data undergoes feature extraction via an Att-GRU block, it is normalized using a BN layer and activated using the ReLU activation function, the mathematical expression of which is:
[0053] y = ReLU(BN(x)) (4)
[0054]
[0055]
[0056] Where x and y are the input and output data, respectively. γ and β are two hyperparameters, μ B and These represent the mean and variance calculated from all values on the same feature map during mini-batch processing. ∈ is a very small constant value added for numerical stability. The activated feature map passes through a feature decoding layer to obtain the velocity model corresponding to the final full waveform inversion, such as... Figure 3 As shown in the training section.
[0057] The Adam optimizer is used to train the parameters of this neural network model, and the loss function chosen is the mean squared error, whose mathematical expression is:
[0058]
[0059] Where y and These are the predicted results and the true labels, respectively. Let n be the loss function of the neural network model and n be the number of samples. During the training of the neural network, the loss function is continuously minimized, and the loss gradient of the model parameters is calculated based on error backpropagation to guide the direction of model parameter updates. In the continuous iterative update process, the performance of the neural network is tested using test set data at preset iteration nodes, and the model parameters are saved until the training phase with the maximum number of iterations is completed.
[0060] 3. Predicting inversion geological velocity models corresponding to unknown actual earthquake observation data
[0061] The output of the neural network model obtained in step 2 is optimally selected based on preset evaluation metrics, and the corresponding model parameters are loaded into the neural network model. Model parameter updates are frozen to keep them constant, and the corresponding geological velocity model is predicted end-to-end using new location seismic observation data as input. Figure 3 As shown in the prediction section.
[0062] For those skilled in the art, various corresponding changes and modifications can be made based on the above technical solutions and concepts, and all such changes and modifications should be included within the protection scope of the claims of this invention.
Claims
1. A full waveform inversion method based on gated recurrent units and attention mechanisms, characterized in that, Includes the following steps: Step 1: Obtain actual working face seismic observation data, and preprocess the obtained actual working face seismic observation data. Using the corresponding geological velocity model as the label set, construct the actual working face seismic observation data into training set and test set through random sampling. Step 2: Construct a neural network model for the forward modeling process in the full waveform inversion mapping based on gated recurrent units and attention mechanisms. Input the training set data obtained in Step 1 and its corresponding geological velocity model label set into the neural network model as input data, and set the hyperparameters required for training the neural network model, including the maximum number of iterations, learning rate, and network depth; initialize the parameters of the neural network model and train it; at the preset iteration detection node, test the performance of the neural network model using the test set data, and save the neural network model parameters at this time. Step 3: Select the optimal neural network model output results saved by each iterative detection node according to the preset evaluation index, and load the corresponding model parameters into the neural network model to obtain the optimal neural network model. Use the unknown seismic observation data as the input of the optimal neural network model, fix the parameters and perform inversion calculation. Based on the output results, obtain the predicted velocity model, which is the inversion mapping result of the subsurface medium physical property imaging. In step 2, the structure of the neural network model includes two cascaded Att-GRU blocks and a regression layer for feature decoding; The forward propagation process of the Att-GRU block is as follows: The input data is first encoded by an attention encoding layer, which increases the attention given to important features in the current information. This can be formalized as follows: Where, x e y and y are the input and output of the attention encoding layer, respectively, and W encoder The weights learned by the attention encoding layer. b represents the attention weights of the attention encoding layer obtained through the attention mechanism. e The bias vector is represented by *, which is the matrix multiplication operator. This is the Hadama product operator; The GRU neural network takes the features after passing through the attention encoding layer as input; the GRU neural network controls the flow of input information through reset gates and update gates; its mathematical expression is: Where, r t To reset the door's control signal, used to control whether the previous hidden state was ignored; u t The gate control signal is used to update the gate and control whether to update the candidate hidden state variables; x t y t h t-1 and h t These are the current input, output, previous state, and current state, respectively. This is the candidate state at the current moment; W o Both are weight matrices, b u b r b h b o All are bias vectors; σ(·) is the sigmoid function; ° is the Hadamard product operator; The attention decoding layer decodes features with long short-term memory and enhances the attention given to important information in the global features. Its mathematical expression is: Where, x d y and y are the input and output of the attention decoding layer, respectively, and W decoder The weights learned by the attention decoding layer. b represents the attention weights of the attention decoding layer obtained through the attention mechanism. d The bias vector is represented by *, which is the matrix multiplication operator. This is the Hadama product operator; After the input data undergoes feature extraction via the Att-GRU block, it is normalized using a BN layer and activated using the ReLU activation function, the mathematical expression of which is: y = ReLU(BN(x)) (4) Where x and y are the input and output data, respectively; γ and β are two hyperparameters, μ B and These are the mean and variance calculated from all values on the same feature map during mini-batch processing; ∈ is a constant value added for the stability of the values. The activated feature map is passed through a regression layer to obtain the velocity model corresponding to the final full waveform inversion.
2. The method according to claim 1, characterized in that, In step 1, the specific process of acquiring actual working face seismic observation data and performing data preprocessing on the obtained actual working face seismic observation data is as follows: Based on the actual conditions of the working face, a layout scheme for the location of seismic data detectors is designed to obtain actual seismic observation data of the working face; the obtained actual seismic observation data of the working face is preprocessed, including eliminating bad channels in the signal, removing random impulse noise, and suppressing power frequency interference.
3. The method according to claim 1, characterized in that, The specific process of step 3 is as follows: The optimal neural network model is obtained by selecting the output results of the neural network model saved at each iterative detection node according to the preset evaluation index obtained in step 2, and loading the corresponding model parameters into the neural network model to obtain the optimal neural network model; the model parameter update is frozen to keep it fixed, and the corresponding geological velocity model is predicted end-to-end using new unknown seismic observation data as input.
4. The method according to claim 1 or 3, characterized in that, The preset evaluation metric is the mean square error between the predicted velocity model and the corresponding real velocity model in the label set.