Improved cae-resgru transient electromagnetic real-time inversion method
Patent Information
- Application Number
- CN202311543526.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-20
- Publication Date
- 2026-08-28
- Estimated Expiration
- 2043-11-20
AI Technical Summary
[0005]本发明所要解决的技术问题在于提供一种改进的CAE-ResGRU瞬变电磁实时反演方法，可以充分解决瞬变电磁探测中无法进行快速且高精度反演的问题
[0036] This invention directly expresses the relationship between electromagnetic data input and resistivity model output. It utilizes a deep neural network to receive forward modeling data, and performs a single mapping using a trained model to predict relatively detailed subsurface stratigraphic inversion results in real time. Furthermore, the CAE model can extract richer input data information, improving model accuracy; the GRU model can effectively process time-series data; the residual network is easy to train and optimize, avoiding the vanishing gradient problem and improving accuracy; and the attention mechanism enables parallel computation in the decoder, resulting in high speed and good performance. This method effectively solves the problem of the inability to perform fast and high-precision inversion in transient electromagnetic detection.
Smart Images
Figure CN117471561B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of geophysical exploration, specifically a CAE-ResGRU transient electromagnetic real-time inversion method based on the attention mechanism for complex environments. Background Technology
[0002] Transient electromagnetic (TEM) technology is a research hotspot in the field of geophysics. Compared with other geophysical exploration methods, it has advantages such as high efficiency, high resolution, large detection depth, and sensitivity to low resistivity. In recent years, it has been widely used in the search for underground resources, urban construction, and geological disaster prevention and detection, demonstrating high practical application value.
[0003] Transient electromagnetic (TEM) exploration utilizes inversion techniques based on observed response data to detect and analyze unknown subsurface geological structures. However, TEM inversion faces several challenges: traditional one-dimensional inversion techniques rely on initial model settings, making them prone to getting trapped in local minima and failing to converge, leading to inversion errors; while high-dimensional inversion is limited by complex forward modeling algorithms, involves massive amounts of data, and is computationally slow. Both methods share the drawback that each inversion requires multiple forward modeling calculations, consuming significant time and preventing real-time inversion.
[0004] Currently, some research in geophysics has begun to utilize various deep learning models for the inversion of traditional problems. While deep learning methods in geophysics are becoming increasingly sophisticated, there is still room for optimization and improvement. Furthermore, the requirements for detailed and accurate mapping of subsurface structures using transient electromagnetic methods are becoming increasingly stringent. Therefore, it is necessary to provide a high-precision, high-efficiency real-time inversion algorithm for transient electromagnetic detection data of complex subsurface environments with multiple strata and anomalies, providing accurate, reliable, and rapid real-time data support for subsurface exploration applications using transient electromagnetic methods. Summary of the Invention
[0005] The technical problem to be solved by this invention is to provide an improved CAE-ResGRU transient electromagnetic real-time inversion method, which can fully solve the problem that rapid and high-precision inversion cannot be performed in transient electromagnetic detection.
[0006] This invention is implemented as follows:
[0007] An improved CAE-ResGRU transient electromagnetic real-time inversion method, comprising:
[0008] Step 1: Obtain transient electromagnetic forward response data, normalize all data and organize them into a dataset, and construct the input matrix;
[0009] Step 2: Extract local features using a convolutional autoencoder module with an added attention mechanism;
[0010] Step 3: The features extracted from the convolutional autoencoder module are added to the residual gated recurrent unit model of the residual network to extract the temporal features of the data and train and optimize the residual gated recurrent unit model. When the specified number of training times is reached or the validation set reaches a certain accuracy, the optimal transient electromagnetic neural network inversion model is obtained.
[0011] Step 4: Input the test data into the transient electromagnetic neural network inversion model trained in Step 3, calculate the deviation between the output value and the true value in the test phase, and obtain the prediction effect of the transient electromagnetic neural network inversion model.
[0012] Step 5: When the prediction effect meets the evaluation requirements, the transient electromagnetic neural network inversion model is used to perform inversion calculations on the transient electromagnetic data.
[0013] Furthermore, transient electromagnetic forward response data were acquired and normalized.
[0014]
[0015] Where X is the initial data, X min X max These are the minimum and maximum values in the initial data, respectively.
[0016] Furthermore, the convolutional autoencoder includes a convolutional encoder and a convolutional decoder. An attention mechanism is added to the output of the convolutional encoder. The calculation process of the convolutional encoder is as follows:
[0017] h n =F(X*W n +b n )
[0018] Among them, h n The features extracted by the nth convolutional kernel, where F represents the activation function, X represents the input sample, * represents the convolution calculation, and W... n b represents the weight value of the nth convolutional kernel of the encoder. n This represents the bias value of the nth convolutional kernel of the encoder;
[0019] The Attention mechanism is used to selectively assign weights to data at different levels, giving higher weights to important information and lower weights to less important information. The calculation is as follows:
[0020]
[0021]
[0022] There are n inputs in total. Let a be the temporary weight of the output value of the i-th convolutional encoder hidden layer with respect to the current input. ni For the i-th attention weight, s i For the i-th feature after weight adjustment, h i This is the output of the i-th hidden layer in the encoder;
[0023] The convolutional decoder decodes the features extracted by each convolutional kernel and synthesizes them into reconstructed samples. The decoding calculation process is as follows:
[0024] Y i =F′(s i *W′ n +b′ n )
[0025] Among them, Y i Represents the reconstructed sample, W′ n Let b′ represent the weight value of the nth convolutional kernel in the decoder. n This represents the bias value of the nth convolutional kernel in the decoder;
[0026] The convolutional autoencoder is trained to minimize the reconstruction error. The reconstruction loss function is:
[0027]
[0028] Where p is the number of input images, X j For the j-th input image, Y j For X j The corresponding reconstructed image.
[0029] Furthermore, in step 3, the difference-gated recurrent unit model consists of a stack of four layers of GRU network and dropout layers, arranged in the manner of GRU layer, dropout layer, activation function, GRU layer, dropout layer. The feature vectors in the convolutional autoencoder and the normalized original samples are used as inputs to the residual gated recurrent unit model, and the transient electromagnetic neural network inversion model is obtained through continuous calculation and optimization.
[0030] Furthermore, the GRU network calculation method is as follows:
[0031] r t =σ(Z) t W zr +h t-1 W hr +b r )
[0032] v t =σ(Z)t W zv +h t-1 W hv +b v )
[0033] h t =v t *h t-1 +(1-v t )*tanh(Z t W zh +(r t *h t-1 W hh +b h )
[0034] Among them, Z t For the output matrix; W zr W zv W zh These are the different input weight matrices at the input end; W hr W hv W hh These are different output matrices from the hidden layer at the previous time step; b r b v b h σ is the bias matrix; σ is the sigmoid activation function; tanh is the hyperbolic tangent activation function; r t To update the gate output; v t For overlapping gate output; h t This represents the output of the hidden layer neuron at time t.
[0035] Compared with the prior art, the beneficial effects of this invention are as follows:
[0036] This invention directly expresses the relationship between electromagnetic data input and resistivity model output. It utilizes a deep neural network to receive forward modeling data, and performs a single mapping using a trained model to predict relatively detailed subsurface stratigraphic inversion results in real time. Furthermore, the CAE model can extract richer input data information, improving model accuracy; the GRU model can effectively process time-series data; the residual network is easy to train and optimize, avoiding the vanishing gradient problem and improving accuracy; and the attention mechanism enables parallel computation in the decoder, resulting in high speed and good performance. This method effectively solves the problem of the inability to perform fast and high-precision inversion in transient electromagnetic detection.
[0037] This invention provides a CAE-ResGRU deep learning method based on an attention mechanism for transient electromagnetic inversion in complex environments. On one hand, using a convolutional autoencoder with an attention mechanism for unsupervised learning not only achieves dimensionality reduction but also quickly and accurately extracts effective features from transient electromagnetic forward modeling data, thus giving the model high robustness and generalization ability. On the other hand, employing a residual GRU deep learning network fully extracts the sequential characteristics of the input data, avoiding the gradient vanishing problem and improving the model's convergence speed and prediction accuracy. The iteratively obtained model significantly reduces training time and improves prediction accuracy. Attached Figure Description
[0038] Figure 1 A flowchart illustrating the method provided in an embodiment of the present invention;
[0039] Figure 2 A schematic diagram of the convolutional autoencoder structure used in the method embodiments of the present invention is provided;
[0040] Figure 3 A schematic diagram of the Attention architecture used in the method embodiments of the present invention is provided;
[0041] Figure 4 A schematic diagram of the ResGRU structure used in the method embodiments of the present invention is provided;
[0042] Figure 5 A schematic diagram of the GRU network architecture used in the method embodiment of the present invention is provided. Detailed Implementation
[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0044] This invention proposes a CAE-ResGRU transient electromagnetic real-time inversion method based on the attention mechanism, suitable for complex environments.
[0045] like Figure 1 As shown, the specific implementation process of the method is as follows:
[0046] Step 1: Obtain transient electromagnetic forward response data, perform normalization processing, and construct the input matrix:
[0047] Based on a large amount of transient electromagnetic forward response data, data normalization was performed:
[0048]
[0049] Where X is the initial data, X min X max These represent the minimum and maximum values in the initial data, respectively.
[0050] After the data is normalized, the dataset is organized to form the input matrix.
[0051] Step 2: Establish a convolutional autoencoder model based on the attention mechanism, inputting normalized data (i.e., the input matrix) for feature extraction.
[0052] The convolutional autoencoder model based on the attention mechanism includes a convolutional encoder and a convolutional decoder. Features obtained after data is encoded by the convolutional encoder are decoded by the convolutional decoder and output. The convolutional encoder consists of three sets of convolutional layers and pooling layers, with the following structure: first convolutional layer, first activation function, first pooling layer; second convolutional layer, second activation function; third convolutional layer, second activation function, second pooling layer. The convolutional decoder consists of three sets of upsampling layers and deconvolutional layers, with the following structure: first upsampling layer, third activation function, fourth convolutional layer; fourth activation function, fifth convolutional layer; second upsampling layer, fourth activation function, fifth convolutional layer. The first and second pooling layers use Max Pooling, and the first and second upsampling layers use Max UnPooling. In addition, an attention mechanism is added between the convolutional encoder and convolutional decoder in the convolutional autoencoder model.
[0053] like Figure 2 , Figure 3 As shown, the normalized transient electromagnetic forward modeling data is input into a convolutional autoencoder model based on the attention mechanism.
[0054] Feature extraction is performed using an improved convolutional autoencoder, and the encoder is continuously constrained by reconstruction loss to extract effective feature vectors from the input data.
[0055] In a convolutional autoencoder, feature extraction and output reconstruction are computed using a convolutional neural network. The encoding computation process of a convolutional decoder is as follows:
[0056] h n =F(X*W n +b n )
[0057] Among them, h n The features extracted by the nth convolutional kernel, where F represents the activation function, X represents the input sample, * represents the convolution calculation, and W... n b represents the weight value of the nth convolutional kernel of the encoder. n This represents the bias value of the nth convolutional kernel of the encoder.
[0058] By introducing an attention mechanism, biased selection can be applied to data at different levels, assigning greater weight to important information and less important information to less significant features. This reduces model complexity and improves computational efficiency. The calculation process is as follows:
[0059]
[0060]
[0061] There are a total of n inputs. Let a be the temporary weight of the output value of the i-th convolutional encoder hidden layer with respect to the current input. ni For the i-th attention weight, s i For the i-th feature after weight adjustment, h i This is the output of the i-th hidden layer in the encoder;
[0062] By using the Attention mechanism to change the originally automatically assigned temporary weights Based on attention weight a ni The output of the encoder's hidden layer is adjusted, and the weights are redistributed to obtain the weighted feature s. i The output result is then obtained through a fully connected layer.
[0063] After processing by the Attention mechanism, feature vectors that further reflect the characteristics of the sample data are obtained. The features extracted by each convolutional kernel are decoded and synthesized into a reconstructed sample. The decoding calculation process of this convolutional decoder is as follows:
[0064] Y i =F′(s i *W′ n +b′ n )
[0065] Among them, Y i Represents the reconstructed sample, W′ n b′ represents the weight value of the nth convolutional kernel in the convolutional decoder. n This represents the bias value of the nth convolutional kernel in the convolutional decoder.
[0066] The convolutional autoencoder is trained to minimize the reconstruction error. The reconstruction loss function is:
[0067]
[0068] Where p is the number of input images, X j For the j-th input image, Y j For X jThe corresponding reconstructed image.
[0069] Step 3: Establish a Residual Gated Recurrent Unit (ResGRU) model, inputting the feature vectors from a convolutional autoencoder based on the Attention mechanism, and train and optimize the Residual Gated Recurrent Unit model. For example... Figure 4 As shown, the feature vectors (the encoder output adjusted by the attention mechanism) from the improved convolutional autoencoder and the normalized original samples are used as inputs to the Residual Gated Recurrent Unit (ResGRU) model. The ResGRU model consists of four stacked GRU layers and dropout layers, arranged in the order of GRU layer, dropout layer, activation function, GRU layer, dropout layer. The activation function used is the Tanh function. By utilizing the residual structure (where the input data passes through GRU layers, dropout layers, GRU layers, and dropout layers before being output, and simultaneously, the same input data is directly added to the output), the network is optimized into a parallel network structure, improving the convergence speed of the Residual Gated Recurrent Unit model. Adding dropout layers prevents overfitting in the Residual Gated Recurrent Unit model and provides stronger generalization ability.
[0070] GRU network in the Residual Gated Recurrent Unit (ResGRU) model (see...) Figure 5 As shown, compared to RNN, it adds an update gate r. t and overlapping gate v t This gives it a memory function, which can solve the information dependency problem. The GRU network calculation method is as follows:
[0071] r t =σ(Z) t W zr +h t-1 W hr +b r )
[0072] v t =σ(Z) t W zv +h t-1 W hv +b v )
[0073] h t =v t *h t-1 +(1-v t )*tanh(Z t W zh +(r t *h t-1 W hh +b h )
[0074] Among them, Z t For the output matrix; W zr W zv W zh These are the different input weight matrices at the input end; W hr W hv W hh These are different output matrices from the hidden layer at the previous time step; b r b v b h σ is the bias matrix; σ is the sigmoid activation function; tanh is the hyperbolic tangent activation function; r t To update the gate output; v t For overlapping gate output; h t This represents the output of the hidden layer neuron at time t.
[0075] The optimization function of the GRU layer uses the Adam algorithm, which can dynamically adjust the learning rate of each parameter. Through continuous calculation and optimization, the final transient electromagnetic inversion model is obtained.
[0076] Step 4: Evaluate the effectiveness of the transient electromagnetic inversion model using the test set, and output the transient electromagnetic inversion results:
[0077] By inputting the test data into the optimal transient electromagnetic neural network inversion model obtained through training and optimization, and calculating the deviation between the output value and the true value during the test phase, the prediction effect of the inversion model can be obtained.
[0078] Example
[0079] Taking a complex geological structure detection scenario with up to 5 underground layers and multiple unknown anomalies as an example, this paper details the CAE-ResGRU transient electromagnetic real-time inversion method based on the Attention mechanism proposed in this invention. The specific algorithm implementation steps are as follows:
[0080] 1. 10,000 sets of forward response data were obtained through transient electromagnetic forward modeling simulation. This includes 1,000 examples of forward response data for a uniform half-space model, 1,000 examples for a two-layer geoelectric structure, 2,000 examples for a three-layer geoelectric structure, 3,000 examples for a four-layer geoelectric structure, and 3,000 examples for a five-layer geoelectric structure. The geoelectric structure includes the number of layers, the thickness of each layer, and the corresponding resistivity. For example, a three-layer geoelectric structure consists of three layers: the first layer is 200 meters thick with a resistivity of 100 Ω / m; the second layer is 100 meters thick with a resistivity of 30 Ω / m; and the third layer is 80 meters thick with a resistivity of 200 Ω / m. The data were normalized to reduce dependence on initial values and suppress overfitting.
[0081]
[0082] Where X is the initial data, X min X max These represent the minimum and maximum values in the initial data, respectively.
[0083] Of the 10,000 sets of forward modeling data mentioned above, 9,000 sets are the training set and 1,000 sets are the test set. The number of the five geoelectric models in the test set is extracted proportionally.
[0084] 2. The 9000 sets of data mentioned above are input into a convolutional autoencoder model with an attention mechanism for feature extraction. During model training, the Adam optimization algorithm is used, with an initial learning rate of 0.001, a batch size of 128, and 1000 iterations. The encoder contains 128-channel 3x3 convolutional layers, 64-channel 3x3 convolutional layers, and 16-channel 3x3 convolutional layers. The pooling layers use Max Pooling with a 2x2 kernel and a stride of 2. The decoder contains corresponding 32-channel 3x3 convolutional layers, 32-channel 3x3 convolutional layers, and 16-channel 3x3 convolutional layers. The upsampling layers use Max UnPooling with a 2x2 kernel and a stride of 2.
[0085] 3. The feature vectors from the improved convolutional autoencoder and the normalized original samples are used as input to the ResGRU model. The ResGRU model consists of 4 stacked GRU layers, and the network is optimized into a parallel network structure using residual structure. The activation function is the Tanh function, the number of hidden layer nodes in the GRU network is 64, the number of GRU layers is 4, the batch size is 128, the optimization function is Adam, the learning rate is 0.001, the dropout value is set to 0.05, and the number of iterations is 1000. The training data is iterated repeatedly until the set number of iterations is reached, and the model training is complete.
[0086] 4. Input the test set data sequentially into the already trained Attention-based CAE-ResGRU model to obtain the model's prediction results. Test and evaluate the model on the test set to select the optimal set of model parameters. At this point, the model is complete.
[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An improved CAE-ResGRU transient electromagnetic real-time inversion method, characterized in that, The method includes: Step 1: Obtain transient electromagnetic forward response data, normalize all data and organize them into a dataset, and construct the input matrix; Step 2: Extract local features using a convolutional autoencoder module with an added attention mechanism; Step 3: The features extracted from the convolutional autoencoder module are added to the residual gated recurrent unit model of the residual network to extract the temporal features of the data and train and optimize the residual gated recurrent unit model. When the specified number of training times is reached or the validation set reaches a certain accuracy, the optimal transient electromagnetic neural network inversion model is obtained. Step 4: Input the test data into the transient electromagnetic neural network inversion model trained in Step 3, calculate the deviation between the output value and the true value in the test phase, and obtain the prediction effect of the transient electromagnetic neural network inversion model. Step 5: When the prediction effect meets the evaluation requirements, the transient electromagnetic neural network inversion model is used to perform inversion calculations on the transient electromagnetic data; The convolutional autoencoder includes a convolutional encoder and a convolutional decoder. An attention mechanism is added to the output of the convolutional encoder. The calculation process of the convolutional encoder is as follows: ， in, The features extracted by the nth convolutional kernel, where F represents the activation function, X represents the input sample, and * represents the convolution calculation. This represents the weight value of the nth convolutional kernel of the encoder. This represents the bias value of the nth convolutional kernel of the encoder; The Attention mechanism is used to selectively assign weights to data at different levels, giving higher weights to important information and lower weights to less important information. The calculation is as follows: ， ， There are a total of n inputs. Let be the temporary weight of the output value of the i-th convolutional encoder hidden layer with respect to the current input. For the i-th attention weight, For the i-th feature after weight adjustment, This is the output of the i-th hidden layer in the encoder; The convolutional decoder decodes the features extracted by each convolutional kernel and synthesizes them into reconstructed samples. The decoding calculation process is as follows: ， in, Indicates the reconstructed sample. This represents the weight value of the nth convolutional kernel in the decoder. This represents the bias value of the nth convolutional kernel in the decoder; The convolutional autoencoder is trained to minimize the reconstruction error. The reconstruction loss function is: ， Where p is the number of input images. For the j-th input image, for The corresponding reconstructed image.
2. The improved CAE-ResGRU transient electromagnetic real-time inversion method according to claim 1, characterized in that, Acquire transient electromagnetic forward response data and perform data normalization: ， in, For initial data, , These are the minimum and maximum values in the initial data, respectively.
3. The improved CAE-ResGRU transient electromagnetic real-time inversion method according to claim 1, characterized in that, In step 3, the differential gated recurrent unit model consists of a stack of four GRU networks and dropout layers, arranged in the manner of GRU layer, dropout layer, activation function, GRU layer, dropout layer. The feature vectors in the convolutional autoencoder and the normalized original samples are used as inputs to the residual gated recurrent unit model. After continuous calculation and optimization, the transient electromagnetic neural network inversion model is obtained.
4. The improved CAE-ResGRU transient electromagnetic real-time inversion method according to claim 3, characterized in that, The GRU network calculation method is as follows: ， ， ， in, This is the output matrix; , , These are the different input weight matrices at the input end; , , These are the different output matrices from the hidden layer at the previous time step; , , This is the bias matrix; It is the sigmoid activation function; It is the hyperbolic tangent activation function; To update the gate output; Output is an overlapped gate; This represents the output of the hidden layer neuron at time t.