Transient electromagnetic data noise suppression method

Abstract

The invention provides a transient electromagnetic data noise suppression method based on a neural network, and the method mainly comprises the steps: constructing a CNN-LSTM neural network based on improved SPPFCMA optimization, generating synthetic training data, and designing a mixed loss function. The network comprises a convolutional layer, a pooling layer, a bidirectional long and short time memory layer and a full connection layer. By improving an SPPFCMA module, a CA attention mechanism is added after convolution layer output, and multi-scale pooling branches are adopted. A method of generating synthetic training data includes simulating a noiseless TEM signal using one-dimensional forward modeling and superimposing artificial noise. The training termination condition is that the mixed loss function value of the verification set does not decrease in continuous five iterations, and the method is used for denoising the actually measured data after preprocessing the actually measured data.

Description

Technical Field

[0001] The present invention belongs to the field of data processing, and more specifically relates to a method for suppressing transient electromagnetic data noise. Background Art

[0002] The transient electromagnetic method (TEM) is a time-domain electromagnetic exploration method based on the principle of electromagnetic induction and is widely used in the exploration of mineral resources and hydrological resources. The transient electromagnetic method uses a step wave or other pulsed current field source to emit a primary field into the ground. At the moment when the primary field is cut off, the attenuation characteristics of the secondary induced electromagnetic field generated by the underground geological body over time are measured. By analyzing the signals received by the instrument, the conductivity and burial depth of the target geological body are found, so as to achieve the purpose of detecting underground geological bodies. The transient electromagnetic signal is a secondary field signal that decays over time. When working in a strong noise interference environment such as a mining area or a factory area, the late signals received by the instrument are greatly affected by various natural and man-made interference noises, resulting in low resolution of deep geological bodies. Therefore, when using the transient electromagnetic method, the presence of noise has many adverse effects on the subsequent data processing and geological interpretation work. These interference signals may cover up the required useful signals. It is necessary to effectively suppress the noise and extract the useful information of the signals in order to better use the transient electromagnetic data for subsequent data processing and inversion interpretation.

[0003] The invention patent with the application publication number of CN117056677A proposes a transient electromagnetic signal denoising method based on the improved variational mode decomposition by the sparrow algorithm. This method uses the sparrow optimization algorithm to globally optimize the penalty factor α and the number of modes K in the variational mode decomposition to obtain the optimal parameters [K, α]; the collected transient electromagnetic signals are subjected to variational mode decomposition using the optimal parameter combination obtained after optimization to achieve the denoising process of the original signal. This method can avoid the mode mixing problem that may be caused by artificial parameter selection and improve the signal denoising effect.

[0004] The invention patent with the publication number of CN111650655A discloses a supervised transient electromagnetic signal denoising method based on non - negative matrix factorization. In the training stage of this method, the pure signal is processed by short - time Fourier transform and non - negative matrix factorization to obtain an atomic dictionary representing the respective characteristics of the signals. Then, in the denoising stage, the noisy signal is processed using the atomic dictionary and the denoising model to obtain a preliminarily estimated transient electromagnetic signal. Finally, the above steps are repeated multiple times. The late - stage data of the preliminarily estimated transient electromagnetic signal and the original early - and - middle - stage data of the noisy signal are respectively accumulated, and their arithmetic means are calculated. Then, the two are spliced to estimate the final complete transient electromagnetic signal. This invention can effectively remove the noise in the actual transient electromagnetic signal and improve the accuracy of transient electromagnetic signal inversion. However, because it is necessary to train the data, there are too many steps and a large amount of work, and due to excessive calculation data, it has high requirements for the performance of the processor itself.

[0005] In the early stage, methods such as wavelet transform, Kalman filter, and singular value decomposition are used for transient electromagnetic denoising. These methods all have their own disadvantages. Some lack adaptability, some have unsatisfactory denoising effects, and some have complex operation steps, etc. Summary of the Invention

[0006] The present invention mainly suppresses TEM noise based on a neural network, reduces the influence of noise such as Gaussian noise, power - frequency interference, and sharp pulses on the quality of TEM data, especially for the late - stage data contaminated by noise. Finally, it solves the problem of TEM denoising in a complex noise environment and successfully restores the late - stage data of TEM contaminated by noise.

[0007] To achieve the above - mentioned purpose, the present invention is implemented by the following technical solutions: The method includes:

[0008] 1) Construct a CNN - LSTM neural network optimized based on improved SPPF_CMA. The network sequentially includes a convolutional layer, a pooling layer, an SPPF_CMA module, a bidirectional long - short - term memory layer, and a fully - connected layer;

[0009] 2) The improvement of the SPPF_CMA module includes:

[0010] a. Add a CA attention mechanism after the output of the convolutional layer. Generate direction - aware feature maps through global average pooling in the horizontal and vertical directions to adjust the channel weights of the input features;

[0011] b. Adopt a multi - scale pooling branch, including 3 max - pooling layers with a pooling size of 5, 1 average - pooling layer with a pooling size of 9, and 1 average - pooling layer with a pooling size of 13. Concatenate the output features of each branch with the convolutional output;

[0012] 3) Design a hybrid loss function that includes a time-domain mean squared error term, a frequency-domain amplitude spectrum error term, and an L2 regularization term;

[0013] 4) Generate synthetic training data by one-dimensional forward modeling of noise-free TEM signals and adding artificial noise.

[0014] In one solution, the specific implementation of the CA attention mechanism includes:

[0015] For the input feature map X ∈ R^(H×W×C), perform global average pooling in the horizontal direction using a (H,1) pooling kernel and in the vertical direction using a (1,W) pooling kernel to obtain feature encodings in two directions; concatenate the horizontal and vertical feature encodings, generate an attention weight matrix through a 1×1 convolution and a Sigmoid activation function; multiply the weight matrix with the original feature map channel by channel, and output the weighted feature map X_att.

[0016] In one solution, in the multi-scale pooling branch, the 3 max pooling layers use the same pooling size of 5, but are respectively set with different stride parameters, specifically stride 2, stride 3, and stride 4; the pooling sizes of the two average pooling layers are 9 and 13 respectively, and the strides are both 5; after the feature maps output by each pooling branch are zero-padded to keep the sizes consistent, they are concatenated with the original convolution output along the channel dimension to form a multi-scale fusion feature.

[0017] In one solution, in the generation of synthetic data, the one-dimensional forward modeling uses a central loop device model, and calculates the response curves of different geoelectric models through the time-domain diffusion equation; the artificial noise added includes three parts: Gaussian white noise, 50Hz power frequency harmonics, and random pulse noise, the signal-to-noise ratio is controlled within the range of -5dB to 10dB, and the total energy ratio of the noise is not less than 30% of the amplitude of the original signal.

[0018] In one solution, the training termination condition is: when the value of the hybrid loss function on the validation set does not decrease in 5 consecutive iterations, and the current loss value is greater than 1.05 times the historical minimum loss value, save the historical optimal model parameters and terminate the training. The data volume of the validation set accounts for 15%-20% of the total training set.

[0019] In one solution, the bidirectional long short-term memory layer includes two LSTM sub-layers, forward and backward, with the number of hidden units in each sub-layer being 64-128, and the time step being equal to the number of sampling points of the input TEM data; the forward LSTM processes the time series from early to late, and the backward LSTM processes the reverse sequence from late to early, and their outputs are concatenated in the feature dimension and then input into the fully connected layer.

[0020] In one solution, the measured data needs to be preprocessed before denoising, including:

[0021] Perform time-channel normalization on the original voltage signal to map the amplitudes of each time channel to the interval of -1, 1; perform logarithmic compression processing on the late-stage data (decay time > 10 ms); arrange the multi-measurement point data of the entire survey line in spatial order to form a two-dimensional input matrix, and extract local spatio-temporal feature blocks through a sliding window for input into the network.

[0022] In one solution, the convolutional layer adopts a depthwise separable convolution structure, the convolution kernel size is 3×3, and the channel number expansion factor is set to 4; a batch normalization layer and a LeakyReLU activation function are connected after each convolutional layer, where the negative slope parameter of LeakyReLU is set to 0.01. Beneficial effects of the present invention:

[0023] 1. It has a good denoising effect on late-stage contaminated data: The TEM denoising model of the neural network constructed by us has a good denoising effect on contaminated late-stage TEM data, greatly enhancing the utilization efficiency of late-stage TEM data and deepening the exploration depth.

[0024] 2. It has strong universality and can resist various noises: Our method performs excellently in dealing with man-made noises that are difficult to eliminate by traditional methods. We have achieved targeted removal of common man-made noises such as square waves, charge-discharge triangular waves, power frequency interference, step waves, and sharp pulses. This innovation makes our method show stronger adaptability and practicality in complex actual environments.

[0025] 3. Intelligent denoising improves efficiency: It avoids the need for artificial selection of denoising parameters and other complex artificial operations such as wavelet denoising, variational mode decomposition, and singular value decomposition, and its simple and efficient denoising efficiency far exceeds other methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of the method of the present invention;

[0027] Figure 2 is a diagram of a CNN-LSTM network model optimized based on the improved SPPF. DETAILED DESCRIPTION OF THE INVENTION

[0028] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. Typical embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0029] Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as understood by those skilled in the technical field to which this invention belongs. The terms used in the description of this invention in this specification are only for the purpose of describing specific embodiments, and are not intended to limit this invention. For the convenience of understanding this invention, the following will describe this invention more comprehensively with reference to the relevant drawings. The drawings show typical embodiments of this invention. However, this invention can be implemented in many different forms and is not limited to the embodiments described in this invention. On the contrary, the purpose of providing these embodiments is to make the disclosure of this invention more thorough and comprehensive.

[0030] As Figure 1 shown, a transient electromagnetic data noise suppression method, the present invention constructs a CNN-LSTM neural network optimized based on improved SPPF. This neural network is used to remove the noise in the TEM signal, especially the denoising of TEM late-stage data. It includes:

[0031] S1. Construct a CNN-LSTM neural network optimized based on improved SPPF_CMA. The network sequentially includes a convolutional layer, a pooling layer, an SPPF_CMA module, a bidirectional long short-term memory layer, and a fully connected layer.

[0032] The constructed network consists of a convolutional layer, a pooling layer, SPPF_CMA, a bidirectional long short-term memory layer, and a fully connected layer. The specific network model is shown in Figure 1 . We improve SPPF by adding a CA attention mechanism and two average pooling (Avg Pool) parallel to the max pooling (Max Pool) to obtain SPPF_CMA. The first change here is to add a CA attention mechanism to adjust the weights of each channel, so that the network can pay more attention to important features and suppress irrelevant features. Then, multiple pooling branches (Max Pool and Avg Pool) are used to extract features from different scales. The sizes of the three Max Pool are all: pool_size = 5. The sizes of the two Avg Pool with different scales are respectively: pool_size = 9, pool_size = 13.

[0033] S2. The improvement of the SPPF_CMA module includes:

[0034] As Figure 2 shown, first, a convolutional layer Conv is used to extract local features:

[0035] X conv = Conv(X) = W conv *X + b conv , W conv ∈R k×C

[0036] Among them, X is the input data, and W conv is the convolution kernel, C is the output of the convolutional layer, k is the size of the convolution kernel, and b conv is the bias term of the convolutional layer.

[0037] Then, it is processed through the CA attention mechanism described above to obtain the data X with enhanced features att :

[0038] X att = CA(X conv )

[0039] After that, features of different scales are extracted through multi-scale pooling operations (max pooling and average pooling):

[0040] Max pooling:

[0041] P max1 = MaxPooling(X att , pool_size = 5)

[0042] P max2 = MaxPooling(P max1 , pool_size = 5)

[0043] P max3 = MaxPooling(P max2 , pool_size = 5)

[0044] Average pooling:

[0045] P avg1 = AveragePooling(X att , pool_size = 9)

[0046] P avg2 = AveragePooling(X att , pool_size = 13)

[0047] Finally, all pooling branches and convolutional outputs are concatenated together, and the concatenated features will be used as the input for downstream processing.

[0048] X concat = Concatenate(X conv , P max1 , P max2 , P max3 , P avg1 , P avg2 )

[0049] CA Attention Mechanism: The CA mechanism first performs global average pooling on the input feature map in the horizontal and vertical directions respectively to obtain two direction-aware feature maps. For the input feature map X ∈ R H×W×C , pooling kernels of sizes (H, 1) and (1, W) are used to encode each channel, and the output feature maps of the c-th channel are obtained from the horizontal and vertical directions, with the size of h × w. The formula is as follows:

[0050]

[0051] where x c represents the feature of the input feature map in the c-th channel, with a specified height and width. Then, by obtaining the horizontal direction feature map z h and the vertical direction feature map z w , 1×1 convolution F1 and non-linear activation function δ are used to perform feature aggregation on them, so as to effectively extract the relationship between channels while capturing the region of interest, and finally obtain the intermediate feature maps in two directions. The formula is as follows:

[0052] f = δ(F1([z h , z w )) Then, f is decomposed into two tensors f h and f w along the spatial dimension, and 1×1 convolution kernels F h and F w are used to perform convolution to make the number of channels of the two tensors the same as the input. Finally, the weight map is obtained through the sigmoid activation function σ. The formula is as follows:

[0053]

[0054] Finally, the feature map is multiplied by the two weight maps to obtain the output y c (i, j) of the CA mechanism module. The formula is as follows:

[0055]

[0056] where represent the weights of the c-th channel respectively.

[0057] S3. Design a hybrid loss function, including the time-domain mean square error term, the frequency-domain amplitude spectrum error term, and the L2 regularization term;

[0058] The loss function consists of three parts: the time-domain error, the frequency-domain error, and the regularization term, which can optimize the model in multiple aspects, balance the time-domain and frequency-domain characteristics, and help avoid overfitting and improve the generalization ability.

[0059] Evaluating the difference between the output of the model and the true signal in the time domain can capture the direct form and instantaneous characteristics of the signal. The frequency domain error, on the other hand, measures from the frequency characteristics of the signal and is particularly suitable for capturing the periodicity or frequency components of the signal. When dealing with noisy signals, the frequency domain error helps filter out the noise components and highlight the frequency characteristics of the signal. The time domain error and the frequency domain error evaluate the signal quality from different perspectives. Combining these two errors can help the model avoid getting stuck in local optima in a specific dimension during the training process.

[0060] The addition of the L2 norm effectively prevents the model from overfitting to the training data, enabling the model to learn a smoother function with better generalization ability.

[0061] Loss = L t +L f + λL2

[0062] Where, L t is the time domain error, L f is the frequency domain error, L2 is the L2 norm regularization term, and λ is the regularization coefficient.

[0063]

[0064] In the above formula, N is the number of data points, y true is the true signal, y pre is the predicted signal.

[0065]

[0066] In the above formula, N is the number of data points, y true is the true signal, y pre is the predicted signal, F represents the Fourier transform operation, and f represents the frequency component.

[0067]

[0068] In the above formula, M is the total number of weight parameters, and W represents the weight parameter matrix of the model.

[0069] S4. Generate synthetic training data by simulating noise-free TEM signals through one-dimensional forward modeling and adding artificial noise.

[0070] Dataset construction: To train a neural network, a dataset with a sufficient number of samples needs to be constructed. However, since there is no ideal TEM data without natural sounds, we here use the TEM data obtained from one-dimensional forward modeling and combine it with artificially simulated noise to synthesize a dataset for training the LSTM-Autoencoder neural network. We constructed 1000 different one-dimensional resistivity models and obtained 1000 forward modeling data through forward modeling. These forward modeling data are all transient data obtained at a sampling rate of 100 MHz in the time range of 0 - 0.001 s, and each data has 1000 sampling points. In addition, we artificially simulated different types and intensities of noise, including telluric noise, power frequency noise, and Gaussian noise. The mixed noise with different interference intensities is superimposed on the pure forward modeling data, and finally 20,000 noisy data samples are obtained, which are divided into a training set and a test set at a ratio of 4:1. That is, there are 16,000 samples in the training set and 4,000 samples in the test set.

[0071] The method for constructing the dataset is as follows: By constructing multiple different one-dimensional resistivity models and performing forward modeling to obtain forward modeling data corresponding to the multiple one-dimensional resistivity models; artificially simulating noise data of different types and intensities; where the noise data includes telluric noise, power frequency noise, and Gaussian noise; superimposing the multiple forward modeling data and the multiple noise data with different weights, and using the superimposed multiple data as the dataset; dividing the dataset according to a set ratio to obtain a training set and a test set. The method for obtaining the dataset is:

[0072]

[0073] Among them, D final is the superimposed dataset, is the i-th forward modeling data, is the i-th noise data; ω forward and ω noise are the weight values of the forward modeling data and the noise data, respectively used to adjust the proportion of the forward modeling data and the noise data during superposition; N is the data volume of the forward modeling data.

[0074] ω forward and ω noise , the methods for obtaining them are respectively:

[0075]

[0076] ω noise = 1 - ω forward ;

[0077] SNR target is the preset target signal-to-noise ratio.

[0078] Network training mainly includes: data preprocessing, network parameter adjustment, network training, and model verification.

[0079] (1) Data preprocessing:

[0080] For the input dataset, we first perform normalization on it. Perform min-max normalization on its original data. In the following formula, x is the original data, x mean is the average value of the logarithm of the original data, x max is the maximum value in the original data, x min is the minimum value in the original data, and x' is the normalized data:

[0081]

[0082] After normalizing the data, considering that the training effect is relatively poor when one-dimensional data is input into the neural network. We fold and reconstruct each data into two-dimensional data and then input it into the network. That is, convert the one-dimensional data of 9600×1 into two-dimensional data of 100×96×1.

[0083] (2) Parameter adjustment and model training:

[0084] After the data preprocessing is completed, initially train the model and adjust the model parameters. Except for the fixed input and output dimensions, other parameters are usually adjusted according to the training and verification effects to find the optimal values. After the parameter adjustment is completed, train the model. During the training process, the data needs to be input in batches, calculate the Loss between the predicted output and the theoretical output in each iteration and optimize it with the Adam optimizer until the Loss converges. Finally, save the trained model.

[0085] During the model training process, it is also necessary to verify the training effect to avoid overfitting. Use the previously divided validation set to verify the neural network, input the data in the validation set into the recurrent neural network for prediction, and compare the prediction results with the corresponding ideal outputs. If the validation error reaches convergence and then starts to increase continuously, it means that the network has reached the overfitting state and training needs to be stopped.

[0086] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The said program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above various methods. Among them, the said storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0087] It should be understood that the detailed description of the technical solutions of the present invention by means of the preferred embodiments above is illustrative rather than restrictive. Those of ordinary skill in the art can modify the technical solutions recorded in each embodiment on the basis of reading the specification of the present invention, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A method for suppressing transient electromagnetic data noise, characterized in that: The method includes: Construct a CNN-LSTM neural network based on improved SPPF_CMA optimization, wherein the network sequentially comprises a convolutional layer, a pooling layer, an SPPF_CMA module, a bidirectional long short-term memory layer, and a fully connected layer; The improvements of the SPPF_CMA module include: a. Add the CA attention mechanism after the output of the convolutional layer, generate the direction-aware feature map through global average pooling in the horizontal and vertical directions, and adjust the channel weights of the input features; b. Use multi-scale pooling branches, including 3 maximum pooling layers with a pooling size of 5, 1 average pooling layer with a pooling size of 9, and 1 average pooling layer with a pooling size of 13, and concatenate the output features of each branch with the convolution output; Design a hybrid loss function, which includes the mean square error term in the time domain, the amplitude spectrum error term in the frequency domain, and the L2 regularization term; Synthetic training data are generated by simulating noise-free TEM signals through one-dimensional forward modeling and superimposing artificial noise.

2. A method for suppressing transient electromagnetic data noise according to claim 1, characterized in that: The specific implementation of the CA attention mechanism includes: For the input feature map X∈R^(H×W×C), global average pooling is performed using the (H,1) pooling kernel in the horizontal direction and the (1,W) pooling kernel in the vertical direction to obtain feature encoding in two directions; the horizontal and vertical feature encodings are concatenated, and the attention weight matrix is ​​generated through 1×1 convolution and Sigmoid activation function; the weight matrix is ​​multiplied by the original feature map channel by channel, and the weighted feature map X_att is output.

3. The method for suppressing transient electromagnetic data noise according to claim 1, characterized in that: In the multi-scale pooling branch, the three maximum pooling layers use the same pooling size of 5, but set different step parameters, specifically step 2, step 3 and step 4; the pooling sizes of the two average pooling layers are 9 and 13 respectively, and the step sizes are both 5; the feature maps output by each pooling branch are zero-padded to keep the size consistent, and then spliced ​​with the original convolution output along the channel dimension to form multi-scale fusion features.

4. The method for suppressing transient electromagnetic data noise according to claim 1, characterized in that: In the generation of synthetic training data, a one-dimensional forward simulation adopts a central loop device model, and the response curves of different geoelectric models are calculated by the time domain diffusion equation; the superimposed artificial noise includes three parts: Gaussian white noise, 50 Hz power frequency harmonics and random pulse noise, the signal-to-noise ratio is controlled within the range of -5dB to 10dB, and the total noise energy accounts for no less than 30% of the original signal amplitude.

5. The method for suppressing transient electromagnetic data noise according to claim 1, characterized in that: The training termination condition is: when the mixed loss function value of the validation set does not decrease in five consecutive iterations and the current loss value is greater than 1.05 times the historical minimum loss value, the historical optimal model parameters are saved and the training is terminated; the validation set data volume accounts for 15%-20% of the total training set data volume.

6. The method for suppressing transient electromagnetic data noise according to claim 1, characterized in that: The bidirectional long short-term memory layer includes two LSTM sublayers, forward and backward, each sublayer has 64-128 hidden units, and the time step is equal to the number of sampling points of the input TEM data; the forward LSTM processes the time series from early to late, and the backward LSTM processes the reverse sequence from late to early, and the outputs of the two are spliced ​​in the feature dimension and input into the fully connected layer.

7. The method for suppressing transient electromagnetic data noise according to claim 1, characterized in that: The measured data needs to be preprocessed before denoising, including: Normalize the original voltage signal in time channel so that the amplitude of each time channel is mapped to -1,1 interval; logarithmic compression processing is performed on late data (decay time>10ms); the multi-point data of the entire survey line are arranged in spatial order to form a two-dimensional input matrix, and the local spatiotemporal feature blocks are extracted through a sliding window and input into the network.

8. The method for suppressing transient electromagnetic data noise according to claim 1, characterized in that: The convolution layer adopts a depth-wise separable convolution structure, the convolution kernel size is 3×3, and the channel expansion factor is set to 4; each convolution layer is followed by a batch normalization layer and a LeakyReLU activation function, where the negative slope parameter of the LeakyReLU is set to 0.01.

Images