A transient electromagnetic data noise suppression method

Abstract

The application provides a transient electromagnetic data noise suppression method based on a neural network, mainly comprising the following steps: constructing a CNN-LSTM neural network based on an improved SPPF_CMA optimization, generating synthetic training data, and designing a hybrid loss function. The network comprises a convolutional layer, a pooling layer, a bidirectional long short-term memory layer and a fully connected layer. By improving the SPPF_CMA module, the CA attention mechanism is added after the convolutional layer output, and a multi-scale pooling branch is adopted. The method for generating synthetic training data comprises the following steps: using one-dimensional forward simulation to simulate a noise-free TEM signal and superimposing artificial noise. The condition for terminating training is that the hybrid loss function value of the verification set does not decrease in five consecutive iterations, and the method is used for denoising after pre-processing of the measured data.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of data processing, and more particularly relates to a transient electromagnetic data noise suppression method. BACKGROUND

[0002] Transient electromagnetic method (TEM) is a time-domain electromagnetic exploration method based on electromagnetic induction principle, which is widely used in the exploration of mineral resources and hydrological resources. Transient electromagnetic method is to use step wave or other pulse current field source to emit primary field to the underground, measure the decay characteristics of the secondary induced electromagnetic field with time change at the moment of primary field power-off, and find out the conductivity and depth of the target geological body according to the signals received by the analysis instrument, so as to achieve the purpose of detecting underground geological body. Transient electromagnetic signal is a secondary field signal decaying with time, when working in strong noise interference environment such as mining area and factory area, the late signal received by the instrument is greatly affected by various natural and human interference noises, resulting in low resolution of deep geological body. Therefore, when using transient electromagnetic method, the existence of noise causes many adverse effects on the subsequent data processing and geological interpretation work. These interference signals may cover the useful signals needed, so it is necessary to effectively suppress noise and extract useful information of signals, so as to better use transient electromagnetic data for subsequent data processing and inversion interpretation.

[0003] The invention patent with application publication number CN117056677A proposes a transient electromagnetic signal denoising method based on sparrow algorithm improved variational mode decomposition, which uses sparrow optimization algorithm to globally optimize the penalty factor α and mode number K in variational mode decomposition, and obtains the optimal parameters [K, α]; the optimal parameter combination obtained after optimization is used to perform variational mode decomposition on the collected transient electromagnetic signal, to realize the denoising process of the original signal. This method can avoid the mode aliasing problem caused by manual selection of parameters, and improve the signal denoising effect.

[0004] The invention patent with the application publication number CN111650655A discloses a supervised transient electromagnetic signal denoising method based on non-negative matrix factorization. In the training stage, the pure signal is subjected to short-time Fourier transform and non-negative matrix factorization processing to obtain an atomic dictionary representing the characteristics of the signal. Then, in the denoising stage, the atomic dictionary and the denoising model are used to process the noisy signal to obtain a preliminary estimated transient electromagnetic signal. Finally, the above steps are repeated multiple times, and the late data of the preliminary estimated transient electromagnetic signal and the original early and middle period data of the noisy signal are respectively accumulated to obtain their respective arithmetic mean values. The two are spliced to estimate the final complete transient electromagnetic signal. The invention can effectively remove noise in the actual transient electromagnetic signal and improve the accuracy of transient electromagnetic signal inversion. However, the data needs to be trained, the steps are too many, the workload is large, and the performance of the processor itself is required to be high due to too much calculation data.

[0005] Early transient electromagnetic denoising methods include wavelet transform, Kalman filtering and singular value decomposition. These methods have their own shortcomings, such as lack of adaptability, unsatisfactory denoising effect, and complex operation steps. SUMMARY

[0006] The application mainly suppresses TEM noise based on a neural network, reduces the influence of Gaussian noise, power frequency interference, sharp pulse noise and other noises on the quality of TEM data, especially for late data contaminated by noise. Finally, the TEM denoising in a complex noise environment is solved, and the restoration of the late data contaminated by noise is successfully completed.

[0007] To achieve the above purpose, the application adopts the following technical solutions: the method comprises:

[0008] 1) An improved SPPF_CMA optimized CNN-LSTM neural network is constructed, which comprises a convolution layer, a pooling layer, an SPPF_CMA module, a bidirectional long short-term memory layer and a full connection layer in sequence;

[0009] 2) The improvement of the SPPF_CMA module comprises:

[0010] a. A CA attention mechanism is added after the output of the convolution layer to generate a direction perception feature map through horizontal and vertical global average pooling to adjust the channel weight of the input feature;

[0011] b. A multi-scale pooling branch is adopted, which comprises three maximum pooling layers with a pooling size of 5, one average pooling layer with a pooling size of 9 and one average pooling layer with a pooling size of 13, and the output features of each branch are spliced with the convolution output;

[0012] 3) Design a hybrid loss function, including a time-domain mean square error term, a frequency-domain amplitude spectrum error term and an L2 regularization term;

[0013] 4) Generate synthetic training data by simulating noise-free TEM signals through one-dimensional forward modeling and superimposing artificial noise.

[0014] In one scheme, the specific implementation of the CA attention mechanism includes:

[0015] For the input feature map X∈R^(H×W×C), global average pooling is performed in the horizontal direction using a (H, 1) pooling kernel and in the vertical direction using a (1, W) pooling kernel, respectively, to obtain feature encodings in the two directions; the horizontal and vertical feature encodings are spliced, and a 1×1 convolution and a Sigmoid activation function are used to generate an attention weight matrix; the weight matrix is multiplied with the original feature map channel by channel, and a weighted feature map X_att is output.

[0016] In one scheme, in the multi-scale pooling branch, the three maximum pooling layers use the same pooling size of 5, but different stride parameters, specifically, a stride of 2, a stride of 3 and a stride of 4; the pooling sizes of the two average pooling layers are 9 and 13 respectively, and the strides are both 5; the feature maps output by each pooling branch are zero-padded to keep the size consistent, and then spliced with the original convolution output along the channel dimension to form multi-scale fusion features.

[0017] In one scheme, in the generation of synthetic data, a central loop device model is used for one-dimensional forward modeling, and the response curves of different geoelectric models are calculated through a time-domain diffusion equation; the superimposed artificial noise includes three parts: Gaussian white noise, 50Hz power harmonic noise and random pulse noise, the signal-to-noise ratio is controlled within the range of -5dB to 10dB, and the total noise energy accounts for no less than 30% of the original signal amplitude.

[0018] In one scheme, the training termination condition is: when the hybrid loss function value of the validation set does not decrease for 5 consecutive iterations, and the current loss value is greater than 1.05 times the historical minimum loss value, save the historical optimal model parameters and terminate the training. The amount of validation set data accounts for 15%-20% of the total amount of training set.

[0019] In one scheme, the bidirectional long short-term memory layer includes two LSTM sub-layers in the forward and backward directions, the number of hidden units of each sub-layer is 64-128, and the time step is equal to the number of sampling points of the input TEM data; the forward LSTM processes the time series from early to late, the backward LSTM processes the reverse sequence from late to early, and the outputs of the two are spliced in the feature dimension and input into the fully connected layer.

[0020] In one scheme, the measured data needs to be preprocessed before denoising, including:

[0021] The original voltage signal is time channel normalized, so that the amplitude of each time channel is mapped to the interval of-1, 1; the late data (decay time > 10 ms) is logarithmically compressed; the multi-point data of the whole line are arranged in spatial order to form a two-dimensional input matrix, and the local space-time feature block is extracted through a sliding window and input into the network.

[0022] In one scheme, the convolution layer adopts a depth separable convolution structure, the convolution kernel size is 3*3, and the channel number expansion factor is set to 4; each convolution layer is followed by a batch normalization layer and a LeakyReLU activation function, wherein the negative slope parameter of LeakyReLU is set to 0.01.

[0023] The present application has the following beneficial effects:

[0024] 1. Good denoising effect on late contaminated data: the TEM denoising model of the neural network constructed by us has a good denoising effect on contaminated late TEM data, greatly improving the use efficiency of late TEM data and deepening the exploration depth.

[0025] 2. Strong universality and resistance to multiple noises: our method performs well in dealing with human noise that is difficult to eliminate by traditional methods. We target common human noises such as odd waves, charging and discharging triangular waves, power frequency interference, step waves and sharp pulses, and realize targeted removal. This innovation makes our method more adaptable and practical in complex actual environments.

[0026] 3. Intelligent denoising improves efficiency: avoids the need for manual selection of denoising parameters and other complex manual operations such as wavelet denoising, variable mode decomposition and singular value decomposition, and its simple and efficient denoising efficiency far exceeds other methods. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 The method flowchart of the present application;

[0028] Figure 2 The CNN-LSTM network model diagram based on the improved SPPF optimization. DETAILED DESCRIPTION

[0029] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the related drawings. The drawings show typical embodiments of the present application. However, the present application can be realized in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided to make the disclosure of the present application more thorough and comprehensive.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. In this detailed description of the application, references are made to the accompanying drawings, which illustrate specific embodiments in which the application can be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the application, and it is to be understood that other embodiments can be utilized and that changes can be made without departing from the scope of the present application. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present application is defined only by the appended claims.

[0031] As shown in Figure 1 , a transient electromagnetic data noise suppression method, the application constructs a CNN-LSTM neural network based on improved SPPF optimization. The neural network is used to realize the elimination of noise in TEM signal, especially the denoising of TEM late data. It includes:

[0032] S1, construct a CNN-LSTM neural network based on improved SPPF_CMA optimization, the network contains convolution layer, pooling layer, SPPF_CMA module, bidirectional long short-term memory layer and full connection layer in turn.

[0033] The constructed network is composed of convolution layer, pooling layer, SPPF_CMA, bidirectional long short-term memory layer and full connection layer, and the specific network model is shown in Figure 1 . We improve SPPF by adding CA attention mechanism and two average pooling (Avg Pool) parallel to maximum pooling (Max Pool), and obtain SPPF_CMA. The first change here is to add CA attention mechanism to adjust the weight of each channel, so that the network can pay more attention to important features and suppress irrelevant features. Then use multiple pooling branches (Max Pool and Avg Pool) to extract features from different scales, and the pool size of three Max Pool is: pool_size=5. The pool size of two different scales of Avg Pool is: pool_size=9, pool_size=13.

[0034] S2, the improvement of the SPPF_CMA module includes:

[0035] As shown in Figure 2 , first, a convolution layer Conv is used to extract local features:

[0036]

[0037] Where, X is the input data, W conv is the convolution kernel, C is the output of the convolution layer, k is the size of the convolution kernel, and bconv is a bias term of the convolution layer.

[0038] Then the data X after the enhanced features are obtained by the CA attention mechanism described above att

[0039]

[0040] Then, different scale features are extracted by multi-scale pooling operations (max pooling and average pooling):

[0041] Max pooling:

[0042]

[0043]

[0044]

[0045] Average pooling:

[0046]

[0047]

[0048] Finally, all the pooling branches and convolution outputs are spliced together, and the spliced features will be used as the input of the downstream processing.

[0049]

[0050] CA attention mechanism: the CA mechanism first performs global average pooling on the input feature map in the horizontal and vertical directions respectively to obtain two direction-aware feature maps. For the input feature map X ∈ R H×W×C , use the pooling kernel with size (H, 1) and (1, W) to encode each channel to obtain the output feature map of the c-th channel from the horizontal and vertical directions, with size h × w, as follows:

[0051]

[0052]

[0053] where x c represents the features of the input feature map in the c-th channel, with specified height and width. Then, by obtaining the horizontal direction feature map z h and the vertical direction feature map z w ​, using 1x1 convolution F1 and nonlinear activation function δ to aggregate their features, which can effectively extract the relationship between channels while capturing the region of interest, and finally get two intermediate feature maps in two directions, as follows:

[0054] Then, f is decomposed into two tensors f h and f w along the spatial dimension, and 1x1 convolution kernels F h and F w are used to convolve them respectively to make the number of channels of the two tensors the same as the input, and finally get the weight map through the sigmoid activation function σ, as follows:

[0055]

[0056]

[0057] Finally, the feature map is multiplied by the two weight maps to get the output y c (i,j) of the CA mechanism module, as follows:

[0058]

[0059] where , represent the weight of the cth channel.[1]

[0060] S3, design a hybrid loss function, including time domain mean square error term, frequency domain amplitude spectrum error term and L2 regularization term;

[0061] The loss function consists of three parts: time domain error, frequency domain error and regularization term, which can optimize the model in multiple aspects, balance the time and frequency domain characteristics, and help to avoid overfitting and improve generalization ability.

[0062] In the time domain, the difference between the output of the model and the real signal can capture the direct form and instantaneous characteristics of the signal. The frequency domain error measures the frequency characteristics of the signal, which is particularly suitable for capturing the periodicity or frequency components of the signal. When dealing with signals with noise, the frequency domain error helps to filter out the noise components and highlight the frequency characteristics of the signal. The time domain error and the frequency domain error evaluate the quality of the signal from different angles, and the combination of the two errors can help the model avoid falling into a local optimal solution in a certain dimension during the training process.

[0063] The addition of L2 norm effectively prevents the model from overfitting the training data, making the model learn a smoother function with better generalization ability.

[0064]

[0065] wherein, is the time domain error, is the frequency domain error, is the L2 norm regularization term, is the regularization coefficient.

[0066]

[0067] In the above formula, N is the number of data points, is the true signal, is the predicted signal.

[0068]

[0069] In the above formula, N is the number of data points, is the true signal, is the predicted signal, F represents the Fourier transform operation, represents the frequency component.

[0070]

[0071] In the above formula, M is the total number of weight parameters, represents the weight parameter matrix of the model.

[0072] S4, generate synthetic training data, simulate noise-free TEM signals by one-dimensional forward modeling and stack artificial noise.

[0073] Data set construction, training of neural network needs to construct a data set with enough number of samples. However, there is no ideal TEM data without natural noise, so we use the TEM data obtained by one-dimensional forward modeling and combine the artificially simulated noise to synthesize the data set to train the LSTM-Autoencoder neural network. We constructed 1000 different one-dimensional resistivity models, and obtained 1000 forward data by forward modeling. These forward data are transient data obtained by sampling at 100MHz sampling rate on 0-0.001s time, each data has 1000 sampling points. In addition, we artificially simulated different kinds and intensities of noise including sferic noise, power frequency noise and Gaussian noise. The mixed noise with different interference intensities is superimposed with the pure forward data, and finally 20000 noisy data samples are obtained, which are divided into training set and test set in the ratio of 4:1. That is, the training set has 16000 samples, and the test set has 4000 samples.

[0074] The data set construction method is: a plurality of different one-dimensional resistivity models are constructed, and forward calculation is performed to obtain forward calculation data corresponding to the plurality of one-dimensional resistivity models; artificial simulation of noise data of different types and different intensities is performed; wherein the noise data includes sferic noise, power frequency noise and Gaussian noise; a plurality of the forward calculation data and a plurality of the noise data are superimposed with different weights, and the superimposed plurality of data are used as a data set; the data set is divided into a training set and a test set according to a set proportion. The data set, the acquisition method thereof is:

[0075]

[0076] wherein, is the superimposed data set, is the first i forward calculation data, is the first i noise data; and are weight values of the forward calculation data and the noise data, respectively, for adjusting the proportion of the forward calculation data and the noise data in the superposition; N is the data amount of the forward calculation data.

[0077] and , the acquisition methods thereof are respectively:

[0078]

[0079] ;

[0080] is a preset target signal-to-noise ratio.

[0081] Network training mainly includes: data preprocessing, network parameter adjustment, network training and model verification.

[0082] (1) Data preprocessing:

[0083] For the input data set, we first perform normalization processing on it. The original data is normalized by maximum and minimum. As shown in the following formula, x is the original data, x mean is the average value of the logarithm of the original data, x max is the maximum value in the original data, x min is the minimum value in the original data, and x' is the normalized data:[2]

[0084]

[0085] After the data is normalized, considering that the training effect is relatively poor after one-dimensional data is input into the neural network. We will fold and reconstruct each data into two-dimensional data and then input it into the network. That is, the 9600x1 one-dimensional data is converted into 100x96x1 two-dimensional data.

[0086] (2) Parameter adjustment and model training:

[0087] After data preprocessing, the model is trained and the model parameters are adjusted. In addition to the fixed input and output dimensions, other parameters are usually adjusted according to the training and validation results to find the optimal value. After the parameter adjustment is completed, the model is trained. During the training process, the data is input in batches, the Loss of the predicted output and the theoretical output is calculated in each iteration, and the Adam optimizer is optimized until the Loss converges. Finally, the trained model is saved.

[0088] During the model training process, the training effect also needs to be verified to avoid overfitting. The previously divided validation set is used to verify the neural network, the data in the validation set is input into the recurrent neural network for prediction, and the predicted results are compared with the corresponding ideal output. If the validation error reaches convergence and then starts to increase constantly, it means that the network has reached the overfitting state and needs to stop training.

[0089] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0090] It should be understood that the above detailed description of the technical solutions of the present application by means of preferred embodiments is illustrative rather than limiting. A person of ordinary skill in the art can modify the technical solutions recorded in each embodiment or make equivalent substitutions for part of the technical features on the basis of the present application; and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A method for suppressing transient electromagnetic data noise, characterized in that: The method includes: A CNN-LSTM neural network based on improved SPPF_CMA optimization is constructed. The network sequentially includes convolutional layers, pooling layers, SPPF_CMA modules, bidirectional long short-term memory layers, and fully connected layers. The improvements to the SPPF_CMA module include: a. Add a CA attention mechanism after the output of the convolutional layer, generate orientation-aware feature maps through global average pooling in the horizontal and vertical directions, and adjust the channel weights of the input features; b. A multi-scale pooling branch is adopted, which includes three max pooling layers with a pooling size of 5, one average pooling layer with a pooling size of 9, and one average pooling layer with a pooling size of 13. The output features of each branch are concatenated with the convolution output. Design a hybrid loss function that includes a time-domain mean square error term, a frequency-domain amplitude spectrum error term, and an L2 regularization term; Generate synthetic training data, simulate noiseless TEM signals through one-dimensional forward modeling, and superimpose artificial noise; The specific implementation of the CA attention mechanism includes: For the input feature map X∈R H×W×C Global average pooling is performed using a (H,1) pooling kernel in the horizontal direction and a (1,W) pooling kernel in the vertical direction to obtain feature encodings in two directions. The horizontal and vertical feature encodings are concatenated, and an attention weight matrix is ​​generated by a 1×1 convolution and a sigmoid activation function. The weight matrix is ​​then multiplied channel-by-channel with the original feature map to output the weighted feature map X. att ; The bidirectional long short-term memory layer contains two LSTM sub-layers, forward and backward. Each sub-layer has 64-128 hidden units, and the time step is equal to the number of sampling points of the input TEM data. The forward LSTM processes the time series from early to late, and the backward LSTM processes the reverse sequence from late to early. The outputs of the two are concatenated along the feature dimension and then input into the fully connected layer. Before denoising, the measured data needs to be preprocessed, including: The original voltage signal is time-channel normalized so that the amplitude of each time channel is mapped to... 1,1 interval; logarithmic compression is performed on late-stage data with decay time >10ms; multi-point data of the entire measurement line are arranged in spatial order to form a two-dimensional input matrix, and local spatiotemporal feature blocks are extracted and input into the network through a sliding window; In the multi-scale pooling branch, the three max pooling layers use the same pooling size of 5, but are set with different stride parameters, specifically stride 2, stride 3 and stride 4; the pooling sizes of the two average pooling layers are 9 and 13, respectively, and the stride is 5 for both; the feature maps output by each pooling branch are zero-padding to keep the size consistent, and then spliced ​​with the original convolution output along the channel dimension to form multi-scale fused features; The expressions for the three max-pooling layers are as follows: ; ; ; The expressions for the two average pooling layers are as follows: ; ; In the generated synthetic training data, the one-dimensional forward modeling uses a central loop device model, and the response curves of different geoelectric models are calculated through the time-domain diffusion equation; the superimposed artificial noise includes three parts: Gaussian white noise, 50Hz power frequency harmonics, and random impulse noise, with the signal-to-noise ratio controlled within the range of -5dB to 10dB, and the total noise energy accounting for no less than 30% of the original signal amplitude; The training termination condition is as follows: when the mixed loss function value of the validation set does not decrease in 5 consecutive iterations, and the current loss value is greater than 1.05 times the historical minimum loss value, the historical best model parameters are saved and training is terminated; the validation set data volume accounts for 15%-20% of the total training set. The convolutional layer adopts a depth-separable convolutional structure with a kernel size of 3×3 and a channel expansion factor of 4. Each convolutional layer is followed by a batch normalization layer and a LeakyReLU activation function, where the negative slope parameter of LeakyReLU is set to 0.01.

Images