A transient electromagnetic inversion method based on deep neural network reparameterization

Abstract

The application discloses a transient electromagnetic inversion method based on deep neural network reparameterization regularization, belongs to the technical research field of geophysical electromagnetic data processing and analysis, and comprises the following steps: constructing an untrained deep neural network; inputting random hidden vectors conforming to the input layer structure of the deep neural network and boundary values of resistivity and stratum thickness required by transient electromagnetic inversion into the deep neural network to output a geological model containing a regularization effect; performing iterative updating on a transient electromagnetic inversion objective function by using a self-adaptive matrix estimation algorithm, judging whether a convergence condition is met, if the convergence condition is not met, taking the weight of each layer of the deep neural network as a variable needing to be updated, iteratively updating the weight of each layer, and making the deep neural network generate a new geological model containing the regularization effect. The application not only makes it easy to apply the regularization effect on the generated geological model in the physical constraint without external training data, but also has good anti-noise performance.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of geophysical electromagnetic data processing and analysis, and particularly relates to a transient electromagnetic inversion method based on deep neural network reparameterization regularization. BACKGROUND

[0002] The transient electromagnetic method is a geophysical exploration method based on electromagnetic induction principle. It uses a non-grounded loop or grounded line source to emit a pulse magnetic field to the underground, and uses a coil or grounded electrode to capture the decay characteristics of the secondary induction eddy current field in the underground medium after the pulse is turned off, so as to study the resistivity distribution of the underground medium and solve related geological problems. In recent years, the transient electromagnetic method has been widely used in urban underground engineering detection, unexploded ordnance detection, coal mine aquifer detection and other fields due to its small size, light weight and relatively low cost.

[0003] In transient electromagnetic method exploration, the decay characteristics of the secondary field are calculated using known geoelectric parameters, which is called forward. On the contrary, the geoelectric parameters are obtained from the detection data to reconstruct the underground structure (such as resistivity, rock thickness, etc.), which is called inversion. However, in practical applications, the limited amount of observation data for high-dimensional parameter inversion of real geological bodies will lead to a highly underdetermined problem, so the traditional transient electromagnetic inversion has a pathological problem, which will lead to unstable inversion results. At present, in order to ensure the stability of the inversion solution, researchers mostly choose to introduce an additional constraint term into the mismatch function of the inversion method through regularization means to avoid the pathological problem, but this constraint term needs to be finely adjusted by a regularization factor to ensure the stability of the inversion result, which will inevitably lead to the introduction of additional complexity of the inversion method and the increase of the calculation cost, and the noise resistance of the method is also difficult to effectively improve by such means.

[0004] Therefore, there is an urgent need to develop an efficient and stable transient electromagnetic inversion method that can avoid fine adjustment of the regularization factor while ensuring the stability of the inversion result and has good noise resistance. This will provide a new technical means for fast processing and reliable interpretation of transient electromagnetic inversion data, and significantly improve the application value of transient electromagnetic technology in inversion of field data. SUMMARY

[0005] The purpose of the present application is to provide a transient electromagnetic inversion method based on deep neural network reparameterization regularization, which proposes a new regularization method, so that the inversion method can ensure the stability of the inversion result without fine adjustment of the regularization factor, and has good noise resistance.

[0006] The object of the application is achieved by the technical solutions as follows: a transient electromagnetic inversion method based on deep neural network reparameterization regularization, the method comprising the following steps:

[0007] S1: constructing an untrained deep neural network, the component architecture of the deep neural network being a convolutional neural network;

[0008] S2: setting random latent vectors conforming to the input layer structure of the deep neural network and boundary values of resistivity and formation thickness required by transient electromagnetic inversion as inputs of the deep neural network;

[0009] S3: combining the above steps S1 and S2, the geological model containing the regularization effect as the output of the deep neural network constructed in step S1, the geological model containing resistivity values and formation thickness values;

[0010] S4: setting the transient electromagnetic forward equation required for inversion, and taking the geological model generated in step S3 as the input of the transient electromagnetic forward equation to obtain the corresponding transient electromagnetic forward response voltage value;

[0011] S5: jointly establishing a transient electromagnetic inversion objective function with the transient electromagnetic observation data collected on site and the transient electromagnetic forward response voltage value data obtained in step S4; it should be noted that the transient electromagnetic observation data in the application is the secondary field voltage amplitude;

[0012] S6: using the Adam algorithm to iteratively update the transient electromagnetic inversion objective function established in step S5, if the convergence condition is not met or the maximum number of iterations is not reached, taking the weight of each layer of the deep neural network as the variable to be updated, iteratively updating the weight of each layer of the network, so that the deep neural network generates a new set of geological models containing the regularization effect (reparameterization process), repeating the process of steps S4 and S5 until the convergence condition is met, and obtaining the final inversion model.

[0013] Further, in the step S1, the untrained deep neural network includes a neural network architecture of at least two convolutional layers and up-sampling layers. According to an embodiment of the present application, the deep neural network sequentially includes an input layer, a fully connected layer, a first up-sampling layer, a first convolutional layer, a first random deactivation layer, a second up-sampling layer, a second convolutional layer, a second random deactivation layer, a third up-sampling layer, a third convolutional layer, a third random deactivation layer, a fourth up-sampling layer, a fourth convolutional layer, a fourth random deactivation layer, a fifth convolutional layer, and an output layer. The untrained deep neural network is used to ensure that the inversion result is mainly driven by the current observation data and the physical law, and the structure of the deep neural network only provides a constraint framework. The core role of the deep neural network here is to serve as an intelligent and efficient model parameterization tool. It is not a direct replacement for the physical forward simulation, nor is it a complete end-to-end learning of the mapping from transient electromagnetic data to geological model, but is embedded in the traditional physics-driven inversion framework, fundamentally changing the representation and optimization process of the geological model.

[0014] In the steps S2 and S3, the random hidden vector is a one-dimensional continuous numerical vector generated by random numbers, which is subject to uniform distribution. The length of the random hidden vector is consistent with the length of the input vector of the input layer of the deep neural network. The random hidden vector can effectively avoid the subjective bias introduced by the preset initial geological model. The resistivity and the boundary value of the layer thickness are empirically set based on the prior geological information of the test site, and the geological model generated by the deep neural network is boundary-constrained. The deep neural network can only rely on the network layers, convolution kernel size, channel number, and connection method of its structure itself to make each neuron only connected with the local region of the input, so as to promote the generated geological model to have local smoothness and continuity in space, and the resistivity value of adjacent grids will not have a sharp mutation. Therefore, the structure itself bears the main regularization responsibility, so that the inversion can overcome the ill-conditioned and stably converge to a solution with good geological significance. The internal properties and inductive bias provided by the deep neural network are used to constrain the generated geological model, that is, the regularization effect.

[0015] The transient electromagnetic forward equation in the step S4 includes the following formula:

[0016] ;

[0017] wherein is the transient electromagnetic forward response voltage value; is the transmitting current size; is the magnetic permeability of free space; is the apparent resistivity; is the time; is the transmitting coil radius; is the time scale factor, .

[0018] Further, in the step S5, the transient electromagnetic inversion objective function includes:

[0019] The transient electromagnetic inversion objective function is defined as , which is used to measure the fitting difference between the voltage value data of the transient electromagnetic forward response and the transient electromagnetic observation data , and is specifically expressed as:

[0020] ;

[0021] Wherein is the transient electromagnetic data; is the geological model generated by the deep neural network; is the random hidden vector; is the weight of the network. The root mean square error RMSE is set as the specific manifestation of the second order two norm:

[0022] ;

[0023] Wherein is the number of interpolation time channels contained in each set of data after preprocessing; is the weight matrix related to the data error, which is used to measure the influence of the observation data on the inversion result under the interference of noise, and is expressed by the standard deviation , the greater the influence of noise, the smaller the weight proportion, that is is expressed as:

[0024] ;

[0025] Wherein represents the diagonal matrix operator.

[0026] In the step S6, the weight updating process includes: calculating the gradient of the transient electromagnetic inversion objective function on the weight, and dynamically adjusting the learning rate based on the adaptive matrix estimation algorithm, and updating the weight value combined with the first order moment estimation and the second order moment estimation of the gradient.

[0027] The final inversion model is the geological model output by the deep neural network when the convergence condition is met.

[0028] Compared with the prior art, the application has the beneficial effects that: the application proposes a transient electromagnetic inversion method based on deep neural network reparameterization regularization aiming at the problem that the inversion method complexity and the calculation cost are increased due to the additional introduction and fine adjustment of the regularization term in the inversion of the transient electromagnetic observation data, and the noise resistance performance of the traditional inversion method is difficult to effectively improve. The application uses the deep neural network to generate a geological model to replace the artificial setting of the geological model in the traditional inversion method, which not only makes the network easily impose the regularization effect on the generated geological model in the physical constraint without external training data, and the inductive bias captured by the deep neural network is an effective image prior, so that the inversion method of the application also has good performance in the noise resistance task. The transient electromagnetic inversion method is provided with an effective scheme which can avoid fine adjustment of the regularization factor while ensuring the stability of the inversion result and having good noise resistance performance. This has important application prospects in the engineering actual transient electromagnetic exploration. BRIEF DESCRIPTION OF DRAWINGS

[0029] The accompanying drawings are included to provide a further understanding of the application, and are incorporated in and constitute a part of this application, illustrate embodiments of the application and explain the principles of the application, and are not intended to limit the application. In the drawings:

[0030] Figure 1 The flow chart of the transient electromagnetic inversion method based on deep neural network reparameterization regularization is shown in the figure.

[0031] Figure 2 The deep neural network structure diagram in the embodiment of the application is shown in the figure.

[0032] Figure 3 The transient electromagnetic observation data provided by the embodiment of the application is shown in the figure.

[0033] Figure 4 The inversion two-dimensional image result of the inversion geological model after the transient electromagnetic observation data in the embodiment of the application is used in the method of the application is shown in the figure. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and advantages of the application clearer, the application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

[0035] In combination with Figure 1 The transient electromagnetic inversion method based on deep neural network reparameterization regularization includes the following steps:

[0036] S1: constructing an untrained deep neural network, the component architecture of the deep neural network is a convolutional neural network;

[0037] S2: setting the random latent vector conforming to the input layer structure of the deep neural network and the boundary values of the resistivity and the formation thickness required by the transient electromagnetic inversion as the input of the deep neural network; the length of the random latent vector is consistent with the length of the input vector of the input layer of the deep neural network;

[0038] S3: combining the above steps S1 and S2, the geological model (including resistivity value and formation thickness value) containing regularization effect as the output of the deep neural network in step S1;

[0039] S4: setting the transient electromagnetic forward equation required for inversion, and taking the geological model generated in step S3 as the input of the transient electromagnetic forward equation to obtain the corresponding transient electromagnetic forward response voltage value;

[0040] S5: combining the transient electromagnetic observation data collected on site and the transient electromagnetic forward response voltage value data obtained in step S4 to establish a transient electromagnetic inversion objective function;

[0041] S6: using the adaptive moment estimation algorithm to iteratively update the transient electromagnetic inversion objective function established in step S5, if the convergence condition is not met or the maximum iteration number is not reached, taking the weight of each layer of the deep neural network as the variable to be updated, iteratively updating the weight of each layer of the network, so that the deep neural network generates a new set of geological models containing regularization effect (reparameterization process), repeating the process of steps S4 and S5 until the convergence condition is met, and obtaining the final inversion model.

[0042] Further, in the above step S1, the untrained deep neural network, such as Figure 2As shown, the deep neural network sequentially comprises an input layer, a fully connected layer, a first upsampling layer, a first convolutional layer, a first random dropout layer, a second upsampling layer, a second convolutional layer, a second random dropout layer, a third upsampling layer, a third convolutional layer, a third random dropout layer, a fourth upsampling layer, a fourth convolutional layer, a fourth random dropout layer, a fifth convolutional layer, and an output layer. The input data is a one-dimensional sequence signal. The input data first enters the fully connected layer. The fully connected layer can comprehensively integrate and transform the input data. Then, the features are processed using 4x1 convolution kernels of different channel numbers (such as 64 channels, 32 channels, 16 channels, 8 channels, and a single channel) to extract local features. In branches that need to restore the resolution of the feature map, upsampling operations are performed to increase the size of the feature map. During the training stage, some neurons are randomly deactivated (Dropout) to prevent overfitting and enhance the generalization ability of the model. Finally, the output result of the network is obtained, and the entire process is completed. The size of the first convolutional layer convolution kernel is 4x1, and the channel number is 64, that is, the first convolutional layer uses 64 convolution kernels, each kernel size is 4x1 (the kernel size of 4x1 represents that the sliding window of the one-dimensional convolution kernel in the length direction is 4 units), which is used to extract the local features of the input sequence, and 64 feature maps are output. The size of the second convolutional layer convolution kernel is 4x1, and the channel number is 32, that is, the number of convolution kernels of the second convolutional layer is reduced to 32, which further abstracts the features and reduces the channel number to reduce the calculation amount or gradually focus on important features, and 32 feature maps are output. The size of the third convolutional layer convolution kernel is 4x1, and the channel number is 16, that is, the number of convolution kernels of the third convolutional layer is reduced to 16, which further abstracts the features and reduces the channel number to reduce the calculation amount or gradually focus on important features. The size of the fourth convolutional layer convolution kernel is 4x1, and the channel number is 8, that is, the channel number of the fourth convolutional layer is continuously reduced to 8, and the convolution kernel size remains unchanged. At this time, the feature map has been compressed for many times and contains higher-level semantic information. The size of the fifth convolutional layer convolution kernel is 4x1, and the channel number is 1, which is used to generate the target sequence. The reason for using an untrained deep neural network is to ensure that the inversion result is mainly driven by the current observation data and physical laws. The structure of the deep neural network only provides a constraint framework. The core role of the deep neural network here is to serve as an intelligent and efficient model parameterization tool. It is not a direct replacement for the physical forward modeling simulator, nor is it a complete end-to-end learning of the mapping from transient electromagnetic data to geological model. Instead, it is embedded into the traditional physics-driven inversion framework, fundamentally changing the representation and optimization process of the geological model.

[0043] In the step S2 and the step S3, the random hidden vector is a one-dimensional vector structure composed of continuous numerical elements; the length of the one-dimensional vector is equal to the length of the input vector of the input layer of the deep neural network; each element in the one-dimensional vector is generated by a random number generator and obeys a uniform distribution; the random hidden vector effectively avoids subjective bias introduced by the preset initial geological model. The resistivity and the boundary value of the stratum thickness are empirically set based on the prior information of the test site, and the deep neural network generates a boundary-constrained geological model. The deep neural network only relies on the network layers, the convolution kernel size, the channel number, the connection mode and the like of the structure itself, so that each neuron is only connected with the local area of the input, and the generated geological model has local smoothness and continuity in space, and the resistivity values of adjacent grids will not change dramatically. Therefore, the structure itself undertakes the main regularization responsibility, so that the inversion can overcome the ill-conditioned and stably converge to a solution with good geological significance, and the internal properties and inductive bias are provided to constrain the generated model, that is, the regularization effect.

[0044] The transient electromagnetic forward equation in the step S4 includes the following formula:

[0045] ;

[0046] wherein is a transient electromagnetic forward response voltage value; is a transmitting current size; is a magnetic permeability of free space, and the value is ; is an apparent resistivity; is time; is a transmitting coil radius; is a time scale factor, .

[0047] Further, in the step S5, the transient electromagnetic inversion objective function includes:

[0048] The transient electromagnetic inversion objective function is defined as , which is used to measure the fitting difference between the voltage value data of the transient electromagnetic forward response and the transient electromagnetic observation data , and is specifically expressed as:

[0049] ;

[0050] wherein is transient electromagnetic data; is a geological model generated by the deep neural network; is a random hidden vector; is a weight of the network. The root mean square error RMSE is set as a specific manifestation of the second-order two norm:

[0051] ;

[0052] where is the number of interpolation time traces contained in each group of pre-processed data; is the weight matrix related to data error, which is used to measure the influence of the observed data on the inversion result under the interference of noise, and is expressed by the standard deviation , the greater the influence of noise, the smaller the weight proportion, that is is expressed as:

[0053] ;

[0054] where represents the diagonal matrix operator.

[0055] For the iterative update of the transient electromagnetic inversion objective function , due to the fixed learning rate, it often performs poorly in different parameter dimensions, and is prone to slow convergence or unstable oscillation, so the adaptive moment estimation algorithm is selected as the parameter update optimization algorithm, which is an improved algorithm combining the momentum method and the RMSProp idea (RMSprop is a widely used adaptive learning rate optimization algorithm in deep learning, and the core principle is to dynamically adjust the learning rate of each parameter through the exponential weighted average of the gradient square). The specific theoretical basis is: in each iteration update, not only rely on the current gradient information, but also estimate the first and second order statistics of the gradient by exponential weighting, and through bias correction and normalization, dynamically adjust the learning rate. Let be the gradient of the transient electromagnetic inversion objective function at the th iteration, and

[0056] ;

[0057] where is the gradient value of the th iteration; is the operator for calculating the gradient of the deep neural network generated geological model , is the transient electromagnetic inversion objective function at the deep neural network generated geological model , is the deep neural network generated geological model generated at the th iteration.

[0058] In order to smooth the gradient update, the adaptive moment estimation algorithm first performs exponential weighted averaging on the historical information of the gradient, thereby forming the first moment estimation:

[0059] ;

[0060] where, controls the decay rate of the first moment. Usually we take , which means the current gradient has 10% weight in the average while the history gradient has 90% weight. This operation can make the optimization path smoother when the surface is rough. . and are the first moment estimates at the th and th iteration, respectively; is the gradient value at the th iteration. Besides considering the mean of the gradient, the adaptive moment estimation algorithm also performs exponential weighted average on the square of the gradient to obtain the second moment estimate:

[0061] ;

[0062] where, is usually taken as 0.99, which means a higher weight is given to the history gradient square. and are the second moment estimates at the th and th iteration, respectively, is the gradient value at the th iteration . This procedure is similar to the RMSProp method, which is used to characterize the scale variation of the gradient. When the gradient fluctuates greatly, the second moment will become larger, thus suppressing the learning rate in the subsequent update to avoid numerical instability. In addition, since , will cause bias in the early estimates, the adaptive moment estimation introduces a correction factor to eliminate this effect:

[0063] ;

[0064] The corrected first moment estimate and the corrected second moment estimate are closer to the true gradient distribution, is the th power of (for quantifying the bias caused by the excessive weight of history information in the initial stage), is the th power of , and finally, combined with the corrected moment estimates, the parameter update formula for each step is obtained:

[0065] ;

[0066] in, The global learning rate, To prevent small constants with zero denominators . and They are the first Second and third The geological model generated by the deep neural network after the next iteration, the adaptive moment estimation algorithm can independently adjust the learning rate for each parameter based on the first and second moment estimates of the gradient: the learning rate decreases when the gradient is large (stable update), and the learning rate increases when the gradient is small (accelerated convergence).

[0067] This example includes a schematic diagram of transient electromagnetic data from 100 measurement points, as shown below. Figure 3 As shown, Figure 3 For clarity, only a portion of the measuring points are shown. Figure 3 In the diagram, the z-axis represents the amplitude of the transient electromagnetic observation data, the x-axis represents time, and the y-axis represents the number of measurement points for the transient electromagnetic observation data; the transient electromagnetic observation data refers to the amplitude of the secondary field voltage. The measurement point spacing is 1 meter, and each measurement point records 22 time channels within the time range of 1.04 μs to 0.461 ms. The inversion two-dimensional image results using the method of this invention are shown below. Figure 4 As shown, Figure 4 The horizontal axis represents the number of measurement points for transient electromagnetic observation data, and the vertical axis represents the formation depth value, which is the sum of the thicknesses of all formations. Figure 4 The data clearly shows the geological structure of the area where the transient electromagnetic observation data was collected, and has good geological continuity. It also does not produce unreasonable results due to noise interference. The inversion method of this invention can ensure the stability of the inversion results while avoiding fine adjustment of the regularization factor, and has good noise resistance, providing a new and effective solution for transient electromagnetic inversion.

[0068] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A transient electromagnetic inversion method based on deep neural network reparameter regularization, characterized in that, include: S1: Construct an untrained deep neural network, wherein the network architecture of the deep neural network is a convolutional neural network; S2: Input the random hidden vector that conforms to the input layer structure of the deep neural network, as well as the boundary values ​​of resistivity and formation thickness required for transient electromagnetic inversion, into the deep neural network. S3: The random latent vector and the boundary value are processed by the deep neural network to output a geological model with regularization effect; S4: Set the transient electromagnetic forward modeling equations required for the inversion, and use the generated geological model as the input of the transient electromagnetic forward modeling equations to obtain the corresponding transient electromagnetic forward modeling response voltage values; S5: The residual norm between the transient electromagnetic observation data collected on-site and the transient electromagnetic forward modeling response voltage value data is used as a data fitting term to form the transient electromagnetic inversion objective function. S6: Use the adaptive moment estimation algorithm to iteratively update the transient electromagnetic inversion objective function, determine whether the convergence condition is met, if not, use the weights of each layer of the deep neural network as variables to be updated, iteratively update the weights of each layer of the network, so that the deep neural network generates a new geological model with regularization effect, and repeat steps S4 and S5 until the convergence condition is met, and output the final inversion model.

2. The transient electromagnetic inversion method based on deep neural network reparameter regularization according to claim 1, characterized in that, The deep neural network comprises, in sequence, an input layer, a fully connected layer, a first upsampling layer, a first convolutional layer, a first random deactivation layer, a second upsampling layer, a second convolutional layer, a second random deactivation layer, a third upsampling layer, a third convolutional layer, a third random deactivation layer, a fourth upsampling layer, a fourth convolutional layer, a fourth random deactivation layer, a fifth convolutional layer, and an output layer.

3. The transient electromagnetic inversion method based on deep neural network reparameter regularization according to claim 1, characterized in that, The random latent vector is a one-dimensional vector structure composed of continuous numerical elements; the length of the one-dimensional vector is equal to the length of the input vector of the input layer of the deep neural network; each element in the one-dimensional vector is generated by a random number generator and follows a uniform distribution.

4. The transient electromagnetic inversion method based on deep neural network reparameter regularization according to claim 1, characterized in that, In step S4, the transient electromagnetic forward equation is: ； in This represents the voltage value of the transient electromagnetic forward response. As a time scale factor, ; The magnitude of the transmitting current; It is the permeability of free space; Apparent resistivity; For time; R is the radius of the transmitting coil.

5. The transient electromagnetic inversion method based on deep neural network reparameter regularization according to claim 1, characterized in that, The objective function for transient electromagnetic inversion is expressed as follows: ； in The objective function for transient electromagnetic inversion; Transient electromagnetic data; Geological models generated by deep neural networks; It is a random latent vector; It is the network weight; The voltage values ​​are for the transient electromagnetic forward response. The data are transient electromagnetic observation data; the data fitting term is the root mean square error in the form of second-order L2 norm: ； in This represents the number of interpolation time channels included in each set of data after preprocessing. The weight matrix related to data error is expressed as follows: ; Represents diagonal matrix operators; The standard deviation is denoted as .

6. The transient electromagnetic inversion method based on deep neural network reparameter regularization according to claim 1, characterized in that, In step S6, the convergence condition includes the objective function value decreasing to a preset threshold or reaching the set maximum number of iterations.

7. The transient electromagnetic inversion method based on deep neural network reparameter regularization according to claim 1, characterized in that, In step S6, the weight update process includes: calculating the gradient of the transient electromagnetic inversion objective function with respect to the weights, dynamically adjusting the learning rate based on the adaptive moment estimation algorithm, and updating the weight values ​​by combining the first-order moment estimation and the second-order moment estimation of the gradient.

8. The transient electromagnetic inversion method based on deep neural network reparameter regularization according to claim 1, characterized in that, The final inversion model is the geological model output by the deep neural network when the convergence condition is met.

Images