Three-dimensional fault detection self-supervision pre-training method based on multistage mask auto-encoder

Abstract

The invention provides a three-dimensional fault detection self-supervision pre-training method based on a multistage mask auto-encoder, and aims to improve fault detection performance in three-dimensional seismic data analysis through self-supervision learning. Data preprocessing and view generation are carried out on three-dimensional seismic data, small-scale, medium-scale and large-scale sub-volume views are generated, and local features, local and global features and global structure information are concerned respectively. A mask operation is adopted to simulate data missing, a mask view is generated, and a model is forced to reconstruct a missing part through known part information, so that robust feature representation is learned. A mask view is coded and reconstructed through an auto-encoder, the feature extraction capacity is optimized, and then the adaptability of the model to complex seismic data is improved. And finally, through weighted combination multi-scale reconstruction loss and global multi-scale consistency loss calculation, a training process is optimized, so that the model shows excellent performance in a downstream task.

Description

Technical Field

[0001] The present invention belongs to the field of geophysics and artificial intelligence applications, and specifically relates to a three-dimensional fault detection self-supervised pre-training method based on a multi-level masked autoencoder. Background Art

[0002] Three-dimensional fault detection plays a vital role in seismic data analysis and is widely used in fields such as oil exploration, mineral resource development, and earthquake early warning. The goal of this task is to identify underground fault structures by analyzing seismic wave reflection data, thereby helping to understand the structure of underground rock formations, predict seismic activity, and rationally exploit resources. However, the complex structure of three-dimensional seismic data contains a large amount of noise, which makes fault extraction particularly difficult. Furthermore, traditional fault detection methods rely heavily on manually annotated data, but the difficulty and cost of obtaining annotated data limit their application in large-scale seismic data analysis.

[0003] To address the above challenges, deep learning methods have been introduced into seismic data analysis, and significant progress has been made in image processing and feature extraction. However, most current deep learning methods rely on large amounts of labeled data for training, and these methods often struggle to fully realize their potential when dealing with scarce or complex labeled data. Self-supervised learning, as an emerging machine learning method, can extract effective features from unlabeled data, thereby greatly reducing reliance on manually labeled data. This method uses self-generated supervisory signals for training, enabling the model to automatically learn useful information from large amounts of unlabeled data, reducing reliance on labeled data and improving the model's adaptability in seismic data analysis. It demonstrates strong potential, especially in scenarios where labeled data is scarce. Summary of the Invention

[0004] In view of the above, the purpose of the present invention is to provide a self-supervised pre-training method for three-dimensional fault detection based on a multi-level masked autoencoder, which aims to improve the three-dimensional fault detection performance in downstream tasks by constructing effective pre-training tasks. By pre-training unlabeled data with a masked autoencoder, the model can automatically learn effective feature representations when processing three-dimensional seismic data, providing high-quality feature representations for downstream tasks. This pre-training method not only optimizes the performance of the three-dimensional fault detection model, but also improves the accuracy and robustness of the model in complex seismic data scenarios. Ultimately, the present invention significantly enhances the model's adaptability to different data scenarios, especially when labeled data is scarce, demonstrating its strong practical application value.

[0005] To achieve the above object, the technical solution of the present invention mainly includes the following steps:

[0006] A. Data preprocessing and view generation:

[0007] This method randomly selects data blocks from 3D seismic data and then crops, scales, and downsamples them to generate sub-volume views at three different scales. The small-scale view focuses on capturing fine-grained local features, the medium-scale view balances local and global features, and the large-scale view focuses on global structural information. By using these different scales, the model can comprehensively consider multiple feature information during training, thereby improving its ability to process complex seismic data.

[0008] B. Mask operation and data construction:

[0009] To simulate missing data and enhance the effectiveness of self-supervised learning, the present invention performs a masking operation on each generated view. Specifically, the masking operation randomly blocks portions of the view to generate masked views, which serve as input data for subsequent feature learning. The masking operation simulates data loss, forcing the model to reconstruct the missing parts using known partial information, thereby learning more robust and effective feature representations.

[0010] C. Feature encoding and reconstruction:

[0011] Each mask view is encoded using the encoder portion of the autoencoder to generate feature representations containing local, mid-scale, and global features. These features are then reconstructed using the decoder portion. By minimizing reconstruction error, the model optimizes features at different scales, thereby enhancing its understanding and representation of seismic data.

[0012] D. Multi-scale reconstruction loss calculation:

[0013] During the reconstruction process, the present invention calculates the reconstruction loss for each view to obtain the reconstruction loss for small-scale views, medium-scale views, and global views. To fully utilize these multi-scale features, the present invention uses a weighted combination of these loss terms, allowing the model to comprehensively consider global and local information during training, thereby improving the effectiveness and robustness of feature representation.

[0014] E. Global multi-scale consistency loss calculation:

[0015] To further enhance the robustness of the model, this paper introduces a global multi-scale consistency loss. During training, by ensuring consistency between the unmasked global view features and the masked multi-scale view features, the model is able to maintain feature alignment across scales. This approach, through the calculation of a contrastive loss, strengthens the synergy between global and local features, improving the accuracy and stability of the model.

[0016] F. Loss function construction:

[0017] Ultimately, the training goal is to minimize the weighted sum of the multi-scale reconstruction loss and the global multi-scale consistency loss, thereby optimizing training results. By properly adjusting the weighting coefficients of the loss terms, we can effectively balance the learning process of features at different scales, improving the model's performance in both feature extraction and reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is the model structure diagram of the present invention DETAILED DESCRIPTION

[0019] The following combination Figure 1 The present invention is described in further detail:

[0020] A. Data preprocessing and view generation:

[0021] First, the method of the present invention randomly selects data blocks from 3D seismic data and generates three sub-volume views of different scales through cropping, scaling and downsampling. The specific steps are as follows:

[0022] (1) Select a 3D data block: Assume that the original 3D data block Where H, W, and D are the height, width, and depth of the data respectively.

[0023] (2) Small-scale view generation: downsample the original data block to generate a small-scale view in This view provides high-resolution local features and is suitable for capturing fine-grained information.

[0024] (3) Medium-scale view generation: The original data block is moderately cropped and downsampled to generate a medium-scale view. in This view balances global and local features, retaining details while containing certain global structural information.

[0025] (4) Large-scale view generation: Downsample the original data block to generate a large-scale view in This view focuses on global structural information.

[0026] B. Mask operation and data construction:

[0027] In order to enhance the self-supervised learning effect, the present invention uses masking operation to simulate data missing. The process of masking operation is as follows:

[0028] (1) Application of mask operation: for each scale view v generated i(i=l,m,g) performs mask operation, which randomly selects a part of the area to block and generates mask views v of different scales i M (i=l,m,g):

[0029]

[0030] Here, mask(·) is a masking operation. The purpose of the masking operation is to allow the model to learn with only partial information, so that the model can reconstruct the missing information. r is the masking rate, which represents the proportion of the area occluded during the masking operation. When r = 0.8, approximately 80% of the data area is occluded, and the remaining 20% of the data area is visible for learning.

[0031] C. Feature encoding and reconstruction:

[0032] During the feature learning process, the mask view is encoded and reconstructed through the autoencoder. The specific process is as follows:

[0033] (1) Encoder part: mask views ν of different scales i M Use encoder f(·) to encode and get the corresponding feature representation z i (i=l,m,g). Assume the encoder output z i It is composed of the feature representation extracted from the visible part and the category label, and the formula is expressed as:

[0034] z i =f(ν i M )(i=l,m,g)

[0035]

[0036] in, is the category tag, is the feature representation extracted from the visible part, and for the subsequent global multi-scale consistency loss calculation, the global unmasked view v g The encoder f(·) is also used to encode the output z c , the formula is:

[0037] z c =f(v g )

[0038]

[0039] in, is the category tag, is the feature representation extracted from the visible part.

[0040] (2) Decoder: For each encoded z of different scales i , reconstructed by the decoder g(·), the goal of reconstruction is to restore the masked part, different scales z i The output is The formula is:

[0041]

[0042] where g(·) is the decoder, is the predicted output of the model, and the goal is to compare it with the original data v i As close as possible.

[0043] D. Multi-scale reconstruction loss calculation

[0044] During the reconstruction process of the model, for each view (small-scale view, medium-scale view and large-scale view), the reconstruction loss of each view is calculated by the mean square error (MSE), and these loss terms are combined for optimization. By weighted combination of reconstruction losses of different scales, the model can comprehensively consider feature information of different scales during training and enhance the processing ability of complex seismic data. Specifically, the reconstruction loss of small-scale view, medium-scale view and large-scale view are defined as and The formula is as follows:

[0045]

[0046] in, is the model’s prediction for the jth sample in the small-scale, medium-scale, and large-scale views, and ν l j ,ν m j ,ν g j represents the actual unmasked data value of the jth sample in the small-scale, medium-scale, and large-scale views.

[0047] E. Global Multi-Scale Consistency Loss Calculation

[0048] To further enhance the robustness of the model, this paper introduces a global multi-scale consistency learning strategy. During training, the model ensures consistency between unmasked global view features and masked local and mid-scale view features. By using a contrastive learning loss, the model maintains feature alignment across scales, improving its adaptability to seismic data.

[0049] First, the global complete view category is marked and category labels from multi-scale mask views and Projected into a shared space. To achieve this goal, we introduce the projection layer P f (·), connected to the encoder f(·). These projection layers are implemented through fully connected layers, and the representation after projection is defined as follows:

[0050]

[0051] Then the global-global consistency loss Global-mesoscale consistency loss and global-small-scale consistency loss The calculation formula is as follows:

[0052]

[0053] Among them, H(·) represents the contrast loss between two feature representations (using cosine similarity), and Γ(·) is the normalization operation of the features. c j ,e g j ,e m j ,e l j is the global full view category tag and category labels from multi-scale mask views and In this way, the model can effectively maintain the consistency of features between different scales and improve its ability to identify fault structures in seismic data.

[0054] F. Loss Function Construction

[0055] Finally, the training objective is to minimize the weighted sum of the multi-scale reconstruction loss and the global multi-scale consistency loss:

[0056]

[0057] The final loss function consists of multiple parts, including small-scale reconstruction loss Mesoscale reconstruction loss Large-scale reconstruction loss Global-global consistency loss Global-mesoscale consistency loss and global-small-scale consistency loss These loss terms are balanced by weighting coefficients α1, α2, α3, α4, and α5 to ensure that the model fully considers the learning of multi-scale features during training.

Claims

1. A self-supervised pre-training method for 3D fault detection based on a multi-stage masked autoencoder, characterized in that: The following steps are involved: A. Data preprocessing and view generation: By randomly selecting data blocks from 3D seismic data and then cropping, scaling, and downsampling them, three sub-volume views of different scales are generated. The small-scale view focuses on capturing fine-grained local features, the medium-scale view considers both local and global features, and the large-scale view focuses on global structural information, thus providing a multi-scale feature foundation for subsequent steps. B. Mask operation and data construction: Based on the above multi-scale views, a masking operation is performed on each view by randomly blocking some areas to generate a masked view. This method can simulate data loss and force the model to reconstruct the missing parts using known information, thus forming an effective input for the feature encoding step. C. Feature encoding and reconstruction: Based on the constructed mask views, an autoencoder is used to encode each mask view to obtain feature representations. These features are then reconstructed through a decoder. This optimization process, which minimizes reconstruction error, further improves the model's ability to understand and represent seismic data and lays the foundation for the next step of multi-scale loss calculation. D. Multi-scale reconstruction loss calculation: After completing the autoencoder reconstruction optimization, the reconstruction loss is calculated for small-scale, medium-scale, and large-scale views respectively, and the weighted combination of the losses at each scale ensures that the model can integrate global and local information; E. Global multi-scale consistency loss calculation: Under the premise of optimizing the multi-scale reconstruction loss, a global multi-scale consistency learning method is introduced to ensure that the unmasked global view features are consistent with the masked multi-scale view features, and the synergy between global and local features is strengthened through contrastive learning loss. F. Loss function construction: Finally, based on the organic connection of each step, the multi-scale reconstruction loss and the global multi-scale consistency loss are summed according to the weights to form the final training goal. By reasonably adjusting the weights, the learning process of features at different scales is effectively balanced, thereby significantly improving the overall performance of the model in the feature extraction and reconstruction process.

Images